Background information

Last modified: 15 Sep 2026

This section provides background information on some of the primary legal frameworks that support the use of data in Longitudinal Population Studies (LPS). UK General Data Protection Regulation (GDPR) applies to the processing of personal data and is covered extensively elsewhere. This page therefore focuses on common law duty of confidentiality and alternative legal basis, such as Section 251 support, and participants rights and expectations, and how this underpins ethical principles in LPS research.

To caveat, this guidance reflects our understanding and does not constitute legal opinion. If in doubt, you should seek your own legal counsel.

UK LPS are often called the ‘consented studies’, but the role of consent in these studies is complicated. All LPS participants are essentially volunteers who have chosen to join the LPS, and all have the right to withdraw whenever they want, although important caveats exist. In this sense all participation is ‘consented’ in that it is based on an active and informed relationship. But this implied consent may not cover all proposed activities the study undertakes or be defensible if the participant does not understand LPS may also seek consent for specific aspects of activity in the LPS, such as using genetic information or linking to health records. However, as LPS can run over decades, it is hard to keep consent up to date and reflecting changes in technology, best practice and new ways of working, such as UK LLC. This means for LPS ‘consent’ is difficult to manage.

From a legal perspective consent is:

• Almost always NOT the basis by which LPS comply with UK GDPR and data protection law - scientific research is a permitted purpose in its own right; and,

• OFTEN the basis by which the Common Law Duty of Confidentiality is met. In common law, information given under an expectation of privacy (e.g. information from a patient to their doctor), should stay private unless permission to share it is in place.

A BRIEF INSIGHT INTO THE LAW: the law is notoriously complex. This includes the law that controls the use of data in LPS. At a very high level, the following points are critical to understand:

• Statute is legislation which are written and passed by the UK Parliament or devolved assemblies. This provides codified law. The Data Protection Act (DPA) is an example;

• Some law is not consistent across the UK, while English & Welsh law are broadly aligned within our context, Scottish law can be different in important ways;

• “Common Law” is the collection of precedents set in court cases which are then applied more widely. This includes very important principles, such as the Common Law Duty of Confidentiality. Common Law can be difficult to interpret as many cases can issue decisions which impact on principles and these are typically made in different contexts – the transferability of these judgements to our context therefore includes an element of subjectivity;

• Regulators are tasked with setting the codes of practice which must be followed to apply statute and common law in practice. For example, the Information Commissioner’s Office is the regulator for Data Protection law and the Health Research Authority (HRA) is the regulator for using Patient Information in research;

• Legislation, case law and regulations interact with fundamental human rights (e.g. EU and United Nation codified rights) and ethical principles (e.g. Declaration of Helsinki for health research). These interactions can be highly complicated and can introduce tensions between individual rights (of the participant) and broader societal benefits and rights (the public goods longitudinal research is designed to produce).

We note the guidance and codes of practice provided by the regulators are often broad and generic, and their application to LPS context can be subjective. Much of the guidance in this resource hub is designed to support LPS to apply these requirements appropriately and to avoid the potential misapplication of generic requirements to our specific context.

UK LLC have worked with participant and public volunteers to develop a series of infographics designed to explain this to participants.

UK General Data Protection Regulations (GDPR)

The UK GDPR applies to the processing of Personal Data by LPS. Most LPS process identifiable data given that they hold participant contact databases and administer processes which require the use of identifiers (mailing information and data collection exercises, conducting fieldwork or study assessments, linking to participant records including via UK LLC). LPS therefore manage Personal Data (broadly, the legal term for identifiable data) and therefore UK General Data Protection Regulations (UK GDPR) and Common Law Duty of Confidentiality apply.

There is clear guidance and a broad consensus that LPS should not be using consent as the UK GDPR/DPA legal basis for processing participant Personal Data. This is because the expectation for specific consent with UK GDPR/DPA is typically not possible within longitudinal research. Rather, LPS should make use of alternative legal basis within the regulations:

1. Performance of a task carried out in the public interest (Article 6(1)(e) in the GDPR), and, where sensitive personal information is involved:

2. Scientific or historical research purposes or statistical purposes (Article 9(2)(j) in accordance with Article 89(1)).

This does not mean that consent is not necessary, it is the primary basis for meeting Common Law requirements (see below) and is central to ethical expectations and building trustworthy relationships. We also note that changes introduced via the incoming Data (Use and Access) Act 2025 may change this interpretation over time.

Further key facts about UK GDPR/DPA

Most LPS pseudonymise their data to remove identifiers and replace these with an ID number. Where pseudonymous data can be linked back to the identifiers pseudonymous data, it is still Personal Data under UK GDPR/DPA.

UK GDPR makes a distinction between data and sensitive data: all health data are considered sensitive as are other classes of information including many demographic characteristics. The processing of these will need additional safeguards.

UK GDPR sets out a series of rights for individuals. For research, there are derogations in place which mean that not all UK GDPR rights need to be applied. We note that some of these derogations, such as the right to object, are not implemented by many LPS due to ethical and trust considerations.

The Information Commissioner’s Office is the regulator for UK GDPR/DPA. There is extensive guidance on UK GDPR/DPA on the ICO website.

What are the requirements to meet Common Law Duty of Confidentiality?

LPS must establish a legal basis for addressing Duty of Confidentiality.

What is statute law?

This refers to primary legislation - also called Acts of Parliament or statutes. These are formal written laws made by the UK Parliament.

What is the Common Law Duty of Confidentiality?

This refers to the expectations resulting from the UK's cumulative case law relating to the handling of individuals' confidential data.

Common Law Duty of Confidentiality (which has the same status as statute law) applies when a person provides an LPS with information in the expectation that it will be treated in confidence. Whilst it can be debated as to what data is given with an expectation of confidence, it is our view at UK LLC that participants provide data to their study (e.g. in a questionnaire or fieldworker interview) with an expectation that this information is confidential. However, studies can set a reasonable expectation as to how this confidential data are used. For example, that survey data will be de-identified and shared with legitimate researchers for public benefit research.

For an LPS to process Personal Data it is therefore the case that they must make ‘best endeavours’ to set a ‘reasonable expectation’ about how this processing will impact on the confidentiality of the data and the rights of the participant. For example, the onward sharing of de-identified data with researchers, or the linkage of study data with participants’ records, or data about the place in which participants live.

To demonstrate compliance with Common Law Duty of Confidentiality LPS must either:
(i) collect informed consent where the scope of the consent covers the scope of the intended processing;
(ii) implement a mechanism to set aside Common Law Duty of Confidentiality requirements (such as Section 251 support to use NHS records without consent);
(iii) by fully anonymising the data;
(iv) or by meeting a public interest test.

The approach to addressing Common Law varies across the UK. For example, Section 251 only applies in England and Wales and only applies to health records; in contrast the Digital Economy Act also enables Common Law duties to be set aside and applies across the UK, although it cannot be used to support data from organisations whose primary function is health services (i.e. the NHS). While it is very clear that confidential data can be shared subject to a public interest test, we note this route is not widely used in England & Wales for research, yet is standard practice in Scotland.

In practice consent is the preferred mechanism however attempts to collect this retrospectively are likely to result in partial response where likelihood of responding is patterned by health and social characteristics.

UK LLC do not recommend permanently anonymising longitudinal data, as the destruction of the link between data and participant identifiers seriously curtails the future potential of the data. There are many examples of LPS taking advantage of unforeseen opportunities which rely on the retention of participant identifiers and there is no reason why this won’t continue to be the case.

There is some debate within the longitudinal research community as to whether address-based linkage are subject to the duty of confidentiality. UK LLC view place-based linked data as having a duty of confidence. This applies to data which are clearly not confidential in some situations (e.g. local rainfall data), yet we consider likely do gain that duty of confidence when linked to a participant and their wider participant data record.

Reasonable expectations and the “No Surprises” principle

LPS must take all reasonable measures to ensure participants have a ‘reasonable expectation’ as to how their data are used – this is also known as the ‘no surprises’ principle. While not an approved legal or regulatory position, we think of this via the thought experiment where if a participant visited a study or UK LLC and learnt about how everything worked, would they walk away thinking “I didn’t know much of that, but I am not surprised about any of the big picture as to what is happening”. I.e., to set a reasonable expectation, you do not need to be exhaustive in detail, but you do need to make the big principles around participation very clear. This extends both to who is using which aspects of your data and for what purpose, and then the safeguards that control this use.

For this reason, studies (and organisations such as UK LLC) must provide “fair processing” to participants through active communication routes, e.g. postal or electronic correspondence or newsletters. These can be supplemented by passive information via websites, social media or posters in community locations. See next page for more information and the ICO guidance on “layered” transparency materials.

Issues around consent

This section is not designed to explore the concept of consent, this is well documented by the regulators and the challenges of consent are extensively debated in the literature. We recommend for general advice you start by consulting the ICO and HRA/MRC.

However, we know through experience that many longitudinal studies are challenged about the nature or validity of their consent. This is especially true as consent is taken infrequently and changes in context and scientific opportunities influence the interpretation of consent statements and the information provided to participants at the time of consent and in subsequent fair processing. Note – in this guidance, “consent” is designed to mean the consent form and all accompanying information at the time (whether active mailings or study websites etc).

Some key facts to note:

• There is an awareness that expectations around consent have radically changed over time and that historical consent is not expected to meet today’s standards;

• Consent has no “end date”, a consent form completed e.g. 50 years ago is still valid;

• Yet, consent covers the described use of data in the consent form and information materials provided at that time;

• Explicit restrictions continue to apply over time (e.g., “we will never share your data with commercial companies” means that consent can not cover such sharing);

• Many consent materials contain ambiguities or contradictions, and interpretation of these should be informed by participant consultation and consideration of risks to the participant.

Where consent is no longer seen as valid, or not sufficient in scope to cover a proposed purpose (such as record linkage, or data analysis via the UK LLC TRE), then studies likely options are to re-consent or to seek exemptions under Common Law.

Seeking re-consent is often not possible or leads to partial response where the likelihood of a participant responding is patterned by health and social characteristics and therefore can introduce important biases into research or potentially further marginalises some harder to research communities such as children in the care system, or those in temporary accommodation.

This issue is explored in these publications:

• Cornish RP, Macleod J, Boyd A, Tilling K. Factors associated with participation over time in the Avon Longitudinal Study of Parents and Children: a study using linked education and primary care data. International journal of epidemiology. 2021 Feb 1;50(1):293-302.

• Boyd A. Understanding Population Data for Inclusive Longitudinal Research. Bristol: University of Bristol. 2021 Oct.

This can lead to a challenge, where LPS need to choose between re-consent and accepting a biased sub-set of responders OR seeking a consent alternative (described below). This decision is challenging as:

• Consent alternatives may have restrictions which do not apply to consented data (e.g. National Data Opt-Out applies in England and Wales to NHS data used under Section 251 (described below) but not consent);

• Participants have a complex view of consent, whilst the evidence is most participants want the research of longitudinal studies to be fair and equitable, they also view consent as an important ethical construct and part of the etiquette of the participant/study trust relationship. This issue is explored in this paper. [Audrey S, Brown L, Campbell R, Boyd A, Macleod J. Young people’s views about consenting to data linkage: findings from the PEARL qualitative study. BMC medical research methodology. 2016 Mar 21;16(1):34].

Sometimes seeking consent is not possible, if the person has died or no longer has capacity to make a choice (e.g. someone with dementia). For this reason, there are laws which make it possible to use data without breaching common law. These laws exist to help ensure research is fair and inclusive and society can benefit from research. For the use of NHS data in England, this law is called Section 251.

Section 251

Section 251 provides a lawful basis for an LPS to use identifiable health information for research without consent and where seeking consent is not practicable. Section 251 only applies in England and Wales.

While it is not possible to seek consent and then seek Section 251 for non-responders, it is possible to seek Section 251 and then attempt to seek consent over time when it is practical (e.g. during a fieldwork home visit). This is seen as desirable as Section 251 is intended as a temporary measure, so seeking consent where possible can provide the “exit route” from Section 251. Note, that once you have asked a person whose data is currently supported by Section 251 for consent, you are bound by that decision: consent, dissent and where non-response must be taken as dissent. We therefore only recommend seeking consent where you are confident you will get a response.

Further information on S.251 and Confidentiality Advisory Group (CAG) approvals and applications can be found here

Confidential Patient Information

Confidential Patient Information

This is information about any patient, alive or dead, that meets the following 3 requirements:

• is identifiable or likely to be identifiable, for example from other data likely to be held by the person or organisation receiving the data - if a patient could be identified from it.

• was given in circumstances where the individual is owed an obligation of confidence.

• conveys some information about the physical or mental health or condition of an individual, a diagnosis of their condition, or information on their care or treatment.

When a researcher obtains Confidential Patient Information from a health and adult social care provider in England with Section 251 support, they should expect that, once that organisation has stated its compliance with the national data opt-out policy then the records of patients who have registered a national data opt-out will have been removed. This may mean that the dataset provided might not be complete.

For avoidance of doubt where the approval for access to the data relies upon Section 251 support (i.e. approval under the NHS (Control of Patient Information Regulations) 2002 (regulation 2 or 5)) support then all data disclosed under this approval will be deemed to be confidential patient information even where the specific disclosure does not contain any health or care information. From: NHS Digital – when does a national data opt out apply.

As a general rule - if a participant has opted out of allowing their data to be used for research or planning through their GP surgery or online, a study will not be able to access their health records even if they decide to agree to linkage through the study. You can find out more about the National Opt-out here.

When researchers receive data, where national data opt-outs have been applied, there is a potential to introduce bias to their studies which researchers will need to consider. For example, there might be higher levels of opt-out across certain geographical areas, age groups or specific conditions that might impact on the results of studies. From: NHS Digital – what the national data opt-out means for researchers.

Utilising the Digital Economy Act (DEA):

The DEA provides a means to meet Duty of Confidentiality for the use of non-health routine records in public good research. To use the DEA the UK LLC must be accredited to UK Statistic Authority standards for a secure data processing environment. The DEA applies across the UK but does not apply to health data. The DEA will provide the legal basis for Duty of Confidentiality but LPS participating in non-health linkages via UK LLC will still need to meet their ethical requirements to ensure transparent data use and ‘no surprises’ and provide a means to object. Consent may be used as part of meeting the study’s ethical requirements.

UK LLC Confidentiality Due Diligence Panel

UK LLC has worked with NHS England’s Advisory Group for Data (AGD) to formalise a consistent review process. As part of the review framework, UK LLC has set up a Confidentiality Due Diligence Panel consisting of experts in Data Protection, Research Governance and public contributors. The Panel advises on whether new partnering LPS have established a basis for addressing their Common Law Duty of Confidentiality appropriately and ensured compliance with the principle of ‘no surprises’. For each LPS application the Panel is required to review the LPS consent materials (consent forms and participant information sheets).

UK LLC conducts an NHS England approved due diligence review of each LPS’ legal basis. The UK LLC Confidentiality Due Diligence Panel includes senior sponsor representation (ethics, governance and legal data protection leads) from the University of Bristol and public contributors. The Panel reviews each LPS’ consent materials to identify if there are any statements that either limit or introduce ambiguity into the scope of an LPS’ proposed data processing. Evidence from this process is provided to data owners (e.g. NHS England, Office for National Statistics) for their records and due diligence purposes. The minutes from the panel are published on the UK LLC website.

FAQs about UK LLC