Page 8 of 14
Clinical evaluation
When building evidence for the value of your product and assessing its use, it’s worth considering a clinical evaluation. Clinical evaluation is typically a longitudinal evaluation but one which will always be managed within the ethical and scientific remit of the geography you’re working in.
Randomised controlled trials (RCTs) are a type of clinical evaluation in which participants are randomly assigned to one of two or more clinical interventions. They are quantitative, comparative, controlled experiments, traditionally seen as the gold standard for evaluating the effectiveness of interventions.
The advantage of clinical trials and in particular RCTs is that they’re highly prescriptive with all relevant details worked out in advance of the research beginning. This includes having strict inclusion and exclusion criteria and well-defined intervention and endpoints.
Creating a research protocol
The first step in a clinical evaluation is to create an outline research protocol that describes the basic details of the study. It must be created and agreed by the research lead before applying for approval and before starting the study. The aim is to ensure that as well as answering specific research questions developed by the research team, the research is carefully designed to safeguard the health and safety of the participants in compliance with the conditions and principles of good clinical practice (GCP) and the applicable EU guidelines and regulations.
The research protocol is also a legal document that goes through approval by regulatory and ethical bodies. All parties and organisations involved in the study agree to comply with it. Once a final version has been approved, it should be stored with other key documents related to the study – sometimes called a Trial Master File.
Resources
Two templates for research protocols from the NHS Health Research Authority:
- Research protocol template for clinical trial research
- Research protocol template for qualitative research
Research proposal
Whether for a qualitative study, quantitative study or a mixed methods study, the research proposal for clinical studies should also be created for internal purposes. This can be part of the design development plan.
Roles and responsibilities in a clinical study
It is a statutory requirement that all clinical studies that fall under the Medicines for Human Use (Clinical Trials) Regulations 2004 have a named sponsor.
The sponsor must satisfy that the study meets all relevant standards and ensures that arrangements are put and kept in place for adequate management, monitoring and reporting. It is important to identify a sponsor as early as possible. It is typically the same person as research lead.
The research lead/sponsor has a number of other responsibilities:
- Managing local investigator(s) and ensuring that their files are available for audit and inspection.
- Document/record generation and archiving (see Study documentation, below)
- Identifying for each project the procedure for collecting, validating and analysing clinical study data
- Ensuring that all study data are kept in a secure location and in accordance with the terms of the Data Protection Act 1998, and managing restrictions by role around personally identifiable information (PII)
- Ensuring that any environment/ laboratory that will be used for the clinical study is adequate to conduct the trial properly and safely
- When using electronic study data and/or remote electronic study systems, ensuring that the system conforms to established requirements for completeness, accuracy, reliability and consistent intended performance
- Being responsible for the collection of data either remotely on a server or on a hard disk and ensuring the existence of local back-up systems (to guard against loss of data due to software and environment disasters) and disaster recovery procedures.
- Setting procedures for each project for dealing with the unlikely occurrence of research misconduct.
Aspects of clinical evaluation
Study documentation: This contains all the essential documents relating to a clinical study. It serves to demonstrate compliance with the standards of GCP and is to be overseen by the research lead or sponsor. Study documentation must be kept so that the data is accessible even after the study is completed. This is because future studies may suggest further follow-up, or allegations of fraudulent behaviour could be made, or concerns about side effects might arise and participants might need to be contacted.
Training Records: Staff working on clinical studies must ensure that they’re familiar with the requirements of GCP and that they maintain their own training records to show that all members of the trial team are qualified by education, training and experience to perform his or her tasks.
Safety Reporting: It is essential that all adverse events that occur during the course of participants’ involvement in a research project, are appropriately recorded and reported to ensure their continuing safety. (See our section on adverse events for more detail).
Consent: is another key issue. The research lead needs to write or delegate the writing of an informed consent form, which should be stored as part of the study documentation. This form is to reassure study participants about ethical considerations and inform them about their rights. Part of the informed consent form should be an explanation of how to make a complaint if a participant is unhappy with any aspect of their involvement. See our previous section on consent for more details.
Resources
During creation of an informed consent form, it is important to follow the recommendations of the World Medical Association’s Declaration of Helsinki (WMA, 1996) and consult the following sources for up to date information on the required content and format, including:
-
Health Research Authority’s guidance on informed consent
Note: If the study is being conducted in a country outside the UK, national regulations should be checked.
Note: If you’re dealing with vulnerable groups of participants, take particular care when preparing informed consent forms. Vulnerable groups include minors (under 16), incapacitated adults (defined as “adult[s] unable by virtue of physical or mental incapacity to give informed consent”) and people with learning difficulties.
Monitoring and risk management
Depending on the nature of the study, you may need to carry out monitoring to verify that:
- the rights and wellbeing of human subjects are protected
- the reported study data is accurate, complete and verifiable from source documents
- the conduct of the study is in compliance with the current protocol, with GCP and the applicable regulatory requirements
Monitoring can take place through on-site visits or by centralised monitoring:
- On-site monitoring is an ‘in person’ evaluation carried out by the Sponsor (Research Lead) or delegated representative at the site where the study is taking place. It can identify data entry errors and missing source data, and check local documentation. It can also provide valuable support for site staff in the trial processes and give assurance of quality of trial processes at site. Onsite monitoring is important in the early stages of a study and where the associated trial risks are considered to be high.
- Centralised monitoring is a remote evaluation carried out by the sponsor (research lead) or delegated representative at a location other than the site at which the trial is being conducted. It can be used to augment on-site monitoring and may be suitable for site activities that can be done remotely, such as those involving data checking.
Challenges of randomised controlled trials (RCTs)
RCTs also present some difficulties such as budget constraints, shrinking of patient populations, high patient dropout, trial sites being unable to reach enrollment targets or being delayed, and lower number of screened and enrolled patients due to protocol amendments.
Most importantly, the application of strict inclusion and exclusion criteria often leads to the inclusion of a very small proportion of the patients, which means that the efficacy of a treatment is only assessed in a specific population under ideal circumstances. However, the studied population does not necessarily reflect the real world and therefore the effectiveness of RCT is not guaranteed.
To better understand this, it is best to clearly differentiate between efficacy and effectiveness:
- Efficacy is the extent to which an intervention does more good than harm under ideal circumstances.
- Effectiveness is the extent to which an intervention does more good than harm when provided under the usual circumstances of healthcare practice.
The healthcare environment is increasingly competitive., For products to stand out and satisfy the demands of regulators, insurance companies, healthcare providers and patients, the effectiveness of atreatment or therapy must be assessed in real-world circumstances. This has pushed pharmaceutical and medical device companies to think beyond the control environment of clinical trials and find alternatives to supplement or follow up their product’s evaluation. We look at some of these in the next section.
Case study
Clinical evaluation of a novel test of cognition using wearable technology
Challenge
To build publishable evidence as to whether or not a specific target population would engage with and use a novel assessment of cognition, and/or whether the novel assessment was testing underlying aspects of cognition.
Approach
- Began with the primary and secondary research questions.
- Agreed on a target population of users and a recruitment process. Target population was adults with a clinical diagnosis of major depressive disorder (MDD).
- Agreed on a geographic location and a project approach.
- Outlined a research protocol, sponsorship, personnel and high-level project plan with dependencies.
- Highlighted the need for NHS ethics approval and the parameters with which the research could be agreed through NHS proportionate review.
- Built out the research details to satisfy stakeholders, sponsor, Ctrl Group process and NHS Ethics review, and iterated this material across the team in advance of ethics submission.
- Received NHS ethics approval and completed project in line with agreed approach
Results
- Clinical evaluation posted onto ClinicalTrials.gov as ‘A single centre pilot study to evaluate real-time passive and active high-frequency cognitive and mood assessment data in major depressive disorder using digital wearable technology’.
- Research delivered against the primary and secondary objectives. It helped build the evidence base for target population engagement with given cognitive assessment, as well as showing a correlation between the tested assessment and gold standard clinical measures of cognition.
- Ability to refer to longitudinal clinical evaluation with a clinically distinct population as part of the evidence base around Cognition Kit