Guidelines for Reporting Outcomes in Trial Reports: The CONSORT-Outcomes 2022 Extension

Nancy J. Butcher, PhD1,2; Andrea Monsour, MPH1; Emma J. Mew, MPH, MPhil1,3; et al

JAMA. 2022;328(22):2252-2264. doi:10.1001/jama.2022.21022

Key Points

Question What outcome-specific information should be included in a published clinical trial report?

Findings Using an evidence-based and international consensus–based approach that applied methods from the Enhancing the Quality and Transparency of Health Research (EQUATOR) methodological framework, 17 outcome-specific reporting items were identified.

Meaning Inclusion of these items in clinical trial reports may enhance trial utility, replicability, and transparency and may help limit selective nonreporting of trial results.

Abstract

Importance Clinicians, patients, and policy makers rely on published results from clinical trials to help make evidence-informed decisions. To critically evaluate and use trial results, readers require complete and transparent information regarding what was planned, done, and found. Specific and harmonized guidance as to what outcome-specific information should be reported in publications of clinical trials is needed to reduce deficient reporting practices that obscure issues with outcome selection, assessment, and analysis.

Objective To develop harmonized, evidence- and consensus-based standards for reporting outcomes in clinical trial reports through integration with the Consolidated Standards of Reporting Trials (CONSORT) 2010 statement.

Evidence Review Using the Enhancing the Quality and Transparency of Health Research (EQUATOR) methodological framework, the CONSORT-Outcomes 2022 extension of the CONSORT 2010 statement was developed by (1) generation and evaluation of candidate outcome reporting items via consultation with experts and a scoping review of existing guidance for reporting trial outcomes (published within the 10 years prior to March 19, 2018) identified through expert solicitation, electronic database searches of MEDLINE and the Cochrane Methodology Register, gray literature searches, and reference list searches; (2) a 3-round international Delphi voting process (November 2018-February 2019) completed by 124 panelists from 22 countries to rate and identify additional items; and (3) an in-person consensus meeting (April 9-10, 2019) attended by 25 panelists to identify essential items for the reporting of outcomes in clinical trial reports.

Findings The scoping review and consultation with experts identified 128 recommendations relevant to reporting outcomes in trial reports, the majority (83%) of which were not included in the CONSORT 2010 statement. All recommendations were consolidated into 64 items for Delphi voting; after the Delphi survey process, 30 items met criteria for further evaluation at the consensus meeting and possible inclusion in the CONSORT-Outcomes 2022 extension. The discussions during and after the consensus meeting yielded 17 items that elaborate on the CONSORT 2010 statement checklist items and are related to completely defining and justifying the trial outcomes, including how and when they were assessed (CONSORT 2010 statement checklist item 6a), defining and justifying the target difference between treatment groups during sample size calculations (CONSORT 2010 statement checklist item 7a), describing the statistical methods used to compare groups for the primary and secondary outcomes (CONSORT 2010 statement checklist item 12a), and describing the prespecified analyses and any outcome analyses not prespecified (CONSORT 2010 statement checklist item 18).

Conclusions and Relevance This CONSORT-Outcomes 2022 extension of the CONSORT 2010 statement provides 17 outcome-specific items that should be addressed in all published clinical trial reports and may help increase trial utility, replicability, and transparency and may minimize the risk of selective nonreporting of trial results.

Introduction

Well designed and properly conducted randomized clinical trials (RCTs) are the gold standard for producing primary evidence that informs evidence-based clinical decision-making. In RCTs, trial outcomes are used to assess the intervention effects on participants.1 The Consolidated Standards of Reporting Trials (CONSORT) 2010 statement provided 25 reporting items for inclusion in published RCT reports.2,3

Fully reporting trial outcomes is important for replicating results, knowledge synthesis efforts, and preventing selective nonreporting of results. A scoping review revealed diverse and inconsistent recommendations on how to report trial outcomes in published reports by academic, regulatory, and other key sources.4 Insufficient outcome reporting remains common across academic journals and disciplines; key information about outcome selection, definition, assessment, analysis, and changes from the prespecified outcomes (ie, from the trial protocol or the trial registry) is often poorly reported.5-9 Such avoidable reporting issues have been shown to affect the conclusions drawn from systematic reviews and meta-analyses,10 contributing to research waste.11

Although calls for improved reporting of trial outcomes have been made,5,12 what constitutes useful, complete reporting of trial outcomes to knowledge users such as trialists, systematic reviewers, journal editors, clinicians, patients, and the public is unclear.4 Two extensions (for harms in 2004 and for patient-reported outcomes in 2013)13,14 of the CONSORT statement relevant to the reporting of specific types of trial outcomes exist; however, no standard reporting guideline for essential outcome-specific information applicable to all outcome types, populations, and trial designs is available.4

The aim of the CONSORT-Outcomes 2022 extension was to develop harmonized, evidence- and consensus-based outcome reporting standards for clinical trial reports.

Methods

The CONSORT-Outcomes 2022 extension was developed as part of the Instrument for Reporting Planned Endpoints in Clinical Trials (InsPECT) project15 in accordance with the Enhancing the Quality and Transparency of Health Research (EQUATOR) methodological framework for reporting guideline development.16 Ethics approval was not required as determined by the research ethics committee at The Hospital for Sick Children. The development15 of the CONSORT-Outcomes 2022 extension occurred in parallel with the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT)–Outcomes 2022 extension for clinical trial protocols.17

Key Development Phases

First, we created an initial list of recommendations relevant to reporting outcomes for RCTs that were synthesized from consultation with experts and a scoping review of existing guidance for reporting trial outcomes (published within the 10 years prior to March 19, 2018) identified through expert solicitation, electronic database searches of MEDLINE and the Cochrane Methodology Register, gray literature searches, and reference list searches as described.4,18 Second, a 3-round international Delphi voting process took place from November 2018 to February 2019 to identify additional items and assess the importance of each item, which was completed by 124 panelists from 22 countries (eTable 1 in the Supplement). Third, an in-person expert consensus meeting was held (April 9-10, 2019), which was attended by 25 panelists from 4 countries, including a patient partner and a public partner, to identify the set of essential items relevant to reporting outcomes for trial reports and establish dissemination activities. Selection and wording of the items was finalized at a postconsensus meeting by executive panel members and via email with consensus meeting panelists.

Other Information

The detailed methods describing development of the CONSORT-Outcomes 2022 extension appear in eAppendix 1 in the Supplement, including the number of items evaluated at each phase and the process toward the final set of included items (eFigure in the Supplement). The scoping review trial protocol and findings have been published4,18 and appear in eAppendix 1 in the Supplement and the search strategy appears in eAppendix 2 in the Supplement. The self-reported characteristics of the Delphi voting panelists and the consensus meeting panelists appear in eTables 1-2 in the Supplement. Details regarding the patient and public partner involvement appear in eAppendix 1 in the Supplement.

Results

In addition to the inclusion of the CONSORT 2010 statement checklist items, the CONSORT-Outcomes 2022 extension recommends that a minimum of 17 outcome-specific reporting items be included in clinical trial reports, regardless of trial design or population. The scoping review and consultation with experts identified 128 recommendations relevant to reporting outcomes in trial reports, the majority (83%) of which were not included in the CONSORT 2010 statement. All recommendations were consolidated into 64 items for Delphi voting; after the Delphi survey process, 30 items met the criteria for further evaluation at the consensus meeting and possible inclusion in the CONSORT-Outcomes 2022 extension. The CONSORT 2010 statement checklist items and the 17 items added by the CONSORT-Outcomes 2022 extension appear in Table 1.19

A fillable version of the checklist appears in eTables 3-4 in the Supplement and on the CONSORT website.20 When using the updated checklist, users should refer to definitions of key terms in the glossary21-38 (Box) because variations in the terminology and definitions exist across disciplines and geographic areas. The 5 core elements of a defined outcome (with examples) appear in Table 2.39,40

Box. Glossary of Terms Used in the CONSORT-Outcomes 2022 Extension

Composite outcome: A composite outcome consists of ≥2 component outcomes (eg, proportion of participants who died or experienced a nonfatal stroke). Participants who have experienced any of the events specified by the components are considered to have experienced the composite outcome.21,22

CONSORT 2010: Consolidated Standards of Reporting Trials (CONSORT) statement that was published in 2010.2,3

CONSORT-Outcomes 2022 extension: Additional essential checklist items describing outcome-related content that are not covered by the CONSORT 2010 statement.

Construct validity: The degree to which the scores reported in a trial are consistent with the hypotheses (eg, with regard to internal relationships, the relationships of the scores to other instruments, or relevant between-group differences) based on the assumption that the instrument validly measures the domain to be measured.30

Content validity: The degree to which the content of the study instrument is an adequate reflection of the domain to be measured.30

Criterion validity: The degree to which the scores of a study instrument are an adequate reflection of a gold standard.30

Cross-cultural validity: The degree to which the performance of the items on a translated or culturally adapted study instrument are an adequate reflection of the performance of the items using the original version of the instrument.30

Minimal important change: The smallest within-patient change that is considered important by patients, clinicians, or relevant others.4,5 The change may be in a score or unit of measure (continuous or ordinal measurements) or in frequency (dichotomous outcomes). This term is often used interchangeably in health literature with the term minimal important difference. In the CONSORT-Outcomes 2022 extension, the minimal important change conceptually refers to important intrapatient change (item 6a.3) and the minimal important difference refers to the important between-group difference. Minor variants of the term, such as minimum instead of minimal, or the addition of the adjective clinically or clinical are common (eg, the minimum clinically important change).23

Minimal important difference: The smallest between-group difference that is considered important by patients, clinicians, or relevant others.24-27 The difference may be in a score or unit of measure (continuous or ordinal measurements) or in frequency (dichotomous outcomes). Minor variants of the term, such as minimum instead of minimal, or the addition of the adjective clinically or clinical are common (eg, the minimum clinically important difference).23

Outcome: Refers to what is being assessed to examine the effect of exposure to a health intervention.1 The 5 core elements of a defined outcome appear in Table 2.

Primary outcome: The planned outcome that is most directly related to the primary objective of the trial.28 It is typically the outcome used in the sample size calculation for trials with the primary objective of assessing efficacy or effectiveness.29 Many trials have 1 primary outcome, but some have >1. The term primary end point is sometimes used in the medical literature when referring to the primary outcome.4

Reliability: The degree to which the measurement is free from error. Specifically, the extent to which scores have not changed for participants and are the same for repeated measures under several conditions (eg, using different sets of items from the same rating scale for internal consistency; over time or test-retest; by different persons on the same occasion or interrater; or by the same persons, such as raters or responders, on different occasions or intrarater).30

Responsiveness: The ability of a study instrument to accurately detect and measure change in the outcome domain over time.31,32 Distinct from an instrument’s construct validity and criterion validity, which refer to the validity of a single score, responsiveness refers to the validity of a change score (ie, longitudinal validity).30

Secondary outcomes: The outcomes prespecified in the trial protocol to assess any additional effects of the intervention.28

Smallest worthwhile effect: The smallest beneficial effect of an intervention that justifies the costs, potential harms, and inconvenience of the interventions as determined by patients.33

SPIRIT 2013: Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) statement that was published in 2013.35,36

SPIRIT-Outcomes 2022 extension: Additional essential checklist items describing outcome-related trial protocol content that are not covered by the SPIRIT 2013 statement.17

Structural validity: The degree to which the scores of a study instrument (eg, a patient questionnaire or a clinical rating scale) are an adequate reflection of the dimensionality of the domain to be measured.30

Study instrument: The scale or tool used to make an assessment. A study instrument may be a questionnaire, a clinical rating scale, a laboratory test, a score obtained through a physical examination or an observation of an image, or a response to a single question.34

Target difference: The value that is used in sample size calculations as the difference sought to be detected on the primary outcome between intervention groups and that should be considered realistic or important (such as the minimal important difference or the smallest worthwhile effect) by ≥1 key stakeholder groups.37,38

Validity: The degree to which a study instrument measures the domain it purports to measure.30

Application of these new checklist items from the CONSORT-Outcomes 2022 extension, in conjunction with the CONSORT 2010 statement, ensures trial outcomes will be comprehensively defined and reported. The estimated list of key users, their proposed actions, and the consequential potential benefits of implementing the 17 CONSORT-Outcomes 2022 extension checklist items appears in eTable 5 in the Supplement and was generated from the consensus meeting’s knowledge translation session. Examination and application of these outcome reporting recommendations may be helpful for trial authors, journal editors, peer reviewers, systematic reviewers, patients, the public, and trial participants (eTable 5 in the Supplement).

This report contains a brief explanation of the 17 checklist items generated from the CONSORT-Outcomes 2022 extension. Guidance on how to report the existing checklist items can be found in the CONSORT 2010 statement,2 in Table 1, and in an explanatory guideline report.41 Additional items that may be useful to include in some trial reports or in associated trial documents (eg, the statistical analysis plan or in a clinical study report42) appear in eTable 6 in the Supplement, but were not considered essential reporting items for all trial reports.

CONSORT-Outcomes 2022 Extension Checklist Items for the Descriptions of the Methods for Selecting, Measuring, and Describing the Outcomes Used in a Trial

Item 6a.1. Provide a Rationale for the Selection of the Domain for the Trial’s Primary Outcome

This item expands on CONSORT 2010 statement checklist item 6a to explicitly ask for reporting on the rationale underlying the selection of the outcome domain for use as the primary outcome. At a broad conceptual level, the outcome’s domain refers to the name or concept used to describe an outcome (eg, pain).10,39 The word domain can be closely linked to and sometimes used equivalently with the terms construct and attribute in the literature.40 Even though a complete outcome definition is expected to be provided in the trial report (as recommended by CONSORT 2010 statement checklist item 6a),2,40 the rationale for the choice of the outcome domain for the trial’s primary outcome is also essential to communicate because it underpins the purpose of the proposed trial.

Important aspects for the rationale may include (1) the importance of the outcome domain to the individuals involved in the trial (eg, patients, the public, clinicians, policy makers, funders, or health payers), (2) the expected effect of the intervention on the outcome domain, and (3) the ability to assess it accurately, safely, and feasibly during the trial. It also should be reported whether the selected outcome domain originated from a core outcome set (ie, an agreed standardized set of outcomes that should be measured in all trials for a specific clinical area).43

Item 6a.2. Describe the Specific Measurement Variable (eg, Systolic Blood Pressure), Analysis Metric (eg, Change From Baseline, Final Value, Time to Event), Method of Aggregation (eg, Mean, Proportion), and the Time Point for Each Outcome

This item expands on CONSORT 2010 statement checklist item 6a that recommends completely defining prespecified primary and secondary outcomes, and provides specific recommendations that mirror SPIRIT 2013 statement checklist item 12 (and its explanatory text for defining trial outcomes).35 CONSORT-Outcomes 2022 extension checklist item 6a.2 recommends describing each element of an outcome including its measurement variable, specific analysis metric, method of aggregation, and time point. Registers such as ClinicalTrials.gov already require that trials define their outcomes using this framework.10,35,39 Failure to clearly report each element of the outcomes from a trial enables undetectable multiple testing, data cherry-picking, and selective nonreporting of results in the trial report compared with what was planned.10,44

Item 6a.3. If the Analysis Metric for the Primary Outcome Represents Within-Participant Change, Define and Justify the Minimal Important Change in Individuals

This item expands on CONSORT-Outcomes 2022 extension checklist item 6a.2. In cases in which the participant-level analysis metric for the primary outcome represents intraindividual change from an earlier value (such as those measured at baseline), a definition and justification of what is considered the minimal important change (MIC) for the relevant study instrument should be provided. In the CONSORT-Outcomes 2022 extension, the MIC was defined as the smallest within-patient change that is considered important by patients, clinicians, or relevant others (common alternative terminologies appear in the Box).24,25,31 The MIC is important to report for all trials that use a within-participant change metric, such as those that plan to analyze the proportion of participants showing a change larger than the MIC value in each treatment group (eg, to define the proportion who improved)45 or in n-of-1 trial designs.46

Describing the MIC will facilitate understanding of the trial results and their clinical relevance by patients, clinicians, and policy makers. Users with trial knowledge may be interested in the MIC itself as a benchmark or, alternatively, in a value larger than the known MIC. Describing the justification for the selected MIC is important because there can be numerous MICs available for the same study instrument, with varying clinical relevance and methodological quality depending on how and in whom they were determined.47-50 If the MIC is unknown for the study instrument with respect to the trial’s population and setting, this should be reported.

Item 6a.4. If the Outcome Data Were Continuous, but Were Analyzed as Categorical (Method of Aggregation), Specify the Cutoff Values Used

This item expands on CONSORT-Outcomes 2022 extension checklist item 6a.2 to prompt authors, if applicable, to describe the prespecified cutoff values used to convert any outcome data collected on a continuous (or ordinal) scale into a categorical variable for their analyses.10,35 Providing an explanation of the rationale for the choice of the cutoff value is recommended; it is not unusual for different trials to apply different cutoff values. The cutoff values selected are most useful when they have clear clinical relevance.51 Reporting this information will help avoid multiple testing (known as “p-hacking”), data cherry-picking, and selective nonreporting of results in the trial report.10,44,52

Item 6a.5. If Outcome Assessments Were Performed at Several Time Points After Randomization, State the Time Points Used for the Analysis

This item expands on CONSORT-Outcomes 2022 extension checklist item 6a.2 regarding the time point to prompt authors, if applicable, to specify the time point used in the main analysis if outcome assessments were performed at multiple time points after randomization (eg, trial assessed blood pressure daily for 12 weeks after randomization). Specifying the preplanned time points of assessment used for the analyses will help limit the possibility of unplanned analyses of multiple assessment time points and the selective nonreporting of time points that did not yield large or significant results.35,39

Providing a rationale for the choice of time point is encouraged (eg, based on the expected clinical trajectory after the intervention or the duration of treatment needed to achieve a clinically meaningful exposure to treatment). The length of follow-up should be appropriate to the management decision the trial is designed to inform.53

Item 6a.6. If a Composite Outcome Was Used, Define all Individual Components of the Composite Outcome

A composite outcome consists of 2 or more component outcomes that may be related. Participants who have experienced any 1 of the defined component outcomes comprising the composite outcome are considered to have experienced the composite outcome.21,22 When used, composite outcomes should be prespecified, justified, and fully defined,51 which includes a complete definition of each individual component outcome and a description of how those will be combined (eg, what analytic steps define the occurrence of the composite outcome).

However, composite outcomes can be difficult to interpret even when sufficiently reported. For example, a composite outcome can disguise treatment effects when the effects on the component outcomes go in opposite directions or when the component outcomes have different effect levels (eg, combining death and disability), furthering the need for quality reporting for every component.22,54