Development of a Canadian guidance for reporting real-world evidence for regulatory and health-technology assessment (HTA) decision making

Background
Real-world evidence (RWE) can complement and fill knowledge gaps from randomized controlled trials to assist in health-technology assessment (HTA) for regulatory decision-making. However, the generation of RWE is an intricate process with many sequential decision points, and different methods and approaches may impact the quality and reliability of evidence. Standardization and transparency in reporting these decisions is imperative to appraise RWE and incorporate it into HTA decision-making. A partnership between Canadian health system stakeholders, namely Health Canada and Canada’s Drug Agency (formerly the Canadian Agency for Drugs and Technologies in Health (CADTH)), was established to develop a guidance for standardization of reporting of RWE for regulatory and HTA decision-making in Canada.

Objective
This study describes the methods to develop the Guidance for Reporting Real-World Evidence document and checklist for reporting RWE for regulatory and HTA decision-making in Canada.

Methods
the guidance was developed through a three-phase process that identified, extracted, and refined items related to RWE study components and reporting from existing documents. Phase 1 leveraged two existing environmental scans related to use of RWD and RWE and an evidence-mapping process to extract candidate recommendations on reporting and methodological considerations to be included in the guidance. Phase 2 used a two round, modified Delphi process with an Expert Methods Panel to select final recommendations from the candidate list, along with additional relevant items if required. This phase also incorporated considerations for operationalizing the recommendations as well as special considerations for the Canadian context. Phase 3 implemented a stakeholder consultation plan and Methods Expert Panel survey to incorporate and finalize revisions to the guidance based on public and stakeholder feedback

Results
A total of 13 respondents from the Methods Expert Panel completed the initial survey to include or exclude candidate recommendations. Most themes had a moderate to high proportion of items that generated agreement (67-92% of items with agreement). Themes with the lowest proportion of items that generated agreement were variables and data access and cleaning
methods.

Conclusion
Through a three-phase process that leveraged established global quality standards and extensive expert and public consultation, we developed the Guidance for Reporting Real-World Evidence for RWE related to regulatory and HTA decision-making in Canada.

Date Released: September 23, 2024