Research on policy mechanisms to address funding bias and conflicts of interest in biomedical research: a scoping review

Abstract

Background

Industry funding and author conflicts of interest (COI) have been consistently shown to introduce bias into agenda-setting and results-reporting in biomedical research. Accordingly, maintaining public trust, diminishing patient harm, and securing the integrity of the biomedical research enterprise are critical policy priorities. In this context, a coordinated and methodical research effort is required to effectively identify which policy interventions are most likely to mitigate against the risks of funding bias. Subsequently this scoping review aims to identify and synthesize the available research on policy mechanisms designed to address funding bias and COI in biomedical research.

Methods

We searched PubMed for peer-reviewed, empirical analyses of policy mechanisms designed to address industry sponsorship of research studies, author industry affiliation, and author COI at any stage of the biomedical research process and published between January 2009 and 28 August 2023. The review identified literature conducting five primary analysis types: (1) surveys of COI policies, (2) disclosure compliance analyses, (3) disclosure concordance analyses, (4) COI policy effects analyses, and (5) studies of policy perceptions and contexts. Most available research is devoted to evaluating the prevalence, nature, and effects of author COI disclosure policies.

Results

Six thousand three hundreds eighty five articles were screened, and 81 studies were included. Studies were conducted in 11 geographic regions, with studies of international scope being the most common. Most available research is devoted to evaluating the prevalence, nature, and effects of author COI disclosure policies. This evidence demonstrates that while disclosure policies are pervasive, those policies are not consistently designed, implemented, or enforced. The available evidence also indicates that COI disclosure policies are not particularly effective in mitigating risk of bias or subsequent negative externalities.

Conclusions

The results of this review indicate that the COI policy landscape could benefit from a significant shift in the research agenda. The available literature predominantly focuses on a single policy intervention–author disclosure requirements. As a result, new lines of research are needed to establish a more robust evidence-based policy landscape. There is a particular need for implementation research, greater attention to the structural conditions that create COI, and evaluation of policy mechanisms other than disclosure.

Background

Maintaining public trust, diminishing patient harm, and securing the integrity of the biomedical research enterprise are critical policy priorities. Nevertheless, despite the overwhelming body of evidence that links industry funding, industry affiliation, and conflicts of interest (COI) with positive results bias [1, 2], diminished trust [3], biases in setting research agendas [4, 5], and possible patient harm [6], too little has been done to implement meaningful policy change when it comes to preventing and managing COI especially in the context of biomedical research [7]. An initial search of the literature surrounding relevant policies indicates that although clinical practice and medical evaluation have seen policy improvements [7,8,9], considerable debate remains over the progress of meaningful policy reform in research contexts [10, 11]. While several reviews have surveyed extant policies in research organizations [12,13,14], a scoping review of the empirical research is still needed [7, 10, 15].

To get a broad overview of policy efforts, Mialon et al. recently conducted a scoping review that sought to identify the policy mechanisms that could manage or address corporate influence across a wide range of public health research sectors, including diet and nutrition, tobacco and alcohol, gambling, and pharmaceuticals. Though their list is non-exhaustive, the review identified four main types of policy mechanisms: transparency, management, monitoring/identification/education, and prohibition [12]. It was their objective to develop an initial list of mechanisms and examples that have been used to manage and reduce negative corporate influence on public health policy at national and international levels. The researchers concluded that more evidence is required regarding the effectiveness of the identified mechanisms in addressing the varying degrees and different types of influence. This scoping review aims to contribute to these efforts by conducting a targeted review of available policies and research efforts to address industry sponsorship, industry affiliation, and COI in the biomedical research enterprise that could support future policymaking and evaluation.

In recognition of these needs, this scoping review aims to identify and synthesize the available research on policy mechanisms designed to address industry sponsorship, industry affiliation, and COI in the biomedical research enterprise. We focus on the pharmaceutical and medical device industries as industries with active research and development and the predominant industry sponsors of biomedical research. Specifically, this review (1) documents the range of relevant policy mechanisms that have been addressed in the available literature, (2) maps the evidentiary landscape in this area, and (3) supports future research on available policy mechanisms. This work builds on the previous scoping review undertaken by Mialon et al. and prioritizes their definition of mechanisms as “policies, regulations, guidelines, codes of conduct, frameworks, standards, initiatives or other tools to address and/ or manage the influence of corporations on public health policy, research and practice” [12]. The current report differs from Mailon et al.’s review in that it narrows the focus to the biomedical research enterprise and evaluates published peer-reviewed articles rather than surveying extant policies proffered by organizations involved in biomedical research.

Methods

Inclusion/exclusion criteria

This review aimed to examine published, peer-reviewed empirical analyses of policy mechanisms designed to address industry sponsorship of research studies, author industry affiliation, and author COI at any stage of the biomedical research process. We focused on industry sponsorship because industry-funded research has been consistently shown to be a greater risk for favorable results bias and has been shown to be biased in favor of the sponsor’s product when compared to research funded by other sources [1]. Selected articles address research cycle activity at four stages: research question formation, methodological design, study conduct, and publication. Studies varied in their empirical methodological design and included those common to biomedical science, policy analysis, economics, bioethics, and the social sciences. Articles identifying a specific extant policy, a sample of extant policies, or conducting a descriptive policy analysis or evaluation were included. This review excluded articles that (1) did not address financial ties with the drug and device industries, (2) were not specifically related to drug and device research (3), did not engage at least one specific, identifiable COI-related policy or practice, (4) did not report the results of empirical or analytical research, or (5) did not have the full text available and accessible. See Table 1 for further details on exclusion criteria.

Table 1 Exclusion criteria

Search strategy

We implemented a broad search strategy on MEDLINE to locate relevant articles. Previous research has indicated that MEDLINE alone achieves 92.3% coverage for systematic reviews [16]. Furthermore, MEDLINE was selected over EMBASE because the former has 8.8% greater coverage of the social science literature [17]. The search strategy, implemented on PubMed, was informed by library and information science staff, as well as iterative refinement through provisional searching. The specific query used was:

(("Conflict of interest"[Mesh] or "competing interest*"[tiab] or "financial relationship*"[tiab] or “commercial interest*”) AND (polic*[tiab] or guid*[tiab] or evidence[tiab] or address*[tiab] or analy*[tiab] or manag*[tiab] or framework*[tiab] or standard*[tiab])) AND (biomed* OR pharm* OR device* OR drug* OR trial* OR medic* OR clinic*) NOT (“clinical guideline*”[tiab] OR “clinical practice guideline*”[tiab] OR tobacco[tiab] OR food[tiab] )

The search query was modified to exclude articles published prior to 2009, the year of the landmark Institute of Medicine Report on COI, as this report marked the last time a comprehensive synthesis of this literature was undertaken. Due to lack of available translation resources, this report only focused on articles published in English.

Screening

Selection for this scoping review occurred in two phases. During the first phase of screening, SSG, QG, and LB screened titles and abstracts, with two reviewers independently screening each entry to test for inclusion criteria based on an agreed-upon screening protocol. After the reviewers screened twenty-five percent of the initial search returns, the team members met to discuss inconsistencies and evaluated and readjusted the screening protocol. Going forward, SSG and QG screened each of the remaining entries, and inconsistencies were resolved by consensus. In phase two, SSG and QG independently conducted a full-text review of the articles returned, again resolving disagreements by consensus. All screening was managed in Covidence.

Data extraction

During the data extraction phase of this review, the research team collected data on paper characteristics, including:

• Publication details (author(s) name, publication year, focal regions for data collection, funding source, and COI disclosure)

• Research design (meta-research, cross-sectional, case control, etc.)

• Outcomes assessed (prevalence of COI policies, prevalence of COI disclosures, disclosure concordance with public databases, etc.)

• Focal population(s) (biomedical journals, journal articles, journal authors, conference presentations, IRB members, etc.)

• Sample size

We also extracted the main results for each paper in detail. The goal of this exercise was to record qualitative and quantitative assessments of the paper’s individual analyses relevant to the current review’s research aims. Here, we distinguish between analyses and papers. In this scoping review, we use the term “analysis” to refer to a particular assessment, usually of a given outcome in a given sample. Many papers included in the dataset reported multiple analyses. In preparation for data synthesis, we conducted an iterative, team-based, descriptive content analysis in accordance with JBI Scoping Review Network guidelines as elaborated by Pollock et al. [18]. Team members individually reviewed extracted study characterizations and identified potential study type categories. Through regular check-ins, we developed a consensus-based taxonomy that grouped all analyses into a designated study type category. The six primary analysis types identified here are COI policy surveys, disclosure concordance assessments, disclosure compliance assessments, disclosure policy effects assessments and analyses, surveys of policy perceptions and contexts. A substantial majority of papers evaluated disclosure policies exclusively, and this is represented in the analysis type taxonomy. See Table 2 for an account of each analysis type and any additional type-specific data extracted.

Table 2 Study types, definitions, and additional data collected (if any) by type

Data synthesis

Data synthesis was conducted by analysis type. For all categories, formal meta-analysis was disallowed due to the heterogeneity of research designs, outcome measures, populations, and sampling frames. For policy surveys, compliance assessments, and concordance evaluations, we provide summary statistics of key findings by outcome measure and population. That is, for policy surveys, we describe the range in COI policy prevalence in publication venues by parties subject to oversight (authors, editors, and referees). For compliance analyses, we provide summary statistics on compliance by outcome measure (COI and/or funding disclosure) and population (articles, meta-research, conference presentations, etc.). For concordance analyses, we provide summary statistics by population (authors, articles, etc.) and comparator (accountability database, subsequent research outputs). For each of these analysis types, we also chart outcomes by population over time to evaluate if there have been changes in outcomes longitudinally. Summary charts display relevant outcomes or comparators and populations. Each point size is scaled to the sample size of the analysis represented. All charts were created using R version 4.3.1 and ggplot2 version 3.4.3. For articles in other categories, we provide a narrative synthesis of findings.

Results

Eighty-one papers reporting on 116 relevant analyses were selected for inclusion in this scoping review, and Fig. 1 provides details of the screening and selection process for these papers. Table 3 provides a summary of paper-level characteristics for those included in this review. Nearly half of (34/81, 42.0%) papers that conducted COI policy surveys. Approximately a quarter (19/81 23.5%) conducted compliance analyses. Fewer papers conducted disclosure concordance analyses (16/81, 19.8%), policy effects assessments (16/81, 19.8%), or analyses of policy perceptions and contexts (9/81, 11.1%). Across categories, studies were conducted in 11 different focal geographic regions, with studies of international scope being the most common (39/81, 48.1%). Only 2/81 (2.5%) articles disclosed industry funding, and nearly half of all included articles reported that there were no author COI (39/81, 48.1%). Just under forty percent of articles reported that there were author COI (32/81), although, many of these reported COI were related to grants from government funding agencies. Finally, a small number (9/81, 11.1%) were missing COI disclosure statements altogether.

Fig. 1

PRISMA-SCR flowchart for identification of included studies

Table 3 Summary of article characteristics (study type & design, focal regions for data collection, study funding, and author COI)

Policy prevalence

In total, we identified 42 policy prevalence analyses in 34 individual articles. A substantial majority conducted quantitative analyses of the prevalence of COI policies at scientific publishing entities (biomedical journals and preprint servers). Median policy prevalence ranged from 31.9% for editors to 97.0% for authors. Editor and referee COI policies were noticeably less prevalent, with some studies finding none in their focal samples. Summary statistics for policy prevalence by covered group are available in Table 4. In addition to the 34 surveys of COI policies at biomedical journals, there were three surveys of COI policies at medical centers, and one each of surveys of COI policies at academic research institutions and preprint servers. Weinfort evaluated COI policy prevalence for researchers at academic medical centers (59/61, 96.7%), non-academic medical centers (67/77, 87.0%), and outpatient medical centers that contributed to clinical trials (27/61, 44.3%) [19]. Additionally, Resnik found that 100/100 (100%) academic research institutions had researcher-level COI policies [20]. The majority of policy surveys appeared after 2010 with increasing frequency over time across author, editor, and referee COI policy surveys. Figure 2 details the prevalence of biomedical journal COI policies by covered groups over time. Trendlines indicate that there is a general increase in policy prevalence over the review period (i.e., post-2009, the year of the IOM report) for each covered individual type. However, these data should be interpreted cautiously, given the heterogeneity of sampling frames. Of the four analyses that found lower than 70% prevalence of author COI policies, the sampling frames were English language journals accredited by the Iranian Publications Commission of the Ministry of Health and Medical Education [21], journals that published three or more of the randomly selected papers from Web of Science researchers for “glaucoma,” “macular,” or “cornea” [22], medical journals indexed by the 2014 Core journals of China [23], and journals sponsored by the European Society of Cardiology (ESC), European Heart House, National Societies Cardiovascular Journals (NSCJ) and ESC-affiliated cardiac societies [24]. Sampling frames for editor and referee policies were similarly heterogeneous, and interpretation is further challenged by the smaller number of studies overall.

Table 4 Summary Statistics (Min, Median, IQR, and Max) for reported policy prevalence by covered group

Fig. 2

Reported Prevalence (%) of Biomedical Journals with COI Policies for Groups Covered (Authors, Editors, or Referees) by Year. Each point is an induvial study, and point size represents study N (journals). Temporal trends visualized with loess [25]

Compliance analyses

Twenty-eight analyses evaluated compliance with COI disclosure policies. Compliance analyses measured the presence of required COI disclosures and / or funding disclosures in journal, published meta-analyses, or conference presentations or abstracts. Strengths of the compliance studies were that many had large, international samples. Compliance rates were far from perfect. Table 5 shows the median percent compliance by population included in the analysis, with journal articles having the lowest median compliance.

Table 5 Summary (min, median, IQR, max) of reported percent compliant by study population (journal articles, meta-analyses, and conference presentations or abstracts). Note: n = 27, excludes Kesselheim

Figure 3 shows compliance rates by population over time. Overall, compliance rates remain steady or decline over time, with even some of the most recent studies reporting low compliance rates [26, 27]. Studies assessing compliance of meta-analyses were more recent and showed higher compliance rates. However, the variability in compliance precludes any conclusions about trends over time. As noted in the policy section above, many institutions introduced COI policies after 2009, but there is no clear improvement in compliance observed. Kesselheim was excluded from the summary statistics on compliance since the authors did not report compliance at the article level, but rather at the journal level for articles sampled from the journals [28]. Specifically, the authors included a figure that indicates that 91–100% of articles were compliant in 30/51 journals, and 81–90% articles were compliant in 6/51 journals, and 71–80% of articles were compliant in 5/51 journals, and 61–70% of articles were compliant in 4/51 journals.

Fig. 3

Reported Compliance with COI and/or funding disclosure requirements by population type (Conference Presentations, Journal Articles, or Meta-Analyses. Each point is an induvial study, and point size represents study N (research outputs). N = 27, excludes Kesselheim, Temporal trends visualized with linear regression. Loess was disallowed due to data sparsity

Compliance analyses were more likely to examine COI disclosure statements of individual authors (20/28, 71.4%) than funding disclosures for the research (8/28, 28.6%). However, for some of the included analyses, funding source disclosures were evaluated as a COI, so author COI disclosures and funding sources for the research could not be distinguished. Another limitation of the compliance studies is that they assessed COI statements as published in articles, meta-analyses, or conference presentations. It is possible that COI were disclosed to journal editors or conference organizers but not published in the documents that were evaluated. All of the included analyses measured a dichotomous presence or absence of a COI statement to assess compliance but did not assess the content of the statements.

Concordance studies

In 16/81(19.8%) papers reported results for 21 analyses to comparatively examine the content of COI disclosures to determine the extent to which the content of disclosure statements was concordant between an index statement (usually in a journal article) and a comparator statement (e.g. in Open Payments). With the advent of transparency regulations in many countries worldwide [29, 30], pharmaceutical and medical device companies must routinely report the details of payments made to health professionals. To evaluate the comprehensiveness and consistency of the content of authors’ self-reported COI disclosures, researchers judged their concordance with payments data reported by pharmaceutical and medical device industry through regulatory and legal processes. Most commonly, researchers utilized United States Open Payments data (12/21, 57%), but also the Danish Health & Medicines Authority disclosure list (1/21, 5%) as well as payments data reported by manufacturers in company financial reports (2/21, 10%) and in legal filings (i.e., allegations under the US False Claims act, 1/21, 5%). With the reliance on Open Payments, most studies were limited to examining COI disclosures of physician authors only. In the absence of data reported by the sponsor, researchers judged the concordance among an author’s COI disclosures reported across multiple publications or research outputs (3/21, 14%). See Table 6 for concordance summary statistics.

Table 6 Summary statistics (min, median, IQR, max) of reported percent concordant by comparator (legal and regulatory filings, public payments databases, or subsequent research outputs)

In judging concordance, most researchers assumed that an author should comprehensively and consistently report all financial relationships with commercial entities existing within a specific time period in all research outputs. Thus, most (20/21, 95%) assessed the extent to which an author’s self-reported COI disclosure included all existing or known industry relationships. In one analysis, researchers assessed whether the dollar value of the industry payment constituting a COI was concordant across an author’s disclosures.

Studies of concordance acknowledged that concordance could occur along a gradient – for example, measuring the extent to which authors self-reported none, some, or all of their industry relationships. Researchers employed various units of analysis, including both research output and author-level analyses. As a result, quantitative meta-analysis was not possible. To visualize a comparison across studies of disclosure concordance, we extracted the proportion of disclosures assessed as completely concordant (all self-reported relationships matched those in the comparator data source) (Fig. 4). Rates of complete concordance between author self-reported COI disclosures and industry-reported data ranged wildly across studies from 1% (physician authors in ophthalmology journals in 2017) [31], to 94% (physician participants in the Eastern Association for Surgery of Trauma, Western Trauma Association, American Association for the Surgery of Trauma meetings from 2016–2019). Variability was noted across clinical specialties (lower concordance in oncology at 24% [32] and ophthalmology (1%) [31], for authors of articles focused on particular health technologies (12% for authors of articles on dermal substitutes in burn management [33], or 15% for authors of articles on placental membrane allograft products) [34], which may reflect the higher prevalence of industry relationships in certain specialties or a concentration of payments around research related to commercialized products [4]. Concordance rates were also lower in studies which sampled authors receiving very high value payments: for example, Chimonas et al. found that 15% of physicians who (1) reported receiving > = $1 million from a single orthopedic device manufacturer on Open Payments and (2) published in the orthopedics field consistently disclosed the name of the company making the payment in their subsequent publications [35]. Conversely, studies reporting high rates of concordance noted that the majority of sampled authors had no COIs and also declared “none,” while still noting high rates of non-concordance between COI disclosures and industry data sources among authors with industry relationships [36]. We did not note any patterns in concordance across time.

Fig. 4

Reported Disclosure Concordance by Comparator Data Set (Legal and Regulatory Filings, Public Payment Databases, and Subsequent Research Outputs). Each point is an individual study; point-size represents study N (research outputs), and study population (journal articles, conference presentations, or both) are indicated by point color

Policy effects

Despite broad investment in disclosure in biomedical research and clinical practice, the available evidence on the efficacy and effectiveness of disclosure from the perspective of readers is underwhelming. There is some evidence to indicate that professional readers, such as clinicians evaluating research literature, find disclosure information useful; however, this evidence is largely from survey data in which respondents self-reported their typical responses to COI disclosures. For example, one survey found that a limited proportion of the American Society for Clinical Oncology (ASCO) physician members reported regularly evaluating COIs [37]. Another paper found that although surveyed physician members of the American College of Obstetricians and Gynecologists (ACOG) reported that clinicians should discount efficacy claims in the presence of researcher COI, no such discounts were observed in a randomized controlled simulation trial of the same population [38]. Similarly, a RCT evaluating the effects of COI disclosures on French general practitioners found no evidence that disclosure affects physician confidence in the conclusions of abstracts [39]. Studies of physicians who served as peer reviewers also found limited effects of disclosure. While one survey of Annals of Emergency Medicine peer reviewers found that although the majority of respondents expressed concern about COI, those reviewers did not generally report any actions taken during peer review to address these concerns [40]. A later survey of additional Annals of Emergency Medicine peer reviewers also found that COI disclosures did not meaningfully affect reviewers’ assessment of manuscript quality [41].

The available evidence on the effects of disclosure on lay members of the public such as patients or research participants is more varied. Analyzing survey data from patients living with multiple sclerosis, an article found that COI disclosure might affect patient participation in multiple sclerosis treatment studies [42]. Another qualitative analysis of research participant surveys found that patients believed disclosed COIs may reduce the quality of care while increasing costs [43]. The same analysis also reported that COI disclosure adversely affected perceptions of study quality [43]. However, another survey of participants in genomic addiction research found that COI disclosure did not adversely affect participant trust [44]. These inconsistent results suggest that similarly to professional readers of COI disclosures, the relevance of disclosures and how they should be evaluated and then addressed, is unclear, resulting in variable approaches, which may differ across populations.

Several social science analyses of disclosure policies suggest that mandatory disclosure regimes do not effectively address the most substantial moral hazards created by the existence of COI and may cause harm in the process. Reporting on newly adopted European disclosure policies, Sperling provided a detailed ethical analysis of disclosure policies [45]. As he noted, disclosure effectively fulfilled moral obligations around transparency, but could be a double-edged sword for trust. While disclosure demonstrates a sort of forthrightness, mandatory disclosure policies may inadvertently compromise patient trust in the medical system. Furthermore, Sperling’s analysis demonstrates that formalized disclosure policies serve to legitimize financial relationships with a well-established corrupting influence on clinical decision-making. These findings echo a previous study of the social effects of disclosure using Austin’s performative speech act theory [46]. Specifically, the paper noted that disclosure functions as a social performance, but one that has widely varied effects on those who are supposed to judge the meaning of disclosures. Audiences for disclosures can increase trust, decrease trust, or make accusations of misconduct. Ultimately, Mayes, Lipworth, and Kerridge’s analysis indicates that disclosure does little to address the hazards of COI while catalyzing unpredictable results [46]. Finally, an analysis of the well-known Dan Markinson case demonstrates that recipients of disclosure information need to be able to understand the potential impacts of COI in order to appropriately assess related risks [47].

In addition to these evaluations of the effects of disclosure on perceptions of research quality and trust in researchers, studies have measured the impact of journal policies on disclosure, including adherence to ICMJE guidelines and the decision to accept reprint fees. For example, based on a controlled pre-post experimental design, an analysis found that the German medical journal that began adhering to ICMJE guidelines saw a significant increase in the reporting of conflicts in comparison to those journals that did not change their COI reporting forms [48]. Another paper evaluated the associations between journal-level COI policies and author-level disclosures [49]. This analysis found that journals accepting reprint fees published more articles by authors who disclosed financial relationships than journals that do not accept reprint fees.

Policy contexts and perceptions

A final set of studies focused on perceptions about and understandings of COI policy and the contexts in which they were developed, implemented, and enforced. These studies examined aspects of the policy context, including perceptions of the need for disclosure, the accessibility and usability of disclosures, challenges related to implementation and oversight, and analyses of policy proposals related to authorship and research funding structures. Three papers analyzed the perceived need for COI disclosures by relevant stakeholders. As previously mentioned, a survey of ACOG members found that 69% reported that disclosure information was important for evaluating reported results, even though this was not observed in the controlled simulation study [38]. Another survey of American Society for Clinical Oncology (ASCO) stakeholders determined the extent to which clinicians and members who are not physicians wanted COIs disclosed [37]. This article found that a majority (77%) found disclosure to be important and leaned towards disclosures that were reported in a written and documented manner. Analyzing how patients enrolled in genomic addiction research perceived COIs, another paper reported a survey demonstrating that more than a third of the participants preferred being made aware of COIs (38.4%) and approximately another third (35.3%) leaned towards having this information available [44].

The available literature also indicates that there are significant challenges to effectively implementing COI policies. Papers evaluating implementation and oversight challenges investigate (a) the difficulty in creating usable and accessible disclosure infrastructures [50], (b) lack of clear policies or guidelines to evaluate COI [51,52,53,54]; (c) contradictions among COI and commercialization/technology transfer policies [55, 56]; and (d) lack of policy enforcement mechanisms [57]. Even when disclosure has been agreed upon as a policy solution, it can be difficult to access and therefore assess disclosure information. Previous research in this area has identified this as a particular challenge for biomedical journals [58]. Similarly, one paper in our analysis evaluated the accessibility and usability of publicly available information on physician-industry relationships on the disclosure websites of U.S. states that mandated public reporting [50]. This analysis reported that the accessibility of disclosure information varied widely across different websites and that this information was generally difficult to reach.

The evidence of lack of clear policy guidelines largely comes from studies of U.S.-based institutional review boards (IRBs). A survey of 300 members of twenty single IRBs or sIRBs (single IRBs, IRBs that coordinate multi-site trials) in government, academic, and corporate settings found significant variances in policy implementation [51]. Although survey respondents indicated that corporate sIRBs tend to have firm institutional COI policies that involve turning down some clients and establishing communication “firewalls,” those engaged with corporate sIRBs often reported a sense that there were strong pro-sponsor biases. Respondents further indicated that government sIRBs adopt many of the same COI disclosure and management policies as other U.S. government agencies. Finally, respondents from academic sIRBs generally indicated an absence of clear COI management policies. These findings partially replicate a prior interview study of chairs and members at 46 U.S.-based academic IRBs. Detailed interviews found that many participants report significant challenges in identifying and evaluating COI [52]. Informants reported particular challenges around evaluating the hazards associated with COI that are currently non-financial but may lead to future remuneration.

Even in cases where relevant COI policies may be clearly defined, the available research also points towards additional problems created by contradictory policy environments. That is, researchers in many institutions are faced with often contradictory or at last discordant COI and innovation/commercialization policies. A legal and ethical analysis detailed how innovation and commercialization initiatives incentives at research performing organizations can serve to incentivize new COI by encouraging researchers to seek licensure agreements with industry [56]. A subsequent ethnographic study of physician-researchers in Danish universities confirmed that these contradictory polices do, in fact, lead to additional industry entanglements in practice and thus inhibit the effective management of COI [55]. Finally, the few studies that specifically evaluated COI policy oversight found a lack of actionable enforcement mechanisms. For example, one assessment of COPE COI policies determined that the COPE framework did not provide any mechanism for enforcing COI-related guidelines [57]. Additionally, the previously discussed interviews with U.S.-based IRB members also indicated that COI oversight at many IRBs is limited to voluntary recusal [52]. This finding is consistent with legal analyses of the U.S. regulatory climate around COI at IRBs. Although U.S. regulations limit the extent to which individuals with COI can serve as IRB members, those same regulations do not require disclosure to IRB chairs or administrators [54].

Discussion

We conducted a detailed scoping review of 116 analyses of COI policies published in 81 papers. The results of our research demonstrate that the bulk of available research is devoted to evaluating the prevalence, nature, and effects of disclosure policies, with an overwhelming focus on individual COI rather than systemic factors. See Table 7 for a summary of key findings. This evidence demonstrates that while disclosure policies have become pervasive. Since the publication of the IOM report in 2009 and the passage of the US Physician Payment’s Sunshine Act in 2010, those policies are not consistently designed, implemented, or enforced. The available evidence also indicates that COI disclosure policies are not particularly effective in mitigating either risks of bias or subsequent negative externalities. Although surveys of physicians and researchers indicate that they perceive value in having access to disclosure information, experimental studies of the effects of disclosure generally indicate that reported COI do not meaningfully affect perceptions of research quality, peer review decisions, or clinical action [38,39,40,41]. Qualitative studies of clinician and researcher decision-making around COI indicate that key concepts and thresholds for action are not well understood [51, 52]. Many who are charged with disclosing and/or evaluating disclosed COI report being unsure which relationships should be disclosed and which should prompt action.

Table 7 Key findings

Implications

The results of this scoping review indicate that the COI policy landscape could benefit from a significant shift in the research agenda. The available literature is almost exclusively focused on a single policy intervention, disclosure requirements, to the extent that “COI policy” and “COI disclosure policy” are used interchangeably in much of the literature and policy space. Since the available evidence indicates that disclosure is not a particularly effective strategy for reducing the risks of bias associated with COI, we echo prior suggestions that disclosure should not be a primary priority for policy intervention [59, 60]. Additionally, a substantial portion of the literature merely surveys the existence of COI policies in academic journals or research-performing organizations. The studies evaluate if there are policies and to whom they apply. The studies do not seek to determine why there are differences in policy prevalence or implementation, and the heterogeneity of sampling frames means that we cannot draw conclusions about the underlying reasons for low policy prevalence. Policy and implementation research is unlikely to benefit substantially from additional policy surveys for each subspecialty or geographic region.

The available evidence also disclosure is frequently inconsistent. Median concordance rates are often low between disclosure contexts, and compliance rates vary widely by context. Conference presentations or abstracts have the highest median compliance, and the lowest minimum compliance. Compliance, in this context, may be buoyed here through the use of slide deck templates that frontload disclosure on the second slide. Article disclosures may be easier to lose track of given that templates are used later in the writing process, and that COI disclosures are attended to alongside a long of other disclosure requirements many of which may not apply to individual articles (e.g., acknowledgements or data and code deposits). Future research should consider additional mechanisms or practices that journal editors might use to better enforce compliance.

Ultimately, we suggest that no more policy survey research is needed. Furthermore, the evidence on COI policies would be improved through cultivating new and emerging lines of inquiry. A robust research agenda evaluating policy mechanisms beyond disclosure would be of substantial benefit. For example, the widely-cited American Medical Student Association’s scorecard for COI policies at medical schools includes differential recommendations about which types of industry relationships should be prohibited, monitored, and/or disclosed [61]. Similarly, clinical practice guidelines committees make use of COI risk ratings to inform their work [62]. Unfortunately, research on the effects of COI tends not to disaggregate by relationship characteristics (e.g., relationship type, funding amount, relationship duration, or level of sponsor control) [2]. As a result, policy that stratifies by relationship type or funding amount is not grounded in evidence, and additional research is needed to evaluate which relationship characteristics carry the most risk. Furthermore, Mialon et al.’s scoping of 49 proposed COI policy mechanisms points towards additional areas for policy development that may be fruitful.

The available theory and evidence posits that industries systematically enact multi-faceted and multi-directional strategies to enroll key stakeholders and to shape favorable evidentiary and policy environments [63,64,65]. COI policies that focus on individual relationships, by themselves, may be insufficient to blunt the risks posed by certain industry relationships nor even begin to address the structural and systemic conditions that give rise to COI. This inadequacy is evident in the wealth of evidence examining the association of COI and research outcomes bias [1]. Additionally, COI policies and management strategies function in a complex policy ecosystem that includes adjacent initiatives like journal authorship policies and institutional innovation and commercialization initiatives. More rigorous standardization of and management of authorship criteria may be able to effectively buttress COI policies, as traditionally conceived [53, 66]. The COI evidence base could also benefit from additional attention to the effects of contradictory institutional policies that seek to limit COI while simultaneously encouraging commercialization [55, 56]. These complexities point towards the need for more investigation into COI at systemic and aggregate levels [67]. Emerging research in this area suggests that the biasing effects of COI may aggregate at the systems level and that novel evidence-based policy interventions may be required [2]. This will likely mean a shift in attention away from individual COI and towards addressing industry influence and cultivating research independence at the systems and structural levels.

For example, it is also important to note that recourse to industry funding is closely linked to the broader research funding landscape. A 2021 qualitative analysis of interview data from European stakeholders suggested that COI could decline in the presence of more opportunities for non-commercial funding [68]. These findings echo contemporaneous results from a recent US National Bureau of Economic Research analysis of university research funding in the US [69]. The study found that in the 15 years prior to 2021, research funding has been marked by a simultaneous decline in federal funding and increase in industry funding. This shift toward funding monoculture is associated notably with an increasing focus on patents and commercialization and may increase the biasing hazards of COI. These findings suggest that COI policy research would benefit substantially from greater attention to the larger research funding landscapes and related analyses of the effects of policies that encourage seeking industry funding. The available literature provides a variety of recommendations of possible policy strategies for reducing scientific reliance on industry funding [70]. It is critical that new programs of research be launched to evaluate the efficacy of these kinds of solutions. Importantly, this cannot be realized without adequate funding. One possible solution here is for additional countries to embrace the Italian model of taxing pharmaceuticals companies with the express aim of funding research in the public benefit [71].

Limitations

This review has several limitations. The available evidence base is not well-equipped to support a robust understanding of COI policies. The empirical evidence focuses overwhelmingly on disclosure to the exclusion of other proposed policies. Furthermore, a substantial portion of the available evidence is limited to evaluations of if policies exist and/or who they cover rather than the effects of those policies. While compliance and concordance studies offer some insights into the implementation of disclosure policies, they provide scant information on the extent to which polices achieve target aims. Finally, our ability to draw firm conclusions from this evidence base is limited by an evidence base frequently marked by poor research design and reporting quality. For example, units of analysis are not consistently operationalized across papers and analyses. A particular and recurrent problem is the conflation of author-level COI and study funding. Many articles treat these together as a single binary variable, an issue endemic to the COI literature more broadly [2]. Additionally, each paper in the dataset contained an average of 1.45 analyses. As a result, there was an unfortunate lack of methodological and results detail for individual analyses.

Conclusions

In 2009, the IOM set a clear research agenda for better understanding COI policies and their effects [72]. The report asserted that policies must not only address important and common conflicts but must also be practical in terms of their implementation and cost and consider likelihood of bias and extent of harm. The report also noted that the policies must not be a roadblock for research and clinical practice and their benefits must outweigh their cost. These guidelines collectively outline a robust agenda for COI policy research. The results of this scoping review indicate that the IOM’s vision for a robust COI policy agenda has not come to fruition. This may, in part explain why Torgerson et al.’s ten-year retrospective on the IOM report indicates that there has been limited progress in terms of COI policy implementation, especially as related to COI in biomedical research [7]. Indeed, there is some evidence to suggest that delays in policy implementation are partially the results of intentional efforts to maintain current economic incentives despite the well-established risk profile [67]. Regardless of the cause of delay, the present scoping review adds additional evidence that there remains much work to establish a robust COI policy research agenda and evidence base. This is an essential first step towards establishing a robust policy landscape.