Reporting of Conflicts of Interest in Cochrane Reviews
Reporting of Conflicts of Interest in Cochrane Reviews
The electronic database search yielded 272 unique titles and abstracts for review. Of these, 110 were excluded after review of the title and abstract because, although published in 2010, the review content was not classified as being up to date as of 2008 or later, no randomised controlled trials were included in the review, or the review did not assess the efficacy, effectiveness, or harm of a drug or class of drug against an alternative treatment. Of the 162 Cochrane reviews that underwent review of the full text, 11 were excluded because they were not systematic reviews of the efficacy, effectiveness, or harm of a drug or class of drug against an alternative treatment, leaving 151 eligible systematic reviews (Figure). Chance corrected agreement on inclusion and exclusion decisions between reviewers, as assessed with the Cohen’s κ statistic, was 0.95.
(Enlarge Image)
Figure 1.
Flow chart of selection of Cochrane reviews of drug trials published in 2010 with searches up to date as of 2008 or later
The 151 selected Cochrane reviews evaluated a broad range of drug interventions, including 18 on treatment efficacy or effectiveness, two on harms, and 131 on both efficacy or effectiveness and harms. Between one and 121 trials were included in each systematic review. The content of 27 Cochrane reviews (18%) was classified as up to date in 2008, 59 (39%) in 2009, and 65 (43%) in 2010. The review status of 39 reviews (26%) was listed as “new,” 51 (34%) as “new search” with or without a change to review conclusions, 50 (33%) as “edited” (that is, any modification which does not involve a search for new studies) with or without a change to review conclusions, and 11 (7%) as “stable” (that is, no further changes expected to the review). The 151 selected Cochrane reviews included systematic reviews from 36 of the 53 Cochrane Review Groups that were registered in 2010. (See Supplementary Appendix 2 for the characteristics of the 151 selected Cochrane reviews.)
Of the 151 selected Cochrane reviews, 125 (83%) reported review funding from non-industry sources, three (2%) reported review funding from both pharmaceutical industry and non-industry sources, three (2%) stated that the review was not funded, and 20 (13%) did not include a sources of support declaration (see Supplementary Appendix 2). Of the three reviews that reported funding from both industry and non-industry sources, we were able to clarify that for two studies this referred to previous funding to the authors unrelated to the review itself (personal communication, Christopher Eccleston, coordinating editor, Cochrane Pain, Palliative, and Supportive Care Review Group, 10 April 2012); for the third study, this reflected a contribution from Merck for a previous version of a review that had been carried out from 1998 to 2000, before the 2004 Cochrane policy that prohibited industry funding of reviews (personal communication, Jackie Price and Gerry Stansby, coordinating editors, Cochrane Peripheral Vascular Diseases Review Group, 4 May 2012). In 42 of the 151 Cochrane reviews (28%, 95% confidence interval 21% to 35%), at least one review author reported one or more financial ties to the pharmaceutical industry (see Supplementary Appendix 3).
Forty six of the 151 selected Cochrane reviews (30%, 95% confidence interval 24% to 38%) reported information on the funding source of least some of the included trials. Thirty reviews (20%, 14% to 27%) reported information on trial funding for all included trials and 16 (11%, 7% to 17%) reported for some, but not all, included trials ( Table 1 ; also see Supplementary Appendix 4). Four Cochrane reviews did not report or partially reported trial funding sources, but did state that the trial drug was provided by a pharmaceutical company for at least some trials (see Supplementary Appendix 5). One hundred and five Cochrane reviews did not report trial funding sources (70%, 62% to 76%), including (based on data extraction protocols) 11 reviews (7%) that recorded, but did not report, trial funding sources, 16 (11%) that provided a data extraction protocol that did not list trial funding source information, and 78 (52%) for which it could not be determined whether or not data on trial funding sources had been collected.
Among the 46 Cochrane reviews that reported any trial funding sources, partially or fully, this information was reported in as few as one and as many as seven locations in each review. In all, 22 different reporting patterns (see Supplementary Appendix 6) were observed. In each of the 46 Cochrane reviews that reported trial funding sources partially or fully, this information was reported in at least one of four locations in the review: in the risk of bias section of the text, in the risk of bias table attached to the characteristics of included studies table, in other text, or in a part of the characteristics of included studies table other than the risk of bias table. Trial funding source was reported in the context of the risk of bias assessment in 28 of the 151 Cochrane reviews (19%), including eight (5%) that reported this information in the risk of bias text only, four (3%) that reported in the risk of bias table only, 14 (9%) that reported in both of these locations, and two (1%) that reported in both of these locations plus the risk of bias figure. Twenty four reviews (16%) reported the sources of trial funding in other text, and 24 (16%) reported the sources of trial funding in the characteristics of included studies table in the “methods” or “notes” fields. Information on trial funding source was reported in other locations less often, including another table (one review, 1%), a footnote in a summary of findings table (five reviews, 3%), the abstract (one review, 1%), the plain language summary (two reviews, 1%), and in the context of sensitivity analyses (six reviews, 4%) ( Table 2 ; see also Supplementary Appendix 4).
Partial or full information on trial funding sources was reported in 16 of 39 reviews with “new” status (41%, 95% confidence interval 27% to 57%) and 30 of 112 reviews with an updated or amended status (27%, 19% to 36%), including 17 of 50 reviews with “edited” status (34%, 22% to 48%), 11 of 51 reviews with “new search” status (22%, 12% to 35%), and two of 11 reviews with “stable” status (18%, 5% to 48%). Trial funding source was partially or fully reported in nine of 27 reviews classified as up to date in 2008 (33%, 19% to 52%), 18 of 59 classified as up to date in 2009 (31%, 20% to 43%), and 19 of 65 classified as up to date in 2010 (29%, 20% to 41%).
Sixteen (11%, 95% confidence interval 7% to 17%) of the 151 Cochrane reviews reported trial author-industry financial ties or employment by industry. Eleven reviews (7%, 4% to 11%) reported information on author-industry financial ties from included trials, including two (1%) that reported for all included trials and nine (6%) that reported for some, but not all, included trials. Ten reviews (7%, 4% to 12%) partially reported trial author employment by industry, and none reported this fully. Of the 10 reviews that reported trial author-industry employment for some included trials, five also partially reported other trial author-industry financial ties ( Table 1 ; see also Supplementary Appendix 4). All of the reviews that reported information on trial author-industry financial ties or trial author-industry employment reported information on trial funding sources. All studies that extracted trial author-industry financial ties or employment data reported it in the review. In addition, in 30 reviews (20%) the data extraction protocols indicated that this information was not extracted and 105 (70%) did not provide enough information to determine if this information had been extracted.
Trial author-industry financial ties or employment by industry were reported in between one and three locations in each review that reported this information. Information on trial author-industry financial ties or employment was reported in the context of the risk of bias assessment in 15 reviews (10%), including five (3%) that reported this information only in the risk of bias section of the text, seven (5%) only in the risk of bias table, two (1%) in both locations, and one (1%) in both locations plus the risk of bias figure. Trial author-industry financial ties or employment were reported in other review text in three reviews (3%) and in the characteristics of included studies table in one review (1%) ( Table 2 ; see also Supplementary Appendix 4).
Overall, considering either partial or full reporting, 30 Cochrane reviews (20%) reported only information on trial funding sources, six reported on trial funding sources and trial author-industry financial ties (4%), five on trial funding sources and trial author-industry employment (3%), and five on all three (3%). Considering only full reporting, 28 reviews (19%) reported only on trial funding sources and two reviews reported on trial funding sources and trial author-industry financial ties (1%).
Of the 42 Cochrane reviews that had at least one review author with disclosed financial ties to industry, 12 (29%) reported information on trial funding sources compared with 34 of 109 reviews (31%) in which no review authors disclosed financial ties to industry. Two of 42 Cochrane reviews (5%) with review author ties to industry reported trial author-industry financial ties or employment by industry from included trials, compared with 14 of 109 Cochrane reviews (13%) without review authors with disclosed financial ties to industry.
Results
The electronic database search yielded 272 unique titles and abstracts for review. Of these, 110 were excluded after review of the title and abstract because, although published in 2010, the review content was not classified as being up to date as of 2008 or later, no randomised controlled trials were included in the review, or the review did not assess the efficacy, effectiveness, or harm of a drug or class of drug against an alternative treatment. Of the 162 Cochrane reviews that underwent review of the full text, 11 were excluded because they were not systematic reviews of the efficacy, effectiveness, or harm of a drug or class of drug against an alternative treatment, leaving 151 eligible systematic reviews (Figure). Chance corrected agreement on inclusion and exclusion decisions between reviewers, as assessed with the Cohen’s κ statistic, was 0.95.
(Enlarge Image)
Figure 1.
Flow chart of selection of Cochrane reviews of drug trials published in 2010 with searches up to date as of 2008 or later
The 151 selected Cochrane reviews evaluated a broad range of drug interventions, including 18 on treatment efficacy or effectiveness, two on harms, and 131 on both efficacy or effectiveness and harms. Between one and 121 trials were included in each systematic review. The content of 27 Cochrane reviews (18%) was classified as up to date in 2008, 59 (39%) in 2009, and 65 (43%) in 2010. The review status of 39 reviews (26%) was listed as “new,” 51 (34%) as “new search” with or without a change to review conclusions, 50 (33%) as “edited” (that is, any modification which does not involve a search for new studies) with or without a change to review conclusions, and 11 (7%) as “stable” (that is, no further changes expected to the review). The 151 selected Cochrane reviews included systematic reviews from 36 of the 53 Cochrane Review Groups that were registered in 2010. (See Supplementary Appendix 2 for the characteristics of the 151 selected Cochrane reviews.)
Review Funding and Review Author-Industry Financial Ties of Cochrane Reviews
Of the 151 selected Cochrane reviews, 125 (83%) reported review funding from non-industry sources, three (2%) reported review funding from both pharmaceutical industry and non-industry sources, three (2%) stated that the review was not funded, and 20 (13%) did not include a sources of support declaration (see Supplementary Appendix 2). Of the three reviews that reported funding from both industry and non-industry sources, we were able to clarify that for two studies this referred to previous funding to the authors unrelated to the review itself (personal communication, Christopher Eccleston, coordinating editor, Cochrane Pain, Palliative, and Supportive Care Review Group, 10 April 2012); for the third study, this reflected a contribution from Merck for a previous version of a review that had been carried out from 1998 to 2000, before the 2004 Cochrane policy that prohibited industry funding of reviews (personal communication, Jackie Price and Gerry Stansby, coordinating editors, Cochrane Peripheral Vascular Diseases Review Group, 4 May 2012). In 42 of the 151 Cochrane reviews (28%, 95% confidence interval 21% to 35%), at least one review author reported one or more financial ties to the pharmaceutical industry (see Supplementary Appendix 3).
Reporting in Cochrane Reviews of Trial Funding Sources
Forty six of the 151 selected Cochrane reviews (30%, 95% confidence interval 24% to 38%) reported information on the funding source of least some of the included trials. Thirty reviews (20%, 14% to 27%) reported information on trial funding for all included trials and 16 (11%, 7% to 17%) reported for some, but not all, included trials ( Table 1 ; also see Supplementary Appendix 4). Four Cochrane reviews did not report or partially reported trial funding sources, but did state that the trial drug was provided by a pharmaceutical company for at least some trials (see Supplementary Appendix 5). One hundred and five Cochrane reviews did not report trial funding sources (70%, 62% to 76%), including (based on data extraction protocols) 11 reviews (7%) that recorded, but did not report, trial funding sources, 16 (11%) that provided a data extraction protocol that did not list trial funding source information, and 78 (52%) for which it could not be determined whether or not data on trial funding sources had been collected.
Among the 46 Cochrane reviews that reported any trial funding sources, partially or fully, this information was reported in as few as one and as many as seven locations in each review. In all, 22 different reporting patterns (see Supplementary Appendix 6) were observed. In each of the 46 Cochrane reviews that reported trial funding sources partially or fully, this information was reported in at least one of four locations in the review: in the risk of bias section of the text, in the risk of bias table attached to the characteristics of included studies table, in other text, or in a part of the characteristics of included studies table other than the risk of bias table. Trial funding source was reported in the context of the risk of bias assessment in 28 of the 151 Cochrane reviews (19%), including eight (5%) that reported this information in the risk of bias text only, four (3%) that reported in the risk of bias table only, 14 (9%) that reported in both of these locations, and two (1%) that reported in both of these locations plus the risk of bias figure. Twenty four reviews (16%) reported the sources of trial funding in other text, and 24 (16%) reported the sources of trial funding in the characteristics of included studies table in the “methods” or “notes” fields. Information on trial funding source was reported in other locations less often, including another table (one review, 1%), a footnote in a summary of findings table (five reviews, 3%), the abstract (one review, 1%), the plain language summary (two reviews, 1%), and in the context of sensitivity analyses (six reviews, 4%) ( Table 2 ; see also Supplementary Appendix 4).
Partial or full information on trial funding sources was reported in 16 of 39 reviews with “new” status (41%, 95% confidence interval 27% to 57%) and 30 of 112 reviews with an updated or amended status (27%, 19% to 36%), including 17 of 50 reviews with “edited” status (34%, 22% to 48%), 11 of 51 reviews with “new search” status (22%, 12% to 35%), and two of 11 reviews with “stable” status (18%, 5% to 48%). Trial funding source was partially or fully reported in nine of 27 reviews classified as up to date in 2008 (33%, 19% to 52%), 18 of 59 classified as up to date in 2009 (31%, 20% to 43%), and 19 of 65 classified as up to date in 2010 (29%, 20% to 41%).
Reporting in Cochrane Reviews of Trial Author-Industry Financial Ties and Trial Author-Industry Employment
Sixteen (11%, 95% confidence interval 7% to 17%) of the 151 Cochrane reviews reported trial author-industry financial ties or employment by industry. Eleven reviews (7%, 4% to 11%) reported information on author-industry financial ties from included trials, including two (1%) that reported for all included trials and nine (6%) that reported for some, but not all, included trials. Ten reviews (7%, 4% to 12%) partially reported trial author employment by industry, and none reported this fully. Of the 10 reviews that reported trial author-industry employment for some included trials, five also partially reported other trial author-industry financial ties ( Table 1 ; see also Supplementary Appendix 4). All of the reviews that reported information on trial author-industry financial ties or trial author-industry employment reported information on trial funding sources. All studies that extracted trial author-industry financial ties or employment data reported it in the review. In addition, in 30 reviews (20%) the data extraction protocols indicated that this information was not extracted and 105 (70%) did not provide enough information to determine if this information had been extracted.
Trial author-industry financial ties or employment by industry were reported in between one and three locations in each review that reported this information. Information on trial author-industry financial ties or employment was reported in the context of the risk of bias assessment in 15 reviews (10%), including five (3%) that reported this information only in the risk of bias section of the text, seven (5%) only in the risk of bias table, two (1%) in both locations, and one (1%) in both locations plus the risk of bias figure. Trial author-industry financial ties or employment were reported in other review text in three reviews (3%) and in the characteristics of included studies table in one review (1%) ( Table 2 ; see also Supplementary Appendix 4).
Reporting in Cochrane Reviews of Trial Funding, Trial Author-Industry Financial Ties, and Trial Author-Industry Employment
Overall, considering either partial or full reporting, 30 Cochrane reviews (20%) reported only information on trial funding sources, six reported on trial funding sources and trial author-industry financial ties (4%), five on trial funding sources and trial author-industry employment (3%), and five on all three (3%). Considering only full reporting, 28 reviews (19%) reported only on trial funding sources and two reviews reported on trial funding sources and trial author-industry financial ties (1%).
Of the 42 Cochrane reviews that had at least one review author with disclosed financial ties to industry, 12 (29%) reported information on trial funding sources compared with 34 of 109 reviews (31%) in which no review authors disclosed financial ties to industry. Two of 42 Cochrane reviews (5%) with review author ties to industry reported trial author-industry financial ties or employment by industry from included trials, compared with 14 of 109 Cochrane reviews (13%) without review authors with disclosed financial ties to industry.
Source...