|Year : 2019 | Volume
| Issue : 7 | Page : 2179-2183
Bias in clinical trials into the effects of complementary and alternative medicine therapies on hemodialysis patients
Mohsen Adib-Hajbaghery1, Zohreh Nabizadeh-Gharghozar1, Parisa Nasirpour2
1 Department of Nursing, Trauma Nursing Research Center, Kashan University of Medical Sciences, Kashan, Iran
2 Department of Nursing and Midwifery, Shiraz University of Medical Sciences, Shiraz, Iran
|Date of Submission||02-Mar-2019|
|Date of Decision||04-Apr-2019|
|Date of Acceptance||06-Jun-2019|
|Date of Web Publication||31-Jul-2019|
Prof. Mohsen Adib-Hajbaghery
Department of Nursing, Trauma Nursing Research Center, Kashan University of Medical Sciences, Kashan
Source of Support: None, Conflict of Interest: None
Background: Chronic renal failure is among the major health challenges in the world. Many clinical trials have been conducted to assess the effects of complementary and alternative therapies on hemodialysis-related outcomes. However, a number of biases may affect the results of these studies. Aims: This study aimed to assess biases in randomized clinical trials into the effects of complementary and alternative therapies on hemodialysis patients. Settings and Design: A critical review on clinical trials into the effects of complementary and alternative therapies therapies on hemodialysis patients. Materials and Methods: This study was conducted on 114 randomized clinical trials which had been published in 2012–2017 into the effects of complementary and alternative therapies on hemodialysis patients. The Cochrane Risk of Bias Tool was employed to assess biases in the included trials. The collected data were presented using the measures of descriptive statistics, namely absolute and relative frequencies. Results: Among 114 included trials, 71.05% (81 trials) had used low bias methods for random sequence generation, while 60.52% (69 trials) had provided no clear information about allocation concealment. Moreover, respecting blinding, 57.89% of trials (66 trials) were low bias. Around 60.52% of trials (69 trials) had no attrition between randomization and final follow-up assessment and 84.21% (96 trials) had apparently reported all intended outcomes. Conclusions: This study shows that 50% of randomized clinical trials into the effects of complementary and alternative therapies on hemodialysis patients have low bias. Yet, quality improvement is still needed to produce more conclusive evidence.
Keywords: Bias, chronic renal failure, complementary and alternative medicine, randomized clinical trial
|How to cite this article:|
Adib-Hajbaghery M, Nabizadeh-Gharghozar Z, Nasirpour P. Bias in clinical trials into the effects of complementary and alternative medicine therapies on hemodialysis patients. J Family Med Prim Care 2019;8:2179-83
|How to cite this URL:|
Adib-Hajbaghery M, Nabizadeh-Gharghozar Z, Nasirpour P. Bias in clinical trials into the effects of complementary and alternative medicine therapies on hemodialysis patients. J Family Med Prim Care [serial online] 2019 [cited 2020 Jan 23];8:2179-83. Available from: http://www.jfmpc.com/text.asp?2019/8/7/2179/263751
| Introduction|| |
Chronic renal failure is among the major health challenges in the world. It is associated with different complications, chiefly uremia. In order to prevent these complications, patients need to receive renal replacement therapies, such as hemodialysis, for the life. Although these therapies help improve biochemical parameters in the body and save patients' lives, they do not necessarily improve quality of life and thus, many patients with renal failure suffer from poor quality of life despite receiving these therapies.
Complementary and alternative medicine (CAM) therapies are widely used to improve patient outcomes and quality of life. The National Center for Complementary and Integrative Health defines CAM as “a group of diverse medical and healthcare interventions, practices, products, or disciplines that are not generally considered part of conventional medicine”. The use of CAM therapies has significantly increased in recent years, so that 20--80% of the world population use these therapies.,
Randomized controlled trials (RCTs) are the most rigorous type of clinical trials and the best method for assessing causal relationships. The unique characteristics of RCTs are the use of an intervention in an intervention group, inclusion of a control group, and the use of random allocation. Despite the rigorousness and usefulness of RCTs, they are always prone to some types of errors. A certain type of error in RCTs is called bias that is broadly defined as any systematic error in the design, implementation, data collection, data analysis, or data interpretation in a study which can distort its findings and conclusions. Biases can significantly affect quality, internal validity, and external validity in RCTs and make results invalid and unreliable. In fact, biases can result in underestimation or overestimation of the effects of interventions in RCTs. Invalid and unreliable results of RCTs not only may not be beneficial, but also may endanger target patients' lives. Previous studies on the RCTs show that a large number of them have some types of biases.,
Biases in RCTs are of different types. The first type is selection bias, and the best method for its minimization is randomization, that is random allocation of participants to study groups. Randomization gives participants equal chance of being allocated to either of the study groups. It includes the two main steps of generating random allocation sequence and implementing the generated random allocation sequence. The second type of bias in RCTs is implementation bias. Blinding can be used to minimize this type of bias and produce more reliable results. Blinding reduces the negative effects of researchers' and participants' awareness of the intended intervention on the results. Moreover, blinding of those who assess the intended outcomes can reduce measurement bias. The third type of bias is related to the attrition of participants which increases the number of missing values and thereby, affects the true estimation of the effects of the intended intervention. Exclusion of participants with missing data from the study can lead to underestimation or overestimation of the effects of the intended intervention. The result of the study by Kim et al., which is a review article, aimed to investigate the effect acupuncture for treating uremic pruritus in patients with end-stage renal disease (ESRD) showed that most of the studies high risk of bias, which leaves their reports unconvincing. The current evidence is insufficient to show that acupuncture is an effective treatment for UP in patients with ESRD because of suboptimal quality and lack of methodological rigor of included studies. Future trials should overcome the limitations of the currently available.
Despite the importance of biases to the validity and reliability of RCTs, there is limited information about biases in CAM-related RCTs among hemodialysis patients. Therefore, the present study was undertaken to span this gap. The aim of the study was to assess biases in RCTs into the effects of CAM therapies on hemodialysis patients.
| Materials and Methods|| |
This is a critical review study that examines the quality of articles on RCTs into the effects of CAM therapies on hemodialysis patients. Inclusion criteria were an RCT design, a CAM therapy intervention, a sample of hemodialysis patients, accessible full-text, and publication in English or Persian in 2012--2018. Exit criteria include non-RCT articles and articles whose full text is not available.
An extensive literature by two researchers was conducted in online databases such as Scientific Information Database Iranmedex, Magiran, SID, Science Direct, Pubmed, Proquest, Scopus, and Web of Sciences Search key terms were RCT, controlled clinical trial, hemodialysis, Renal dialysis, complementary and alternative medicine, aromatherapy, massage, relaxation, herbal therapy, yoga and meditation, energy healing, acupressure, and homeopathy. Search operators such as “AND” and “OR” were used to combine search key terms. MESH was also used to identify keywords. To remove duplicate articles, initially, general information was collected from each included RCT about its title, year of publication, first author, study setting, journal, and so on. Then, by two researchers, they were evaluated for entry criteria which included a survey of the population studied, the type of intervention, and the comparison and outcome group. Aggregated studies entered the next stage, and rejected studies by both were discarded. In the case of studies in which there were disagreements about their entry between the two scholars, agreement was reached with the discussion. Then, the Risk of Bias Tool, developed by the Cochrane group, was employed to assess biases in the included RCTs. This tool assesses biases in RCTs in six main dimensions, namely random sequence generation, allocation concealment, blinding, attrition, selective outcome reporting, and other sources of bias. Each dimension was rated as either “Low bias,” “High bias,” or “Unclear,” The Cochrane Risk of Bias Tool is a valid and reliable tool for the assessment of all RCTs, irrespective of their publication place and language. The collected data were reported using the measures of descriptive statistics, namely absolute and relative frequencies.
Ethical Considerations: Loyalty in translation, lack of plagiarism, and respect for intellectual property rights.
| Results|| |
Initially, 261 RCTs were retrieved and briefly reviewed, from which 147 were ineligible or did not have accessible full-texts. Thus, those 147 RCTs were excluded and the remaining 114 RCTs (22 Iranian and 92 non-Iranian) were included and assessed for biases [Figure 1].
Around 71.05% of the included RCTs (81 cases) had used low-bias methods for random sequence generation. The most and the least commonly used methods were block randomization (45 cases) and coin flipping (three cases). On the other hand, 7.90% of RCTs (nine cases) had used high-bias random sequence generation methods such as day- or week-based allocation, while only 21.05% of them (24 cases) had not reported information on random sequence generation. Respecting allocation concealment, 60.52% of RCTs (69 cases) were rated as unclear for reporting no information, while 31.59% (36 cases) had used low-bias concealment methods and 7.89% (nine cases) had used high-bias methods.
Most included RCTs (92.1%) had used parallel designs and only three had used the crossover design. With respect to blinding, 57.89% of RCTs (66 cases) were low bias, that is, participants, staff, and researcher had been blinded. However, 39.47% of RCTs (45 cases) were high bias because of no blinding and 2.64% (three cases) were unclear and had not provided adequate explanations in this area. Moreover, 39.48% of RCTs (45 cases) had reported attrition rates of 15-35%, from which 73.33% (33 cases) had reported the number of and the reasons for dropouts in each group and hence, were considered as low bias. However, 26.67% of the RCTs with attrition (12 cases) had not reported any information about the number of and the reasons for dropouts and hence were considered as high bias. The remaining 60.52% of RCTs (69 cases) had reported no attrition.
Respecting selective outcome reporting, 84.21% of RCTs (96 cases) seemed to have reported all outcomes, while 15.79% (18 cases) either had not reported or incompletely reported some of the outcomes which had been introduced in their methods sections. Respecting other types of bias (such as providing no information about the validity and reliability of data collection tools, small sample size, and short-term intervention), 60.52% of RCTs (69 cases) were categorized as low bias and 39.48% (45 cases) as high bias [Figure 2].
| Discussion|| |
The aim of this study was to assess biases in CAM-related RCTs on hemodialysis patients. Study findings showed that although 21.05% of the included RCTs had reported random allocation, they had not provided adequate information about the process of randomization. In line with this finding, in the study which check out articles of RCTs published by Iranian researchers in field of obstetrics and gynecology in English journals indexed in first level valid bases, reported that 56% of RCTs included no information about the generation of random allocation sequence and the implementation of the generated sequence. some other studies also noted weaknesses in published RCTs respecting their randomization.,, It is noteworthy that poor randomization is not limited only to the publication phase of RCTs; rather, a study showed that around 60% of RCTs with unclear information about randomization in their final reports had also ambiguities in randomization in their draft proposals. It seems that researchers underestimate the importance of randomization to have equal groups and reduce biases. In order to produce credible results, researchers need to use effective randomization techniques such as tables of random numbers, block randomization, and randomization-related computer programs.
Our findings showed that respecting study design, 92.10% of RCTs had used parallel designs and only three had used other designs such as crossover. Two earlier studies that one reviewed all reports of RCTs of health care interventions and/or processes with individual randomization, published July–December 2004 in six major journals and another study all human subject randomized, controlled trials published in 4 leading urology journals in 1996 and 2004, also reported the same finding., Compared with other designs, parallel designs are more convenient to apply, need shorter amount of time, and bear lower costs and hence, are more frequently used by researchers. On the other hand, in the crossover design, study groups consist of the same individuals and the risk of bias is low. Therefore, this design is a powerful and reliable method to assess causal relationships. However, studies with crossover designs necessitate longer amounts of time.
Another finding of the present study was that 39.47% of RCTs (45 cases) had not used any method for blinding. Three earlier studies also reported that 50% of RCTs had not provided any information about blinding. This rate in another study which assess the bias in randomized controlled trials published in eight professional nursing and midwifery Iranian journals in 2010, was as high as 73.5%. A potential source of bias in RCTs is participants' or researchers' awareness of group assignment. Researchers' awareness of group assignment may result in their deliberate commitment of mistakes for the sake of the new treatment. The inability to refute this claim also undermines the validity of RCTs. Blinding helps effectively prevent these problems and their associated biases. Of course, most interventions cannot be easily concealed. For these interventions, those who assess intervention outcomes can be blinded. Researchers who fail to apply blinding to their RCTs need to provide detailed information about such failure and explain their strategies for reducing the associated biases.
Study findings also indicated that more than one third of RCTs (39.48%) had attrition between randomization and final follow-up assessment. Significant differences in the main outcomes between participants who withdraw from a study and those who complete the study can lead to overestimation or underestimation of the effects of the intended intervention. Moreover, any attrition reduces sample size and thereby the power of the study. When there is no relationship between attrition and exposure or outcome variables, attrition can be managed by increasing sample size; however, when this relationship is statistically significant, results are biased and conclusions may be erroneous.
In the present study, around 26.67% of RCTs with attrition had reported no information about attrition rate and reasons for attrition. This is congruent with the findings reported in some previous studies including evaluating the quality of 62 RCTs published from March 2015 to June 2016 in eight Persian nursing journals with at least 10 years of publishing history and another studying the quality of reports of randomized trials of physiotherapy interventions and another study which assess 124 RCTs published in 16 nursing journals in 2007 and 2008.,, Accurate information about attrition and its reasons and rational justifications for them can improve the quality of RCTs. A solution to attrition bias is the intention to treat analysis technique. The main component of this technique is to perform statistical analysis on all randomized participants irrespective of the intervention they receive, their adherence to the study protocol, and their withdrawal from the study. This technique adheres to the principles of randomization and hence protects RCTs against confounders and biases and helps produce quality evidence in clinical studies.
We also found that 15.79% of RCTs (18 cases) had either not reported or incompletely reported findings related to some outcome variables already introduced in their Methods sections. Two earlier studies including assessed all 314 abstracts of RCTs affiliated to Tehran University of Medical Sciences (n = 249) and Iran University of Medical Sciences (n = 65) indexed in PubMed up to the end of 2010 and another study assessment all clinical trials that had been conducted on humans and had a control group during 1999--2015 and were published in the Journal of Military Medicine were included in the current survey also reported that some RCTs had failed to provide detailed information about their outcome variables., The main reasons behind the exclusion of some outcomes are limited permissible word count for articles and clinical and statistical insignificance of some findings. It is noteworthy that statistically significant results have higher publication chance than statistically insignificant results. Therefore, most researchers may avoid reporting their statistically insignificant results. Registration of RCTs in RCT registries can help other researchers carefully assess any selective outcome reporting.
Respecting other types of bias such as small sample size, our findings showed that more than one third of RCTs had high bias. The study, Salesi et al., found that 82.85% of the trials had a sample size of less than 100. One reason behind small samples in some RCTs might have been inadequate financial support. Other types of bias in RCTs were related to the provision of incomplete information about the validity and reliability of data collection tools, shortness of follow-up assessment period, no consultation with biostatisticians in all steps of RCT, and failure to register RCT in national or international registries.
| Conclusion|| |
This study indicates that more than 50% of CAM-related RCTs on hemodialysis patients have low bias in all types bias. Nonetheless, strategies are needed to improve their quality. A good strategy is to design, conduct, and report RCTs based on standard guidelines. Another strategy can be training workshops for researchers to inform them about possible biases in RCTs and methods to their prevention or management. The other strategy is in-depth blind review of RCTs before publication in order to improve their quality.
In the present study, we only reviewed RCTs that were published in the field of complementary medicine on hemodialysis patients. Therefore, future studies are recommended to assess the bias in other patients. Also, there was a restriction in language that only articles evaluated in Persian and English was published.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Fructuoso M, Castro R, Oliveira L, Prata C, Morgado T. Quality of life in chronic kidney disease. Nefrologia 2011;31:91-6.
Hlubocky FJ, Ratain MJ, Wen M, Daugherty CK. Complementary and alternative medicine among advanced cancer patients enrolled on phase I trials: A study of prognosis, quality of life, and preferences for decision making. J Clin Oncol 2007;25:548-54.
Jocham A, Kriston L, Berberat PO, Schneider A, Linde K. How do medical students engaging in elective courses on acupuncture and homeopathy differ from unselected students? A survey. BMC Complement Altern Med 2017;17:148.
Sa'ed HZ, Al-Jabi SW, Sweileh WM, Tabeeb GH, Ayaseh NA, Sawafta MN, et al
. Use of complementary and alternative medicines in haemodialysis patients: A cross-sectional study from Palestine. BMC Complement Altern Med 2016;16:204.
Harris P, Cooper K, Relton C, Thomas K. Prevalence of complementary and alternative medicine (CAM) use by the general population: A systematic review and update. Int J Clin Pract 2012;66:924-39.
Bahall M. Use of complementary and alternative medicine by patients with end-stage renal disease on haemodialysis in Trinidad: A descriptive study. BMC Complement Altern Med 2017;17:250.
Sianesi B. “Randomisation bias” in the medical literature: A review. IFS Working Papers. 2016.
Talachi H, Orak R, Ravaghi H, Amanollahi A. Assessment of the quality of methodology reporting in the randomized trials. Journal of Health Administration 2012;15:81-92.
Paludan-Müller A, Laursen DRT, Hróbjartsson A. Mechanisms and direction of allocation bias in randomised clinical trials. BMC Med Res Methodol 2016;16:133.
Mayr A, Schmid M, Pfahlberg A, Uter W, Gefeller O. A permutation test to analyse systematic bias and random measurement errors of medical devices via boosting location and scale models. Stat Methods Med Res 2017;26:1443-60.
Yelland LN, Sullivan TR, Voysey M, Lee KJ, Cook JA, Forbes AB. Applying the intention-to-treat principle in practice: Guidance on handling randomisation errors. Clin Trials 2015;12:418-23.
Crocetti MT, Amin DD, Scherer R. Assessment of risk of bias among pediatric randomized controlled trials. Pediatrics 2010;126:298-305.
Mohammady M, Toghian Chaharsougi N, Abdoli S. Risk of bias in randomized controlled trials published in Iranian nursing and midwifery journals in 2010. Iran J Epidemiol 2014;9:24-36.
Jørgensen L, Paludan-Müller AS, Laursen DR, Savović J, Boutron I, Sterne JA, et al
. Evaluation of the cochrane tool for assessing risk of bias in randomized clinical trials: Overview of published comments and analysis of user practice in Cochrane and non-Cochrane reviews. Syst Rev 2016;5:80.
Chan AW, Tetzlaff JM, Altman DG, Laupacis A, Gøtzsche PC, Krleža-Jerić K, et al
. SPIRIT 2013 Statement: Defining standard protocol items for clinical trials. Rev Panam Salud Pública 2015;38:506-14.
Almasi-Hashiani A, Hassanzadeh J, Eshrati B, Khedmati E. An introduction to common systematic errors in medical research. Zahedan J Res Med Sci 2012;14:10-6.
Kim KH, Lee MS, Choi S-M, Ernst E. Acupuncture for treating uremic pruritus in patients with end-stage renal disease: A systematic review. J Pain Symptom Manage 2010;40:117-25.
Higgins JP, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, et al
. The cochrane collaboration's tool for assessing risk of bias in randomised trials. BMJ 2011;343:d5928.
Ghojazadeh M, Tavananezhad N, Karkhanee M, Behzad MN, Aghdash SA. Quality of randomized clinical trial reports published by iranian researchers in the obstetrics and gynecology level 1 journals: Using consort. Iran J Obstet Gynecol Infertil 2013;16:7-15.
Adib-Hajbaghery M, Adib M, Eshraghi Arani N. Evaluating the quality of randomized trials published in Persian nursing journals with more than 10 years of publishing using the CASP checklist. Iran J Nurs 2017;30:1-9.
Moseley AM, Elkins MR, Janer-Duncan L, Hush JM. The quality of reports of randomized controlled trials varies between subdisciplines of physiotherapy. Physiother Can 2014;6636-43.
Page MJ, McKenzie JE, Higgins JP. Tools for assessing risk of reporting biases in studies and syntheses of studies: A systematic review. BMJ Open 2018;8:e019703.
Devereaux P, Manns BJ, Ghali WA, Quan H, Guyatt GH. The reporting of methodological factors in randomized controlled trials and the association with a journal policy to promote adherence to the consolidated standards of reporting trials (CONSORT) checklist. Control Clin Trials 2002;23:380-8.
Suresh K. An overview of randomization techniques: An unbiased assessment of outcome in clinical research. J Hum Reprod Sci 2011;4:8-11.
] [Full text]
Toerien M, Brookes ST, Metcalfe C, De Salis I, Tomlin Z, Peters TJ, et al
. A review of reporting of participant recruitment and retention in RCTs in six major journals. Trials 2009;10:52.
Scales CD, Norris RD, Keitz SA, Peterson BL, Preminger GM, Vieweg J, et al
. A critical assessment of the quality of reporting of randomized, controlled trials in the urology literature. J Urol 2007;177:1090-5.
Wellek S, Blettner M. On the proper use of the crossover design in clinical trials: Part 18 of a series on evaluation of scientific publications. Deutsch črztebl Int 2012;109:276-81.
Hróbjartsson A, Thomsen ASS, Emanuelsson F, Tendal B, Hilden J, Boutron I, et al
. Observer bias in randomised clinical trials with binary outcomes: Systematic review of trials with both blinded and non-blinded outcome assessors. BMJ 2012;344:e1119.
Polit DF, Gillespie BM. The use of the intention-to-treat principle in nursing clinical trials. Nurs Res 2009;58:391-9.
Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: Updated guidelines for reporting parallel group randomised trials. BMC Med 2010;8:18.
Sainani KL. Making sense of intention-to-treat. PM and R 2010;2:209-13.
Amanollahi A, Shokraneh F, Mohammadhassanzadeh H, Ebrahimi KM, Banani G. Quality assessment of randomized controlled clinical trials indexed in PubMed using statement CONSORT. J Health Inf Manag 2012;9:406-15.
Salesi M, Maghari A, Mohammadi E, Yekaninejad M, Ghanbari A. Quality assessment of published randomized controlled trials in the journal of military medicine during 1999-2015. J Mil Med 2017;19:106-25.
Chan AW, Altman DG. Identifying outcome reporting bias in randomised trials on PubMed: Review of publications and survey of authors. BMJ 2005;330:753.
[Figure 1], [Figure 2]