Approaches to Assessing the Quality of Observational Studies of Clinical Practice Based on the Big Data Analysis
Abstract
The article is devoted to the discussion of the problems of assessing the quality of observational studies in real clinical practice and determining their place in the hierarchy of evidence-based information. The concept of “big data” and the acceptability of using such a term to refer to large observational studies is being discussed. Data on the limitations of administrative and claims databases when performing observational studies to assess the effects of interventions are presented. The concept of confounding factors influencing the results of observational studies is discussed. Modern approaches to reducing the severity of bias in real-life clinical practice studies are presented. The criteria for assessing the quality of observational pharmacoepidemiological studies and the fundamental differences between such studies and randomized clinical trials are presented. The results of systematic reviews of real-life clinical trials to assess the effects of direct oral anticoagulants are discussed.
About the Author
S. R. Gilyarevsky
Russian Medical Academy of Continuing Professional Education
Russian Federation
Russian Federation
Sergey R. Gilyarevsky
Moscow
References
1. U.S. Food and Drug Administration. Framework for FDA’s realworld evidence programa (2018) [cited by Jun 30, 2021. Available from: https://www.fda.gov/media/120060/download.
2. Berger ML, Doban V. Big data, advanced analytics and the future of comparative effectiveness research. J Comp Eff Res. 2014;3(2):167-76. DOI:10.2217/cer.14.2.
3. Okada M. Big data and real-world data-based medicine in the management of hypertension. Hypertens Res. 2021;44(2):147-53. DOI:10.1038/s41440-020-00580-3.
4. Hay SI, George DB, Moyes CL, Brownstein JS. Big data opportunities for global infectious disease surveillance. PLoS Med. 2013;10(4):e1001413. DOI:10.1371/journal.pmed.1001413.
5. Ola O, Sedig K. The challenge of big data in public health: an opportunity for visual analytics. Online J Public Health Inform. 2014;5(3):223. DOI:10.5210/ojphi.v5i3.4933.
6. Fanaroff AC, Califf RM, Lopes RD. New Approaches to Conducting Randomized Controlled Trials. J Am Coll Cardiol 2020;75:556-9. DOI:10.1016/j.jacc.2019.11.043.
7. Murray KW, Duggan A. Understanding Confounding in Research Pediatr Rev. 2010;31(3):124-6. DOI:10.1542/pir.31-3-124.
8. Meuli L, Dick F. Understanding Confounding in Observational Studies. Eur J Vasc Endovasc Surg. 2018;55(5):737. DOI:10.1016/j.ejvs.2018.02.028.
9. Fanaroff AC, Califf RM, Windecker S, et al. Levels of Evidence Supporting American College of Cardiology/American Heart Association and European Society of Cardiology Guidelines, 2008-2018. JAMA. 2019;321:1069-80. DOI:10.1001/jama.2019.1122.
10. Gilyarevsky SR, Gavrilov DV, Gusev AV. Retrospective analysis of electronic health records of patients with heart failure: the first Russian experience. Russian Journal of Cardiology. 2021;26(5):4502 (In Russ.). DOI:10.15829/1560-4071-2021-4502.
11. Althubaiti A. Information bias in health research: definition, pitfalls, and adjustment methods. J Multidiscip Healthc. 2016;9:211-7. DOI:10.2147/JMDH.S104807.
12. Matsumoto S, Fukui M, Hamaguchi M, et al. Is home blood pressure reporting in patients with type 2 diabetes reliable? Hypertens Res. 2014;37(8):741-5. DOI:10.1038/hr.2014.66.
13. Schneeweiss S, Rassen JA, Glynn RJ, et al. High-dimensional propensity score adjustment in studies of treatment effects using health care claims data. Epidemiology. 2009;20(4):512-22. DOI:10.1097/EDE.0b013e3181a663cc.
14. Cox DR, Kartsonaki C, Keogh RH. Big data: Some statistical issues. Stat Probab Lett. 2018;136:1115. DOI:10.1016/j.spl.2018.02.015.
15. Beaulieu-Jones BK, Finlayson SG, Yuan W, et al. Examining the Use of Real-World Evidence in the Regulatory Process. Clin Pharmacol Ther. 2020;107(4):843-52. DOI:10.1002/cpt.1658.
16. Langan SM, Schmidt SA, Wing K, et al. The reporting of studies conducted using observational routinely collected health data statement for pharmacoepidemiology (RECORD-PE). BMJ. 2018;363:k3532. DOI:10.1136/bmj.k3532.
17. Benchimol EI, Smeeth L, Guttmann A, et al.; RECORD Working Committee. The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement. PLoS Med. 2015;12(10):e1001885. DOI:10.1371/journal.pmed.1001885.
18. von Elm E, Altman DG, Egger M, et al.; STROBE Initiative. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet. 2007;370(9596):1453-7. DOI:10.1016/S0140-6736(07)61602-X.
19. Hemkens LG, Benchimol EI, Langan SM, et al. The reporting of studies using routinely collected health data was often insufficient. J Clin Epidemiol. 2016;79:104-11. DOI:10.1016/j.jclinepi.2016.06.005.
20. 20 Dreyer NA, Schneeweiss S, McNeil BJ, et al.; GRACE Initiative. GRACE principles: recognizing highquality observational studies of comparative effectiveness. Am J Manag Care. 2010;16(6):467-71.
21. European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (ENCePP). Guide on methodological standards in pharmacoepidemiology (2011) [cited by Jun 30, 2021. Available from: www.encepp.eu/standards_and_guidances/methodologicalGuide.shtml.
22. Blake KV, Devries CS, Arlett P, et al.; for the European Network of Centres for Pharmacoepidemiology Pharmacovigilance. Increasing scientific standards, independence and transparency in post-authorisation studies: the role of the European Network of Centres for Pharmacoepidemiology and Pharmacovigilance. Pharmacoepidemiol Drug Saf. 2012;21(7):690-6. DOI:10.1002/pds.3281.
23. Franklin JM, Schneeweiss S. When and How Can Real World Data Analyses Substitute for Randomized Controlled Trials? Clin Pharmacol Ther. 2017;102(6):924-33. DOI:10.1002/cpt.857.
24. Schneeweiss S. Improving therapeutic effectiveness and safety through big healthcare data. Clin Pharmacol Ther. 2016;99(3):262-5. DOI:10.1002/cpt.316
25. Sherman RE, Anderson SA, Dal Pan GJ, et al. Real-World Evidence What Is It and What Can It Tell Us? N Engl J Med. 2016;375(23):2293-7. DOI:10.1056/NEJMsb1609216.
26. Guimarães PO, Krishnamoorthy A, Kaltenbach LA, et al. Accuracy of Medical Claims for Identifying Cardiovascular and Bleeding Events After Myocardial Infarction: A Secondary Analysis of the TRANSLATE-ACS Study. JAMA Cardiol. 2017;2(7):750-7. DOI:10.1001/jamacardio.2017.1460.
27. Anglemyer A, Horvath HT, Bero L. Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials. In: The Cochrane Collaboration, ed. Cochrane Database of Systematic Reviews. Chichester, UK: John Wiley & Sons, Ltd; 2014 [cited by Jun 30, 2021. Available from: http://doi.wiley.com/10.1002/14651858.MR000034.pub2. Accessed May 12, 2016.
28. Hemkens LG, Contopoulos-Ioannidis DG, Ioannidis JPA. Agreement of treatment effects for mortality from routinely collected data and subsequent randomized trials: meta-epidemiological survey. BMJ. 2016;352:i493. DOI:10.1136/bmj.i493.
29. Dahabreh IJ, Sheldrick RC, Paulus JK, et al. Do observational studies using propensity score methods agree with randomized trials? A systematic comparison of studies on acute coronary syndromes. Eur Heart J. 2012;33(15):1893-901. DOI:10.1093/eurheartj/ehs114.
30. Stampfer MJ, Colditz GA, Willett WC, et al. Postmenopausal estrogen therapy and cardiovascular disease. Ten-year follow-up from the nurses' health study. N Engl J Med. 1991;325(11):756-62. DOI:10.1056/NEJM199109123251102.
31. Rossouw JE, Anderson GL, Prentice RL, et al.; Writing Group for the Women's Health Initiative Investigators. Risks and benefits of estrogen plus progestin in healthy postmenopausal women: principal results From the Women's Health Initiative randomized controlled trial. JAMA. 2002;288(3):32133. DOI:10.1001/jama.288.3.321.
32. Grodstein F, Manson JE, Stampfer MJ. Postmenopausal hormone use and secondary prevention of coronary events in the nurses' health study. a prospective, observational study. Ann Intern Med. 2001;135(1):1-8. DOI:10.7326/0003-4819-135-1-200107030-00003.
33. Hernán MA, Alonso A, Logan R, et al. Observational studies analyzed like randomized experiments: an application to postmenopausal hormone therapy and coronary heart disease. Epidemiology. 2008;19(6):766-79. DOI:10.1097/EDE.0b013e3181875e61.
34. Goodman SN, Schneeweiss S, Baiocchi M. Using Design Thinking to Differentiate Useful From Misleading Evidence in Observational Research. JAMA. 2017;317(7):705-7. DOI:10.1001/jama.2016.19970.
35. Schneeweiss S, Seeger JD, Landon J, Walker AM. Aprotinin during coronary-artery bypass grafting and risk of death. N Engl J Med. 2008;358(8):771-83. DOI:10.1056/NEJMoa0707571.
36. Fergusson DA, Hébert PC, Mazer CD, et al.; BART Investigators. A comparison of aprotinin and lysine analogues in high-risk cardiac surgery. N Engl J Med. 2008;358(22):2319-31. DOI:10.1056/NEJMoa0802395.
37. Agoritsas T, Merglen A, Shah ND, et al. Adjusted Analyses in Studies Addressing Therapy and Harm: Users' Guides to the Medical Literature. JAMA. 2017;317(7):748-59. DOI:10.1001/jama.2016.20029.
38. ISPOR Membership profile [cited by May 12, 2021. Available from: URL: https://www.ispor.org.
39. Proietti M, Romanazzi I, Romiti GF, et al. Real-World Use of Apixaban for Stroke Prevention in Atrial Fibrillation: A Systematic Review and Meta-Analysis. Stroke. 2018;49(1):98-106. DOI:10.1161/STROKEAHA.117.018395.
40. Escobar C, MartÍ-Almor J, Pérez Cabeza A, MartÍnez-Zapata MJ. Direct Oral Anticoagulants Versus Vitamin K Antagonists in Real-life Patients With Atrial Fibrillation. A Systematic Review and Meta-analysis. Rev Esp Cardiol (Engl Ed). 2019;72(4):305-16. DOI:10.1016/j.rec.2018.03.009.
Review
For citations:
Gilyarevsky S.R. Approaches to Assessing the Quality of Observational Studies of Clinical Practice Based on the Big Data Analysis. Rational Pharmacotherapy in Cardiology. 2021;17(4):584-593. (In Russ.) https://doi.org/10.20996/1819-6446-2021-08-01
Views: 329
Email this article 