Randomized controlled trials (RCTs) in the educational sciences in Latin America
DOI:
https://doi.org/10.64325/rhh00q09Keywords:
randomized controlled trials, educational research paradigms, Latin America, positivism, socio-critical paradigm, impact evaluation, educational policiesAbstract
Randomized controlled trials (RCTs) have emerged as a methodology of increasing relevance in the educational sciences, particularly following the international recognition bestowed with the 2019 Nobel Prize in Economics upon Abhijit Banerjee, Esther Duflo, and Michael Kremer. This article presents a comprehensive analysis of the paradigmatic positioning of RCTs within educational research in Latin America, examining their epistemological foundations, their practical application, and the critiques formulated from various theoretical perspectives. Through a systematic literature review, the tension between the positivist-postpositivist paradigm, which provides the philosophical underpinnings for RCTs, and the socio-critical paradigm, predominant in the Latin American research tradition, is analyzed. The results reveal that, while RCTs offer rigorous evidence for specific and limited interventions, they present significant limitations in capturing the complexity of educational processes in contexts of structural inequality. It is concluded that methodological integration, rather than exclusive adherence to a paradigm, constitutes the most promising path to advance towards educational research that is simultaneously rigorous, contextually relevant, and socially committed.
Downloads
References
Brookings Institution. (2019). Changing the culture of economics: The 2019 Nobel laureates. https://www.brookings.edu/articles/changing-the-culture-of-economics-the-2019-nobel-laureates
Connolly, P., Keenan, C., & Urbanska, K. (2018). The trials of evidence-based practice in education: A systematic review of randomised controlled trials in education research 1980–2016. Educational Research, 60(3), 276-291. https://doi.org/10.1080/00131881.2018.1493353 DOI: https://doi.org/10.1080/00131881.2018.1493353
Floretta, J., Friedlander, S., & Dhaliwal, I. (2020). Beyond Randomized Controlled Trials. Stanford Social Innovation Review. https://doi.org/10.48558/XCWR-CW33
Freire, P. (1970). Pedagogía del oprimido. Siglo XXI Editores.
Frontiers in Education. (2022). Evidence-based education: Objections and future directions. https://www.frontiersin.org/articles/10.3389/feduc.2022.941410
Gorard, Stephen & See, Beng & Siddiqui, Nadia. (2017). The trials of evidence-based education: The Promises, Opportunities and Problems of Trials in Education. 10.4324/9781315456898 DOI: https://doi.org/10.4324/9781315456898
Hernández, R., & Mendoza, C. (2023). Metodología de la investigación. Las rutas cuantitativa, cualitativa y mixta (2da ed.). Ciudad de México: McGraw Hill. https://drive.usercontent.google.com/download?id=1JChN0wmF9WC57UOu_oRGXrahiy3Czw5V&export=download&authuser=0
Hernández, R., Fernández, C., & Baptista, P. (2018). Metodología de la investigación (6ta ed.). México D.F, México, México: McGraw-Hill. https://www.esup.edu.pe/wp-content/uploads/2020/12/2.%20Hernandez,%20Fernandez%20y%20Baptista-Metodolog%C3%ADa%20Investigacion%20Cientifica%206ta%20ed.pdf
J-PAL. (2024). Abdul Latif Jameel Poverty Action Lab. https://www.povertyactionlab.org
Kaplan, A., Cromley, J., Perez, T., Dai, T., Mara, K., & Balsai, M. (2020). The role of context in educational RCT findings: A call to redefine “evidence-based practice”. Educational Researcher, 49(4), 285-288. https://doi.org/10.3102/0013189X20921862 DOI: https://doi.org/10.3102/0013189X20921862
Kuhn, T. S. (1962). The structure of scientific revolutions. University of Chicago Press.
Lortie-Forgues, H., & Inglis, M. (2019). Rigorous large-scale educational RCTs are often uninformative: Should we be concerned? Educational Researcher, 48(3), 158-166. https://doi.org/10.3102/0013189X19832850 DOI: https://doi.org/10.3102/0013189X19832850
Magrath, B., Aslam, M., & Johnson, D. (2019). Systems research in education: Designs and methods. Research in Comparative and International Education, 14 (1). https://doi.org/10.1177/1745499919828927 DOI: https://doi.org/10.1177/1745499919828927
Martín Martínez, C. (2020). Efecto de intervenciones de actividad física en el ámbito escolar sobre las habilidades lingüísticas: una revisión sistemática y meta-análisis de ensayos controlados aleatorizados. Tesis de Grado. Universidad Complutense de Madrid. https://hdl.handle.net/20.500.14352/8912
Nobel Prize. (2019). Understanding development and poverty: The experimental approach. https://www.nobelprize.org/uploads/2019/10/advanced-economicsciencesprize2019.pdf
O’Connor, J. (2024). Randomised Controlled Trials in Education: An Experiment Worth Taking? Irish Journal of Education / Iris Eireannach an Oideachais, 48, 1–27. https://www.jstor.org/stable/27332408
Odgaard-Jensen, J., Vist, G. E., Timmer, A., Kunz, R., Akl, E. A., Schünemann, H., Briel, M., Nordmann, A. J., Pregno, S., & Oxman, A. D. (2011). Randomisation to protect against selection bias in healthcare trials. Cochrane Database of Systematic Reviews. https://doi.org/10.1002/14651858.MR000012.pub3 DOI: https://doi.org/10.1002/14651858.MR000012.pub3
Odom, S. L. (2021). Education of students with disabilities, science, and randomized controlled trials. Research and Practice for Persons with Severe Disabilities, 46(3), 132-145. Odom, S. L. (2021). Education of Students with Disabilities, Science, and Randomized Controlled Trials. Research and Practice for Persons with Severe Disabilities, 46(3), 132-145. https://doi.org/10.1177/15407969211032341 DOI: https://doi.org/10.1177/15407969211032341
Olken, B.A. (2020), Banerjee, Duflo, Kremer, and the Rise of Modern Development Economics. Scand. J. of Economics, 122: 853-878. https://doi.org/10.1111/sjoe.12418 DOI: https://doi.org/10.1111/sjoe.12418
Parra, J. D., & Edwards Jr, D. B. (2024). Challenging the gold standard consensus: Randomised controlled trials (RCTs) and their pitfalls in evidence-based education. Critical Studies in Education, 65(5), 513-530. https://doi.org/10.1080/17508487.2024.2314118 DOI: https://doi.org/10.1080/17508487.2024.2314118
Popper, K. (1959). The logic of scientific discovery. Hutchinson. DOI: https://doi.org/10.1063/1.3060577
Puente, J., Lucero, E., Díaz, L., Manuel, J., Ramírez, Q., & Román, D. (2024). Herramientas y metodologías para la evaluación científica: bibliometría, cienciometría e informetría. Seminars in Medical Writing and Education, 3(395), 395. https://doi.org/https://doi.org/10.56294/mw2024395 DOI: https://doi.org/10.56294/mw2024395
Ramírez, J. V., Robles, C. C., Macías, L. P., & Vega, D. M. (2020). Una unidad didáctica de Educación Deportiva como estrategia de educación para la ciudadanía en Educación Física: Un ensayo controlado aleatorizado por grupos. Retos: nuevas tendencias en educación física, deporte y recreación, (38), 44-52. https://dialnet.unirioja.es/servlet/articulo?codigo=7397350&orden=0&info=link
Ramírez-Rivera, D. L., Martínez-Contreras, T., Ruelas, A. L., Henry-Mejia, G., Quizán-Plata, T., Esparza-Romero, J., ... & Díaz-Zavala, R. G. (2021). Efecto de una intervención en línea de cambio en el estilo de vida sobre el puntaje zIMC de escolares mexicanos: protocolo de ensayo controlado aleatorizado piloto cegado a evaluadores durante la pandemia por COVID-19. Revista española de nutrición humana y dietética, 25. https://doi.org/10.14306/renhyd.25.S2.1320 DOI: https://doi.org/10.14306/renhyd.25.S2.1320
Román, D. (2024). Terminología en metodología de la investigación cualitativa. In Seminars in Medical Writing and Education, 3(655), 655. https://doi.org/https://doi.org/10.56294/mw2024655 DOI: https://doi.org/10.56294/mw2024655
Rowe, M., & Oltmann, C. (2016). Randomised controlled trials in educational research: Ontological and epistemological limitations. African Journal of Health Professions Education, 8(1), 6–8. Retrieved from https://www.ajol.info/index.php/ajhpe/article/view/135389 DOI: https://doi.org/10.7196/AJHPE.2016.v8i1.683
Shalem, Y., & De Clercq, F. (2022). Randomized control trials in education (RCTs): What is their contribution to education theory about teaching?. Journal of Education (University of KwaZulu-Natal), (89), 3-22. https://doi.org/10.17159/2520-9868/i89a01 DOI: https://doi.org/10.17159/2520-9868/i89a01
Slavin, R. E. (2020). How evidence-based reform will transform research and practice in education. Educational Psychologist, 55(1), 21–31. https://doi.org/10.1080/00461520.2019.1611432 DOI: https://doi.org/10.1080/00461520.2019.1611432
Smith, J., & Flanagan, S. (2024, 25 de septiembre). Repensar los ensayos controlados aleatorios. Punto de Vista Medición y Evaluación SSIR, (13). https://ssires.tec.mx/es/noticia/repensar-los-ensayos-controlados-aleatorios
Spybrook, J., Zhang, Q., Kelcey, B., & Dong, N. (2020). Learning from cluster randomized trials in education: An assessment of the capacity of studies to determine what works, for whom, and under what conditions. Educational Evaluation and Policy Analysis, 42(3), 354–374. https://doi.org/10.3102/0162373720929018 DOI: https://doi.org/10.3102/0162373720929018
Staines, L., Healy, C., Corcoran, P., Keeley, H., Coughlan, H., McMahon, E., Cotter, P., Cotter, D., Kelleher, I., Wasserman, C., Brunner, R., Kaess, M., Sarchiapone, M., Hoven, C. W., Carli, V., Wasserman, D., & Cannon, M. (2023). Investigating the effectiveness of three school-based interventions for preventing psychotic experiences over a year period – a secondary data analysis study of a randomized controlled trial. BMC Public Health, 23, 219. https://doi.org/10.1186/s12889-023-15107-x DOI: https://doi.org/10.1186/s12889-023-15107-x
Styles, B., & Torgerson, C. (2018). Randomised controlled trials (RCTs) in education research–methodological debates, questions, challenges. Educational Research, 60(3), 255-264.
Styles, B., & Torgerson, C. (2018). Randomised controlled trials (RCTs) in education research –methodological debates, questions, challenges. Educational Research, 60(3), 255–264. https://doi.org/10.1080/00131881.2018.1500194 DOI: https://doi.org/10.1080/00131881.2018.1500194
The Conversation. (2019). 2019 Nobel Prize in Economics: The limits of the clinical trial method. https://theconversation.com/2019-nobel-prize-in-economics-the-limits-of-the-clinical-trial-method-125888
Tyler, C., Akerlof, K. L., Allegra, A., Arnold, Z., Canino, H., Doornenbal, M. A., Goldstein, J. A., Pedersen, D. B., & Sutherland, W. J. (2023). AI tools as science policy advisers? The potential and the pitfalls. Nature, 622, 27–30. https://doi.org/10.1038/d41586-023-02999-3 DOI: https://doi.org/10.1038/d41586-023-02999-3
UNAM. (2004). Implicaciones de los paradigmas de investigación. https://www.revista.unam.mx/vol.5/num1/art1/ene_art1.pdf
UNESCO. (2023). Global education monitoring report 2023 – Technology in education: A tool on whose terms? UNESCO. https://www.unesco.org/gem-report/en/technology
Van Damme, D. (2022). The power of proofs: (Much) beyond RCTs. Centre for Curriculum Redesign. https://curriculumredesign.org/wp-content/uploads/The-Power-of-Proofs.pdf
Welsh, R. O. (2021). Assessing the quality of education research through its relevance to practice: An integrative review of research-practice partnerships. Review of Research in Education, 45(1), 170–194. https://doi.org/10.3102/0091732X20985082 DOI: https://doi.org/10.3102/0091732X20985082
What Works Clearinghouse. (2022). What Works Clearinghouse: Procedures and standards handbook, version 5.0. U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance (NCEE). https://ies.ed.gov/ncee/wwc/Handbooks
Wolf, B., & Harbatkin, E. (2023). Making sense of effect sizes: Systematic differences in intervention effect sizes by outcome measure type. Journal of Research on Educational Effectiveness, 16(1), 134–161. https://doi.org/10.1080/19345747.2022071364 DOI: https://doi.org/10.1080/19345747.2022.2071364
World Bank. (2019). A Nobel Prize for development RCTs! https://blogs.worldbank.org/en/impactevaluations/nobel-prize-development-rcts
Downloads
Published
Issue
Section
License
Copyright (c) 2024 Jose Humberto Puente (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish in this journal agree to the following terms:
a) Authors retain copyright and grant the journal the right of first publication, with the work licensed under a Creative Commons Attribution 4.0 license, which allows third parties to use the published work provided they attribute the authorship of the work and acknowledge its first publication in this journal.
b) Authors may enter into other independent and additional contractual agreements for the non-exclusive distribution of the version of the article published in this journal (e.g., including it in an institutional repository or publishing it in a book) provided they clearly indicate that the work was first published in this journal.
c) Authors are permitted and encouraged to share their work online (e.g., in institutional repositories or on personal websites) before and during the manuscript submission process, as this can lead to productive exchanges and greater and faster citation of the published work.
