Schlagworte
Suchstrategie
Systematic Review
Zitationsvorschlag
Lizenz
Copyright (c) 2023 Katja Buntins, Miriam Mulders, Nadine Schroeder
Dieses Werk steht unter der Lizenz Creative Commons Namensnennung 4.0 International.
Abstract
Ziel dieser Studie ist es, die Vor- und Nachteile verschiedener Suchstrategien bei der Synthese von Forschungsarbeiten, die die Bildungstechnologie der virtuellen Realität (VR) nutzen, empirisch zu untersuchen. Hierbei ist das Ziel, kognitive Verzerrungen seitens der Reviewer:innen durch verschieden konkrete Suchen zu identifizieren. Mittels zweier Suchstrategien sollen die Extrema zwischen einer Suche (AND), die möglichst wenig irrelevante Studien findet, aber dafür auch relevante übersieht und einer Suche (OR), die möglichst breit sucht, aber hierbei viele irrelevante aufnimmt, dargestellt werden. Die Studie will aufzeigen, wie systematische Suchen in der Bildungsforschung gestaltet sein sollten, um die typischen Herausforderungen systematischer Analysen (z.B. Recall-Precision-Problem, kognitive Belastung) adäquat zu berücksichtigen. Die Suchstrategien wurden auf der Grundlage einer vorangegangenen Google Scholar-Suche nach bereits durchgeführten systematischen Übersichten zur VR entwickelt. Hierbei unterschieden sich die zwei verschiedenen Suchstrategien nur in Bezug auf ihre Verknüpfung zwischen einem technologischem (VR) und einem pädagogischen Suchterm. Die beiden Elemente wurden entweder mit einer AND oder einer OR Verbindung verknüpft. Die Suchbeitrage wurden in einem Kreuzdesign von zwei Personen gescreent und in Bezug auf verschiedene Präzisions- und Recallmaße evaluiert. Es fanden sich keine Hinweise dafür, dass die umfangreichere Suche (OR) der engeren Suche (AND) überlegen ist und jedoch leichte Hinweise auf kognitive Verzerrungen im Screening bzw. Suchprozess bei der umfangreicheren Suche (OR). Diese sollten in weiteren Studien weiter evaluiert, untersucht und vor allem repliziert werden.
Literatur
Akers, Jo, R. Aguiar-Ibáñez, and A. Baba-Akbari. 2009. Systematic Reviews: CRD’s Guidance for Undertaking Reviews in Health Care. York, UK: Centre for Reviews and Dissemination, University of York.
Allcoat, Devon, and Adrian von Mühlenen. 2018. “Learning in Virtual Reality: Effects on Performance, Emotion and Engagement”. Research in Learning Technology 26. https://doi.org/10.25304/rlt.v26.2140.
Alvarez, R. Michael, and Carla VanBeselaere. 2005. “Web-Based Survey”. In Encyclopedia of Social Measurement. Edited by Kimberly Kempf-Leonard, 955–62. New York: Elsevier. https://doi.org/10.1016/B0-12-369398-5/00390-X.
Andrews, Frank M., and George F. Farris. 1972. “Time Pressure and Performance of Scientists and Engineers: A Five-Year Panel Study”. Organizational Behavior and Human Performance 8 (2): 185–200. https://doi.org/10.1016/0030-5073(72)90045-1.
Bedenlier, Svenja, Melissa Bond, Katja Buntins, Olaf Zawacki-Richter, and Michael Kerres. 2020. “Learning by Doing? Reflections on Conducting a Systematic Review in the Field of Educational Technology”. In Systematic Reviews in Educational Research: Methodology, Perspectives and Application, edited by Olaf Zawacki-Richter et al., 111–27. Wiesbaden: Springer Fachmedien.
Bond, Melissa, Katja Buntins, Svenja Bedenlier, Olaf Zawacki-Richter, and Michael Kerres. 2020. “Mapping Research in Student Engagement and Educational Technology in Higher Education: A Systematic Evidence Map”. International Journal of Educational Technology in Higher Education 17 (1): 2. https://doi.org/10.1186/s41239-019-0176-8.
Bramer, Wichor M., Dean Giustini, Bianca Kramer, and P. F. Anderson. 2013. “The Comparative Recall of Google Scholar Versus PubMed in Identical Searches for Biomedical Systematic Reviews: A Review of Searches Used in Systematic Reviews”. Systematic reviews 2 (1): 115. https://doi.org/10.1186/2046-4053-2-115.
Bramer, Wichor M., Dean Giustini, and Bianca M. R. Kramer. 2016. “Comparing the Coverage, Recall, and Precision of Searches for 120 Systematic Reviews in Embase, MEDLINE, and Google Scholar: A Prospective Study”. Systematic reviews 5 (1): 39. https://doi.org/10.1186/s13643-016-0215-7.
Buchner, Josef. 2022. “Systematic Reviews Als Analyseinstrument Der Forschungspraxis in Educational Technology Studien”. Presentation at DGfE-Kongress 2022. https://doi.org/10.13140/RG.2.2.19953.35687.
Buchner, Josef, Katja Buntins, and Michael Kerres. 2021. “A Systematic Map of Research Characteristics in Studies on Augmented Reality and Cognitive Load: A Systematic Map of Research Characteristics”. Computers and Education Open 2: 100036. https://doi.org/10.1016/j.caeo.2021.100036.
Cooke, Alison, Debbie Smith, and Andrew Booth. 2012. “Beyond PICO: The SPIDER Tool for Qualitative Evidence Synthesis”. Qualitative Health Research 22 (10): 1435–43. https://doi.org/10.1177/1049732312452938.
Cooper, Chris, Rebecca Lovell, Kerryn Husk, Andrew Booth, and Ruth Garside. 2018. “Supplementary Search Methods Were More Effective and Offered Better Value Than Bibliographic Database Searching: A Case Study from Public Health and Environmental Enhancement”. Research synthesis methods 9 (2): 195–223. https://doi.org/10.1002/jrsm.1286.
Cooper, Chris, Joanna Varley-Campbell, Andrew Booth, Nicky Britten, and Ruth Garside. 2018. “Systematic Review Identifies Six Metrics and One Method for Assessing Literature Search Effectiveness but No Consensus on Appropriate Use”. Journal of Clinical Epidemiology 99: 53–63. https://doi.org/10.1016/j.jclinepi.2018.02.025.
Eden, Dov. 2002. “From the Editors”. Academy of Management Journal 45 (5): 841–46. https://doi.org/10.5465/amj.2002.7718946.
Engström, Johan, Gustav Markkula, Trent Victor, and Natasha Merat. 2017. “Effects of Cognitive Load on Driving Performance: The Cognitive Control Hypothesis”. Human Factors 59 (5): 734–64. https://doi.org/10.1177/0018720817690639.
Evans, David. 2002. “Database Searches for Qualitative Research”. Journal of the Medical Library Association JMLA 90 (3): 290–93. https://pubmed.ncbi.nlm.nih.gov/12113512.
Eysenbach, G., J. Tuische, and T. L. Diepgen. 2001. “Evaluation of the Usefulness of Internet Searches to Identify Unpublished Clinical Trials for Systematic Reviews”. Medical informatics and the Internet in medicine 26 (3): 203–18. https://doi.org/10.1080/14639230110075459.
Furlan, Andrea D., Emma Irvin, and Claire Bombardier. 2006. “Limited Search Strategies Were Effective in Finding Relevant Nonrandomized Studies”. Journal of Clinical Epidemiology 59 (12): 1303–11. https://doi.org/10.1016/j.jclinepi.2006.03.004.
Geersing, Geert-Jan, Walter Bouwmeester, Peter Zuithoff, Rene Spijker, Mariska Leeflang, and Karel Moons. 2012. “Search Filters for Finding Prognostic and Diagnostic Prediction Studies in Medline to Enhance Systematic Reviews”. PloS one 7 (2): e32844. https://doi.org/10.1371/journal.pone.0032844.
Gehanno, Jean-François, Laetitia Rollin, and Stefan Darmoni. 2013. “Is the Coverage of Google Scholar Enough to Be Used Alone for Systematic Reviews”. BMC Medical Informatics and Decision Making 13 (1): 7. https://doi.org/10.1186/1472-6947-13-7.
Gough, David, Sandy Oliver, and James Thomas. 2017. An Introduction to Systematic Reviews. 2nd revised edition. Los Angeles: SAGE.
Grant, Maria J., and Andrew Booth. 2009. “A Typology of Reviews: An Analysis of 14 Review Types and Associated Methodologies”. Health information and libraries journal 26 (2): 91–108. https://doi.org/10.1111/j.1471-1842.2009.00848.x.
Grossetta Nardini, Holly K., Janene Batten, Melissa C. Funaro, Rolando Garcia-Milian, Kate Nyhan, Judy M. Spak, Lei Wang, and Janis G. Glover. 2019. “Librarians as Methodological Peer Reviewers for Systematic Reviews: Results of an Online Survey”. Research Integrity and Peer Review 4 (1): 23. https://doi.org/10.1186/s41073-019-0083-5.
Gusenbauer, Michael, and Neal R. Haddaway. 2020. “Which Academic Search Systems Are Suitable for Systematic Reviews or Meta-Analyses? Evaluating Retrieval Qualities of Google Scholar, PubMed, and 26 Other Resources”. Research synthesis methods 11 (2): 181–217. https://doi.org/10.1002/jrsm.1378.
Haddaway, Neal R., and Helen R. Bayliss. 2015. “Shades of Grey: Two Forms of Grey Literature Important for Reviews in Conservation”. Biological Conservation 191: 827–29. https://doi.org/10.1016/j.biocon.2015.08.018.
Haddaway, Neal R., Alexandra Mary Collins, Deborah Coughlin, and Stuart Kirk. 2015. “The Role of Google Scholar in Evidence Reviews and Its Applicability to Grey Literature Searching”. PloS one 10 (9): e0138237. https://doi.org/10.1371/journal.pone.0138237.
Jensen, Lasse, and Flemming Konradsen. 2018. “A Review of the Use of Virtual Reality Head-Mounted Displays in Education and Training”. Education and Information Technologies 23 (4): 1515–29. https://doi.org/10.1007/s10639-017-9676-0.
Kavanagh, Sam, Andrew Luxton-Reilly, Burkhard Wuensche, and Beryl Plimmer. 2017. “A Systematic Review of Virtual Reality in Education”. Themes in Science and Technology Education 10 (2): 85–119. https://www.learntechlib.org/p/182115/.
Konig, Cornelius J., Markus Buhner, and Gesine Murling. 2005. “Working Memory, Fluid Intelligence, and Attention Are Predictors of Multitasking Performance, but Polychronicity and Extraversion Are Not”. Human Performance 18 (3): 243–66. https://doi.org/10.1207/s15327043hup1803_3.
Köstler, Verena. 2023. “Zwischen Präzision und Sensitivität: Generierung eines Studienkorpus am Beispiel einer Fragestellung zu Künstlicher Intelligenz (KI) in Bildungsprozessen”. MedienPädagogik: Zeitschrift für Theorie und Praxis der Medienbildung 54 (Research Syntheses): 1–27. https://doi.org/10.21240/mpaed/54/2023.08.10.X.
Kupper, Lawrence L., and Kerry B. Hafner. 1989. “How Appropriate Are Popular Sample Size Formulas?”. The American Statistician 43 (2): 101. https://doi.org/10.2307/2684511.
Larsen, Peder Olesen, and Markus von Ins. 2010. “The Rate of Growth in Scientific Publication and the Decline in Coverage Provided by Science Citation Index”. Scientometrics 84 (3): 575–603. https://doi.org/10.1007/s11192-010-0202-z.
Lasser, Jana, Verena Ahne, Georg Heiler, Peter Klimek, Hannah Metzler, Tobias Reisch, Martin Sprenger, Stefan Thurner, and Johannes Sorger. 2020. “Complexity, transparency and time pressure: practical insights into science communication in times of crisis”. Journal of Science Communication 19 (5): N01. https://doi.org/10.22323/2.19050801.
Lee, Edwin, Maureen Dobbins, Kara DeCorby, Lyndsey McRae, Daiva Tirilis, and Heather Husson. 2012. “An Optimal Search Filter for Retrieving Systematic Reviews and Meta-Analyses”. BMC medical research methodology 12 (1): 51. https://doi.org/10.1186/1471-2288-12-51.
Liu, Dejian, Kaushal Kumar Bhagat, Yuan Gao, Ting-Wen Chang, and Ronghuai Huang. 2017. “The Potentials and Trends of Virtual Reality in Education”. In Virtual, Augmented, and Mixed Realities in Education, 105–30. Springer, Singapore. https://doi.org/10.1007/978-981-10-5490-7_7.
Madigan, Ruth, and Richard Romano. 2020. “Does the Use of a Head Mounted Display Increase the Success of Risk Awareness and Perception Training (RAPT) For Drivers?” Applied ergonomics 85: 103076. https://doi.org/10.1016/j.apergo.2020.103076.
Makransky, Guido, and Gustav B. Petersen. 2021. “The Cognitive Affective Model of Immersive Learning (CAMIL): A Theoretical Research-Based Model of Learning in Immersive Virtual Reality”. Educational Psychology Review 33 (3): 937–58. https://doi.org/10.1007/s10648-020-09586-2.
Martín-Martín, Alberto, Mike Thelwall, Enrique Orduna-Malea, and Emilio Delgado López-Cózar. 2021. “Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations’ COCI: A Multidisciplinary Comparison of Coverage via Citations”. Scientometrics 126 (1): 871–906. https://doi.org/10.1007/s11192-020-03690-4.
Merchant, Zahira, Ernest T. Goetz, Lauren Cifuentes, Wendy Keeney-Kennicutt, and Trina J. Davis. 2014. “Effectiveness of Virtual Reality-Based Instruction on Students’ Learning Outcomes in K-12 and Higher Education: A Meta-Analysis”. Computers & Education 70: 29–40. https://doi.org/10.1016/j.compedu.2013.07.033.
Methley, Abigail M., Stephen Campbell, Carolyn Chew-Graham, Rosalind McNally, and Sudeh Cheraghi-Sohi. 2014. “PICO, PICOS and SPIDER: A Comparison Study of Specificity and Sensitivity in Three Search Tools for Qualitative Systematic Reviews”. BMC Health Services Research 14 (1): 579. https://doi.org/10.1186/s12913-014-0579-0.
Mulders, Miriam, Josef Buchner, and Michael Kerres. 2020. “A Framework for the Use of Immersive Virtual Reality in Learning Environments”. International Journal of Emerging Technologies in Learning (iJET) 15 (24): 208–24. https://doi.org/10.3991/ijet.v15i24.16615.
Radianti, Jaziar, Tim A. Majchrzak, Jennifer Fromm, and Isabell Wohlgenannt. 2020. “A Systematic Review of Immersive Virtual Reality Applications for Higher Education: Design Elements, Lessons Learned, and Research Agenda”. Computers & Education 147: 103778. https://doi.org/10.1016/j.compedu.2019.103778.
Rehman, Yasir. 2021. “Difference Between Quantitative and Qualitative Research Question-PICO Vs. SPIDER”. American Academic Scientific Research Journal for Engineering, Technology, and Sciences 77 (1): 188–99. https://asrjetsjournal.org/index.php/American_Scientific_Journal/article/view/6730.
Robinson, Karen A., Ian J. Saldanha, and Naomi A. Mckoy. 2011. “Development of a Framework to Identify Research Gaps from Systematic Reviews”. Journal of Clinical Epidemiology 64 (12): 1325–30. https://doi.org/10.1016/j.jclinepi.2011.06.009.
Rogers, Morwenna, Alison Bethel, and Kate Boddy. 2017. “Development and Testing of a Medline Search Filter for Identifying Patient and Public Involvement in Health Research”. Health Information & Libraries Journal 34 (2): 125–33. https://doi.org/10.1111/hir.12157.
Salvador-Oliván, José Antonio, Gonzalo Marco-Cuenca, and Rosario Arquero-Avilés. 2019. “Errors in Search Strategies Used in Systematic Reviews and Their Effects on Information Retrieval”. Journal of the Medical Library Association JMLA 107 (2): 210–21. https://doi.org/10.5195/jmla.2019.567.
Sampson, Margaret, and Jessie McGowan. 2006. “Errors in Search Strategies Were Identified by Type and Frequency”. Journal of Clinical Epidemiology 59 (10): 1057.e1-1057.e9. https://doi.org/10.1016/j.jclinepi.2006.01.007.
Sampson, Margaret, Jennifer Tetzlaff, and Christine Urquhart. 2011. “Precision of Healthcare Systematic Review Searches in a Cross-Sectional Sample”. Research synthesis methods 2 (2): 119–25. https://doi.org/10.1002/jrsm.42.
Schnotz, Wolfgang, and Christian Kürschner. 2007. “A Reconsideration of Cognitive Load Theory”. Educational Psychology Review 19 (4): 469–508. https://doi.org/10.1007/s10648-007-9053-4.
Shaffril, Mohamed, Hayrol Azril, Samsul Farid Samsuddin, and Asnarulkhadi Abu Samah. 2021. “The ABC of Systematic Literature Review: The Basic Methodological Guidance for Beginners”. Quality & Quantity 55 (4): 1319–46. https://doi.org/10.1007/s11135-020-01059-6.
Shepperd, Martin, David Bowes, and Tracy Hall. 2014. “Researcher Bias: The Use of Machine Learning in Software Defect Prediction”. IEEE Transactions on Software Engineering 40 (6): 603–16. https://doi.org/10.1109/TSE.2014.2322358.
Stock, Wolfgang G., and Mechtild Stock. 2013. Handbook of Information Science. Berlin, Boston: De Gruyter Saur. https://doi.org/10.1515/9783110235005.
Straube, S., J. Heinz, P. Landsvogt, and T. Friede. 2021. “Recall, Precision, and Coverage of Literature Searches in Systematic Reviews in Occupational Medicine: An Overview of Cochrane Reviews Recall, Precision Und Coverage Von Literatursuchen in Systematischen Reviews Aus Dem Bereich Arbeitsmedizin: Ein Überblick Über Cochrane Reviews”. GMS Medizinische Informatik, Biometrie und Epidemiologie 17 (1). https://doi.org/10.3205/mibe000216.
Süß, Heinz-Martin, and Florian Schmiedek. 2000. “Ermüdungs- Und Übungseffekte Bei Mehrstündiger Kognitiver Beanspruchung”. Experimental Psychology 47 (3): 162–79. https://doi.org/10.1026//0949-3964.47.3.162.
Sweller, John. 1988. “Cognitive Load During Problem Solving: Effects on Learning”. Cognitive Science 12 (2): 257–85. https://doi.org/10.1016/0364-0213(88)90023-7.
Sweller, John. 2003. “Evolution of Human Cognitive Architecture”. Psychology of learning and motivation 43: 216–66.
Sweller, John, Jeroen J. G. van Merriënboer, and Fred Paas. 2019. “Cognitive Architecture and Instructional Design: 20 Years Later”. Educational Psychology Review 31 (2): 261–92. https://doi.org/10.1007/s10648-019-09465-5.
Wu, Bian, Xiaoxue Yu, and Xiaoqing Gu. 2020. “Effectiveness of Immersive Virtual Reality Using Head-mounted Displays on Learning Performance: A Meta-analysis”. British Journal of Educational Technology 51 (6): 1991–2005. https://doi.org/10.1111/bjet.13023.
Zhang, Fan, Shamila Haddad, Bahareh Nakisa, Mohammad Naim Rastgoo, Christhina Candido, Dian Tjondronegoro, and Richard de Dear. 2017. “The Effects of Higher Temperature Setpoints During Summer on Office Workers’ Cognitive Load and Thermal Comfort”. Building and Environment 123: 176–88. https://doi.org/10.1016/j.buildenv.2017.06.048.