Predicting Online Search Success from Self-Reported Skills, Search Behavior, Domain Knowledge, and Task Evaluations
PDF (Deutsch)

Keywords

Online Search
Information Literacy
Media Education

How to Cite

Hermida, Martin, Luca Botturi, Loredana Addiamando, Marzia Galloni, Chiara Beretta, Felipe Cardoso, Luca Luceri, Azza Bouleimen, Silvia Giordano, and Mirna Saad. 2025. “Predicting Online Search Success from Self-Reported Skills, Search Behavior, Domain Knowledge, and Task Evaluations: Implications for Teaching Information Literacy”. MediaEducation: Journal for Theory and Practice of Media Education 2025 (Occasional Papers):269-94. https://doi.org/10.21240/mpaed/00/2025.11.07.X.

License

Copyright (c) 2025 Martin Hermida, Luca Botturi, Loredana Addiamando, Marzia Galloni, Chiara Beretta, Felipe Cardoso, Luca Luceri, Azza Bouleimen, Silvia Giordano, Mirna Saad

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Abstract

Searching for information is a key activity in everyday life. Understanding how self-reported skills, search behavior and task characteristics relate to search performance can inform educational practices for teaching information literacy. This study investigates these relationships by having 102 adolescents perform search tasks and assessing their performance in relation to their self-reported information literacy, navigation behavior and task evaluations. Our findings indicate that self-reported information self-efficacy alone is not a reliable predictor of search success. Query revisions, domain knowledge and task importance positively impact search success, while reliance on superficial cues for evaluating information quality correlates with lower search success. We suggest that information literacy educators encourage learners to revise their queries, ignore superficial cues for credibility assessment, and gain an overview of a topic before searching for specific facts. Additionally, we recommend designing search tasks with potentially important consequences when solved incorrectly to increase learners’ engagement.

https://doi.org/10.21240/mpaed/00/2025.11.07.X

References

Bandura, Albert. 1997. Self-Efficacy. The Exercise of Control. New York: W. H. Freeman Company.

Behrens, Shirley J. 1994. “A Conceptual Analysis and Historical Overview of Information Literacy”. College & Research Libraries 55 (4): 309–22. https://doi.org/10.5860/crl_55_04_309.

Bodemer, Brett B. 2017. “The Importance of Search as Intertextual Practice for Undergraduate Research”. College & Research Libraries 78 (4): 413–27. https://doi.org/10.5860/crl-245.

Botturi, Luca, and Lucio Negrini. 2018. “Less Is More: Teenager Digital Information Literacy and Parental Rules”. In ICERI2018 Proceedings, 8633–42. Seville, Spain. https://doi.org/10.21125/iceri.2018.0589.

Brand-Gruwel, Saskia, Iwan Wopereis, and Yvonne Vermetten. 2005. “Information Problem Solving by Experts and Novices: Analysis of a Complex Cognitive Skill”. Computers in Human Behavior, The Pedagogical Use of Information and Communication Technology in Education: A Dutch Perspective, 21 (3): 487–508. https://doi.org/10.1016/j.chb.2004.10.005.

Broder, Andrei. 2002. “A Taxonomy of Web Search”. ACM Sigir Forum 36 (2): 3–10. https://doi.org/10.1145/792550.792552.

Byström, Katrina, and Kalervo Järvelin. 1995. “Task Complexity Affects Information Seeking and Use”. Information Processing & Management 31 (2): 191–213. https://doi.org/10.1016/0306-4573(95)80035-R.

Campbell, Donald J. 1988. “Task Complexity: A Review and Analysis”. Academy of Management Review 13 (1): 40–52. https://doi.org/10.2307/258353.

Carretero, Stephanie, Riina Vuorikari, and Yves Punie. 2017. DigComp 2.1: The Digital Competence Framework for Citizens. Luxembourg: Publications Office of the European Union. https://doi.org/10.2760/38842.

Case, Donald Owen, and Lisa M. Given. 2016. Looking for Information: A Survey of Research on Information Seeking, Needs, and Behavior. Fourth edition. Studies in Information. Bingley, UK: Emerald.

Chen, Gina Masullo, Peter S. Chen, Chen-Wei Chang, and Zainul Abedin. 2017. “News Video Quality Affects Online Sites’ Credibility”. Newspaper Research Journal 38 (1): 19–31. https://doi.org/10.1177/0739532917696087.

CILIP, The Library and Information Association. 2018. ‘CILIP Definition of Information Literacy’. London: The Library and Information Association.

Clark, Melissa. 2017. ”Imposed-Inquiry Information-Seeking Self-Efficacy and Performance of College Students: A Review of the Literature”. The Journal of Academic Librarianship 43 (5): 417–22. https://doi.org/10.1016/j.acalib.2017.05.001.

Downing, Ricard E., Joi L. Moore, and Steven W. Brown. 2005. “The Effects and Interaction of Spatial Visualization and Domain Expertise on Information Seeking”. Computers in Human Behavior 21 (2): 195–209. https://doi.org/10.1016/j.chb.2004.03.040.

Eisenberg, Michael B., and Robert E. Berkowitz. 1992. “Information Problem-Solving: The Big Six Skills Approach”. School Library Media Activities Monthly 8 (5): 27.

Feierabend, Sabine, Thomas Rathgeb, Hediye Kheredmand, and Stephan Glöckler. 2023. “JIM-Studie 2023. Jugend, Informationen, Medien”. Stuttgart: Medienpädagogischer Forschungsverbund Südwest. https://mpfs.de/app/uploads/2024/10/JIM_2023_web_final_kor.pdf.

Feierabend, Sabine, Thomas Rathgeb, and Theresa Reutter. 2018. “JIM 2018. Jugend, Informationen, Medien. Basisuntersuchung Zum Medienumgang 12- Bis 19-Jähriger in Deutschland”. Stuttgart: Medienpädagogischer Forschungsverbund Südwest. https://mpfs.de/app/uploads/2024/11/JIM2018_Gesamt.pdf.

Ford, Nigel, David Miller, and Nicola Moss. 2002. “Web Search Strategies and Retrieval Effectiveness: An Empirical Study”. Journal of Documentation 58 (1): 30–48. https://doi.org/10.1108/00220410210425395.

Fraillon, Julian, John Ainley, Wolfram Schulz, Tim Friedman, and Daniel Duckworth. 2020. PREPARING FOR LIFE IN A DIGITAL WORLD. IEA International Computer and Information Literacy Study 2018 International Report. Springer. https://doi.org/10.1007/978-3-030-38781-5.

Franklin, Bob, and Scott A. Eldridge, eds. 2017. The Routledge Companion to Digital Journalism Studies. London ; New York: Routledge/Taylor & Francis Group.

Gao, Yizhu, Ying Cui, Okan Bulut, Xiaoming Zhai, and Fu Chen. 2022. “Examining Adults’ Web Navigation Patterns in Multi-Layered Hypertext Environments”. Computers in Human Behavior 129 (April):107142. https://doi.org/10.1016/j.chb.2021.107142.

Gebel, Christa, Nadine Jünger, and Ulrike Wagner. 2014. “Informations-Und Engagementbezogenes Medienhandeln von Jugendlichen”. In Jugendliche Und Die Aneignung Politischer Information in Online-Medien, 53–136. Springer.

Gilster, Paul. 1997. Digital Literacy. Weinheim: Wiley Computer Publishing.

Gross, Melissa, and Don Latham. 2007. ‘Attaining Information Literacy: An Investigation of the Relationship between Skill Level, Self-Estimates of Skill, and Library Anxiety’. Library & Information Science Research 29 (3): 332–53. https://doi.org/10.1016/j.lisr.2007.04.012.

Guitert, M., T. Romeu, and P. Baztán. 2017. “CONCEPTUAL FRAMEWORK ON DIGITAL COMPETENCES IN PRIMARY AND SECONDARY SCHOOLS IN EUROPE”. ICERI2017 Proceedings, 5081–90.

Gwizdka, Jacek, and Ian Spence. 2007. “What Can Searching Behavior Tell Us About the Difficulty of Information Tasks? A Study of Web Navigation”. Proceedings of the American Society for Information Science and Technology 43 (1): 1–22. https://doi.org/10.1002/meet.14504301167.

Hansen, Derek L, Holly A Derry, Paul J Resnick, and Caroline R Richardson. 2003. “Adolescents Searching for Health Information on the Internet: An Observational Study”. Journal of Medical Internet Research 5 (4): 1–10. https://doi.org/10.2196/jmir.5.4.e25.

Hasebrink, Uwe. 2017. “Audiences and Information Repertoires”. In The Routledge Companion to Digital Journalism Studies, 364–74. London: Routledge.

Hobbs, Renee. 2010. Digital and Media Literacy: A Plan of Action. A White Paper on the Digital and Media Literacy Recommendations of the Knight Commission on the Information Needs of Communities in a Democracy. Aspen Institute.

Hermida, Martin. 2021. ‘Der Weg der Medien- und Kommunikationswissenschaft in die Volksschule (3F-Modell) Und Die Messung Der Einstellungen Der Lehrpersonen Zur Medienbildung’. In Media Literacy, edited by Guido Keel and Wibke Weber, 95–110. Baden-Baden: Nomos. https://doi.org/10.5771/9783748920656-95.

Hong, Traci. 2006. “The Internet and Tobacco Cessation: The Roles of Internet Self-Efficacy and Search Task on the Information-Seeking Process”. Journal of Computer-Mediated Communication 11 (2): 536–56. https://doi.org/10.1111/j.1083-6101.2006.00026.x.

IFLA. 2005. “Alexandria Proclamation on Libraries, the Information Society in Action”. Alexandria: IFLA – International Federation of Library Associations and Institutions. https://repository.ifla.org/bitstream/123456789/3147/1/alexandria-proclamation-on-libraries-the-information-society-in-action-en.pdf.

Ji, Ziwei, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. “Survey of Hallucination in Natural Language Generation”. ACM Computing Surveys 55 (12): 1–38. https://doi.org/10.1145/3571730.

JISC [Joint Information Systems Committee]. 2014. “Developing Digital Literacies”. 2014. https://www.jisc.ac.uk/guides/developing-digital-literacies.

Kim, Soojung, and Dagobert Soergel. 2006. “Selecting and Measuring Task Characteristics as Independent Variables”. Proceedings of the American Society for Information Science and Technology 42 (1). https://doi.org/10.1002/meet.14504201111.

Kruger, Justin, and David Dunning. 1999. ‘Unskilled and Unaware of It: How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments’. Journal of Personality and Social Psychology 77 (6): 1121–34.

Kurbanoglu, S. Serap, Buket Akkoyunlu, and Aysun Umay. 2006. “Developing the Information Literacy Self‐efficacy Scale”. Journal of Documentation 62 (6): 730–43. https://doi.org/10.1108/00220410610714949.

Lau, Tessa, and Eric Horvitz. 1999. “Patterns of Search: Analyzing and Modeling Web Query Refinement”. In UM99 User Modeling. Proceedings of the Seventh International Conference, 119–28. Vienna: Springer.

Leys, Christophe, Marie Delacre, Youri L. Mora, Daniël Lakens, and Christophe Ley. 2019. “How to Classify, Detect, and Manage Univariate and Multivariate Outliers, With Emphasis on Pre-Registration”. International Review of Social Psychology 32 (1): 5. https://doi.org/10.5334/irsp.289.

Liu, Jingjing, Jacek Gwizdka, Chang Liu, and Nicholas J. Belkin. 2010. “Predicting Task Difficulty for Different Task Types”. Proceedings of the American Society for Information Science and Technology 47 (1): 1–10. https://doi.org/10.1002/meet.14504701173.

Livingstone, Sonia, Giovanna Mascheroni, and Mariya Stoilova. 2023. “The Outcomes of Gaining Digital Skills for Young People’s Lives and Wellbeing: A Systematic Evidence Review”. New Media & Society 25 (5): 1176–1202. https://doi.org/10.1177/14614448211043189.

Lucassen, Teun, Rienco Muilwijk, Matthijs L. Noordzij, and Jan Maarten Schraagen. 2013. “Topic Familiarity and Information Skills in Online Credibility Evaluation”. Journal of the American Society for Information Science and Technology 64 (2): 254–64. https://doi.org/10.1002/asi.22743.

Madden, Andrew D., Sheila Webber, Nigel Ford, and Mary Crowder. 2018. “The Relationship between Students’ Subject Preferences and Their Information Behaviour”. Journal of Documentation 74 (4): 692–721. https://doi.org/10.1108/JD-07-2017-0097.

Metzger, Miriam J. 2007. “Making Sense of Credibility on the Web: Models for Evaluating Online Information and Recommendations for Future Research”. Journal of the American Society for Information Science and Technology 58 (13): 2078–91. https://doi.org/10.1002/asi.20672.

Mosenthal, Peter B. 1998. “Defining Prose Task Characteristics for Use in Computer-Adaptive Testing and Instruction”. American Educational Research Journal 35 (2): 269–307.

Ng, Wan. 2012. “Can We Teach Digital Natives Digital Literacy?” Computers & Education 59 (3): 1065–78.

Norden, Lars-Ake, Linda Mannila, and Arnold Pears. 2017. “Development of a Self-Efficacy Scale for Digital Competences in Schools”. In 2017 IEEE Frontiers in Education Conference (FIE), 1–7. Indianapolis, IN: IEEE. https://doi.org/10.1109/FIE.2017.8190673.

Nygren, Thomas, and Mona Guath. 2019. “Swedish Teenagers’ Difficulties and Abilities to Determine Digital News Credibility”. Nordicom Review 40 (1): 23–42.

Porat, Erez, Ina Blau, and Azy Barak. 2018. “Measuring Digital Literacies: Junior High-School Students’ Perceived Competencies versus Actual Performance”. Computers & Education 126 (November):23–36. https://doi.org/10.1016/j.compedu.2018.06.030.

Ramalho Correia, Ana Maria. 2002. Information Literacy for an Active and Effective Citizenship. White Paper Prepared for UNESCO, the U.S. National Commission on Libraries and Information Science, and the National Forum on Information Literacy. Prague.

Reuters Institute. 2023. “Digital News Report 2023”. Oxford: Reuters Institute for the Study of Journalism.

Rosman, Tom, Anne-Kathrin Mayer, and Günter Krampen. 2015. «Combining Self-Assessments and Achievement Tests in Information Literacy Assessment: Empirical Results and Recommendations for Practice». Assessment & Evaluation in Higher Education 40 (5): 740–54. https://doi.org/10.1080/02602938.2014.950554.

Sample, Angela. 2020. “Historical Development of Definitions of Information Literacy: A Literature Review of Selected Resources”. The Journal of Academic Librarianship 46 (2): 102116. https://doi.org/10.1016/j.acalib.2020.102116.

Spink, Amanda, H. Cenk Ozmutlu, and Seda Ozmutlu. 2002. “Multitasking Information Seeking and Searching Processes”. Journal of the American Society for Information Science and Technology 53 (8): 639–52. https://doi.org/10.1002/asi.10124.

Stanford History Education Group. 2016. “Evaluation Information: The Cornerstone of Civic Online Reasoning”. Stanford.

Tamboer, Sanne L, Mariska Kleemans, and Serena Daalmans. 2022. “‘We Are a Neeeew Generation’: Early Adolescents’ Views on News and News Literacy”. Journalism 23 (4): 806–22. https://doi.org/10.1177/1464884920924527.

Timmers, Caroline F., and Cees A. W. Glas. 2010. “Developing Scales for Information‐seeking Behaviour”. Journal of Documentation 66 (1): 46–69. https://doi.org/10.1108/00220411011016362.

Togia, Aspasia, Stella Korobili, Aphrodite Malliari, and Ilias Nitsos. 2015. ”Teachers’ Views of Information Literacy Practices in Secondary Education: A Qualitative Study in the Greek Educational Setting”. Journal of Librarianship and Information Science 47 (3): 226–41. https://doi.org/10.1177/0961000614532485.

Tsai, Meng-Jung, and Chin-Chung Tsai. 2003. “Information Searching Strategies in Web-Based Science Learning: The Role of Internet Self-Efficacy”. Innovations in Education and Teaching International 40 (1): 43–50. https://doi.org/10.1080/1355800032000038822.

UNESCO. 2007. “Paris Agenda for Media and Information Literacy”. 2007. https://www.diplomatie.gouv.fr/IMG/pdf/Parisagendafin_en.pdf.

UNESCO. 2011. “Digital Literacy in Education”. Policy Brief, 1–12.

Vakkari, Pertti. 2005. “Task-Based Information Searching”. Annual Review of Information Science and Technology 37 (1): 413–64. https://doi.org/10.1002/aris.1440370110.

Van Laar, Ester, Alexander J.A.M. Van Deursen, Jan A.G.M. Van Dijk, and Jos De Haan. 2017. ‘The Relation between 21st-Century Skills and Digital Skills: A Systematic Literature Review’. Computers in Human Behavior 72 (July):577–88. https://doi.org/10.1016/j.chb.2017.03.010.

Waddell, T. Franklin. 2018. “This Tweet Brought to You by a Journalist: How Comment Gatekeeping Influences Online News Credibility”. Electronic News 12 (4): 218–34. https://doi.org/10.1177/1931243117739946.

Walraven, Amber, Saskia Brand-Gruwel, and Henny P. A. Boshuizen. 2009. “How Students Evaluate Information and Sources When Searching the World Wide Web for Information”. Computers & Education 52 (1): 234–46. https://doi.org/10.1016/j.compedu.2008.08.003.

White, Ryen W. 2016. Interactions with Search Systems. Cambridge University Press.

Willoughby, Teena, S. Alexandria Anderson, Eileen Wood, Julie Mueller, and Craig Ross. 2009. “Fast Searching for Information on the Internet to Use in a Learning Context: The Impact of Domain Knowledge”. Computers & Education 52 (3): 640–48. https://doi.org/10.1016/j.compedu.2008.11.009.

Wilson, Thomas D. 1997. “Information Behaviour: An Interdisciplinary Perspective”. Information Processing & Management 33 (4): 551–72.

Wood, Eileen, Domenica De Pasquale, Julie Lynn Mueller, Karin Archer, Lucia

Zivcakova, Kathleen Walkey, and Teena Willoughby. 2016. “Exploration of the Relative Contributions of Domain Knowledge and Search Expertise for Conducting Internet Searches”. The Reference Librarian 57 (3): 182–204.

Zampieri, Sandra, Luca Botturi, and Spartaco Calvo. 2018. “Giovani e tecnologie: tranativi digitali e competenze effettive”. Swiss Journal of Educational Research 40 (2): 307–34. https://doi.org/10.24452/sjer.40.2.5063.