VALIDATED ARTIFICIAL INTELLIGENCE APPLICATIONS IN PANDEMIC PREPAREDNESS: A FOCUSED SCOPING REVIEW OF PEER-REVIEWED, EXPLICITLY VALIDATED STUDIES



Cite item

Full Text

Abstract

Background: Artificial intelligence (AI) has been widely explored during the COVID-19 pandemic, yet the robustness and real-world readiness of such applications remain unclear. Understanding domain-specific strengths and gaps is essential for guiding future investment and policy. Unlike prior syntheses that included preprints or studies without explicit validation, this review restricted inclusion to peer-reviewed studies reporting internal, external, or prospective validation, thereby providing a higher-fidelity assessment of AI readiness.

Aim: To synthesise and critically appraise peer-reviewed AI applications that met explicit validation criteria for pandemic preparedness and response, highlighting methodological strengths, limitations, and priority areas for advancement.

Methodology: Following PRISMA-ScR guidelines, a systematic search was conducted in MEDLINE, Scopus, Web of Science, and IEEE Xplore from inception to June 30, 2024, supplemented by reference screening. Inclusion was restricted to full-text, peer-reviewed empirical studies applying AI or machine learning to pandemic preparedness, detection, response, or recovery in human populations, and reporting either (i) validated quantitative performance (internal, external, or prospective) or (ii) measurable real-world/simulated impact. Screening, data extraction, and risk-of-bias assessment were performed independently and in duplicate using a PROBAST-adapted tool to ensure methodological rigour.

Results: Sixteen studies met inclusion criteria, spanning diagnostic imaging (n = 5), clinical outcome prediction (n = 1), surveillance and epidemiological modelling (n = 3), digital contact tracing (n = 3), and single studies in digital health review, PPE supply-chain optimisation, Long-COVID biomarkers, and mental-health surveillance. External validation was more frequent in imaging studies than in other domains but still not universal; only three studies included prospective, real-world evaluation. Most datasets originated from high-income settings. Compared with earlier broad reviews that included non-validated models, these findings reveal fewer but more methodologically sound AI applications with tangible readiness for translation.

Conclusion: While select AI applications—notably diagnostic imaging and large-scale epidemiological models—demonstrate promising validated performance, prospective real-world evaluations and geographically diverse datasets remain scarce. Funders and journals should prioritise external and prospective validation, transparent reporting (TRIPOD-AI/CONSORT-AI), and open code/data sharing to enable reliable deployment. Strategic investment in rigorously validated, context-appropriate AI solutions—including underexplored areas such as supply-chain optimisation, mental-health surveillance, and biomarker discovery—could strengthen global health-emergency preparedness.

About the authors

Kirolos Eskandar

Helwan University, Giza, Egypt

Author for correspondence.
Email: kiroloss.eskandar@gmail.com
ORCID iD: 0000-0003-0085-3284

Physician Researcher;

Academic Degree: M.D.;

affiliation: Helwan University, Faculty of Medicine and Surgery, Egypt;

Egypt

References

  1. Alderman, J. E., Charalambides, M., Sachdeva, G., Laws, E., Palmer, J., Lee, E., Menon, V., Malik, Q., Vadera, S., Calvert, M., Ghassemi, M., McCradden, M. D., Ordish, J., Mateen, B., Summers, C., Gath, J., Matin, R. N., Denniston, A. K., & Liu, X. (2024). Revealing transparency gaps in publicly available COVID-19 datasets used for medical artificial intelligence development—a systematic review. The Lancet Digital Health, 6(11), e827–e847. – https://doi.org/10.1016/s2589-7500(24)00146-8
  2. Appel, K. S., Geisler, R., Maier, D., Miljukov, O., Hopff, S. M., & Vehreschild, J. J. (2023). A systematic review of predictor composition, outcomes, risk of bias, and validation of COVID-19 prognostic scores. Clinical Infectious Diseases, 78(4), 889–899. – https://doi.org/10.1093/cid/ciad618
  3. Baloch, G., Gzara, F., & Elhedhli, S. (2023). Risk-based allocation of COVID-19 personal protective equipment under supply shortages. European Journal of Operational Research, 310(3), 1085–1100. – https://doi.org/10.1016/j.ejor.2023.04.001
  4. Berkhout, W. E. M., Van Wijngaarden, J. J., Workum, J. D., Van De Sande, D., Hilling, D. E., Jung, C., Meyfroidt, G., Gommers, D., Buijsman, S. N. R., & Van Genderen, M. E. (2025). Operationalization of artificial intelligence applications in the intensive care unit. JAMA Network Open, 8(7), e2522866. – https://doi.org/10.1001/jamanetworkopen.2025.22866
  5. Bridge, J., Meng, Y., Zhu, W., Fitzmaurice, T., McCann, C., Addison, C., Wang, M., Merritt, C., Franks, S., Mackey, M., Messenger, S., Sun, R., Zhao, Y., & Zheng, Y. (2023). Development and external validation of a mixed-effects deep learning model to diagnose COVID-19 from CT imaging. Frontiers in Medicine, 10. – https://doi.org/10.3389/fmed.2023.1113030
  6. Bufano, P., Laurino, M., Said, S., Tognetti, A., & Menicucci, D. (2023). Digital Phenotyping for Monitoring Mental Disorders: Systematic Review. Journal of Medical Internet Research, 25, e46778. – https://doi.org/10.2196/46778
  7. Casey, A. E., Ansari, S., Nakisa, B., Kelly, B., Brown, P., Cooper, P., Muhammad, I., Livingstone, S., Reddy, S., & Makinen, V. (2023). Application of a Comprehensive Evaluation Framework to COVID-19 Studies: Systematic review of translational aspects of artificial intelligence in health care. JMIR AI, 2, e42313. – https://doi.org/10.2196/42313
  8. Chu, W. T., Reza, S. M. S., Anibal, J. T., Landa, A., Crozier, I., Bağci, U., Wood, B. J., & Solomon, J. (2023). Artificial intelligence and infectious disease imaging. The Journal of Infectious Diseases, 228(Supplement 4), S322–S336. – https://doi.org/10.1093/infdis/jiad158
  9. Ciecierski-Holmes, T., Singh, R., Axt, M., Brenner, S., & Barteit, S. (2022). Artificial intelligence for strengthening healthcare systems in low- and middle-income countries: a systematic scoping review. NPJ Digital Medicine, 5(1). – https://doi.org/10.1038/s41746-022-00700-y
  10. Cook, S., Conrad, C., Fowlkes, A. L., & Mohebbi, M. H. (2011). Assessing Google Flu Trends performance in the United States during the 2009 influenza virus A (H1N1) pandemic. PLoS ONE, 6(8), e23610. – https://doi.org/10.1371/journal.pone.0023610
  11. Deka, J., & Molla, K. (2022). Supply chain optimization for PPE in COVID-19 pandemic using AI and operations research. Sustainability, 14(9), 5276. – https://doi.org/10.3390/su14095276
  12. Dolin, P., Li, W., Dasarathy, G., & Berisha, V. (2025). Statistically valid post-deployment monitoring should be standard for AI-based digital health. arXiv. – https://doi.org/10.48550/arXiv.2506.05701
  13. Elmokashfi, A., Sundnes, J., Kvalbein, A., Naumova, V., Reinemo, S., Florvaag, P. M., Stensland, H. K., & Lysne, O. (2021). Nationwide rollout reveals efficacy of epidemic control through digital contact tracing. Nature Communications, 12(1). – https://doi.org/10.1038/s41467-021-26144-8
  14. Ferretti, L., Wymant, C., Kendall, M., Zhao, L., Nurtay, A., Abeler-Dörner, L., Parker, M., Bonsall, D., & Fraser, C. (2020). Quantifying SARS-CoV-2 transmission suggests epidemic control with digital contact tracing. Science, 368(6491), eabb6936. – https://doi.org/10.1126/science.abb6936
  15. Gianfrancesco, M. A., Tamang, S., Yazdany, J., & Schmajuk, G. (2018). Potential biases in machine learning algorithms using electronic health record data. JAMA Internal Medicine, 178(11), 1544. – https://doi.org/10.1001/jamainternmed.2018.3763
  16. Ginsberg, J., Mohebbi, M. H., Patel, R. S., Brammer, L., Smolinski, M. S., & Brilliant, L. (2009). Detecting influenza epidemics using search engine query data. Nature, 457(7232), 1012–1014. – https://doi.org/10.1038/nature07634
  17. Gunraj, H., Wang, L., & Wong, A. (2020). COVIDNet-CT: A tailored deep convolutional neural network design for detection of COVID-19 cases from chest CT images. Frontiers in Medicine, 7, 608525. – https://doi.org/10.3389/fmed.2020.608525
  18. Gupta, R. K., Marks, M., Samuels, T. H. A., Luintel, A., Rampling, T., Chowdhury, H., Quartagno, M., Nair, A., Lipman, M., Abubakar, I., van Smeden, M., Wong, W. K., Williams, B., Noursadeghi, M., & UCLH COVID-19 Reporting Group. (2020). Systematic evaluation and external validation of 22 prognostic models among hospitalised adults with COVID-19: an observational cohort study. The European Respiratory Journal, 56(6), 2003498. – https://doi.org/10.1183/13993003.03498-2020
  19. Jemioło, P., Storman, D., & Orzechowski, P. (2022). Artificial Intelligence for COVID-19 Detection in Medical Imaging—Diagnostic Measures and Wasting—A Systematic Umbrella Review. Journal of Clinical Medicine, 11(7), 2054. – https://doi.org/10.3390/jcm11072054
  20. Kang, M. W., & Choi, W. Y. (2020). Investigation on the hump behavior of Gate-Normal Nanowire Tunnel Field-Effect Transistors (NWTFETs). Applied Sciences, 10(24), 8880. – https://doi.org/10.3390/app10248880
  21. Kelly, C. J., Karthikesalingam, A., Suleyman, M., Corrado, G., & King, D. (2019). Key challenges for delivering clinical impact with artificial intelligence. BMC Medicine, 17(1). – https://doi.org/10.1186/s12916-019-1426-2
  22. Ko, H., Chung, H., Kang, W. S., Kim, K. W., Shin, Y., Kang, S. J., Lee, J. H., Kim, Y. J., Kim, N. Y., Jung, H., & Lee, J. (2020). COVID-19 pneumonia diagnosis using a simple 2D deep learning framework with a single chest CT image: Model development and validation. Journal of Medical Internet Research, 22(6), e19569. – https://doi.org/10.2196/19569
  23. Kogan, N. E., Clemente, L., Liautaud, P., Kaashoek, J., Link, N. B., Nguyen, A. T., Lu, F. S., Huybers, P., Resch, B., Havas, C., Petutschnig, A., Davis, J., Chinazzi, M., Mustafa, B., Hanage, W. P., Vespignani, A., & Santillana, M. (2021). An early warning approach to monitor COVID-19 activity with multiple digital traces in near real time. Science Advances, 7(10). – https://doi.org/10.1126/sciadv.abd6989
  24. Kwofie, S. K., Adams, J., Broni, E., Enninful, K. S., Agoni, C., Soliman, M. E. S., & Wilson, M. D. (2023). Artificial Intelligence, Machine Learning, and Big Data for Ebola Virus Drug Discovery. Pharmaceuticals (Basel), 16(3), 332. – https://doi.org/10.3390/ph16030332
  25. Li, L., Qin, L., Xu, Z., Yin, Y., Wang, X., Kong, B., Bai, J., Lu, Y., Fang, Z., Song, Q., & others. (2020). Artificial intelligence distinguishes COVID-19 from community-acquired pneumonia on chest CT. Radiology. – https://doi.org/10.1148/radiol.2020200905
  26. Milinovich G.J., Williams G.M., Clements A.C.A., Hu W. Internet-based surveillance systems for monitoring emerging infectious diseases. The Lancet Infectious Diseases. 2013;14(2):160–168. – https://doi.org/10.1016/s1473-3099(13)70244-5
  27. Myllyaho L., Raatikainen M., Männistö T., Mikkonen T., Nurminen J.K. Systematic literature review of validation methods for AI systems. Journal of Systems and Software. 2021;181:111050. – https://doi.org/10.1016/j.jss.2021.111050
  28. Olson D.R., Konty K.J., Paladini M., Viboud C., Simonsen L. Reassessing Google Flu Trends data for detection of seasonal and pandemic influenza: A comparative epidemiological study at three geographic scales. PLoS Computational Biology. 2013;9(10):e1003256. – https://doi.org/10.1371/journal.pcbi.1003256
  29. Patel M.A., Knauer M.J., Nicholson M., Daley M., Van Nynatten L.R., Cepinskas G., Fraser D.D., et al. Organ and cell-specific biomarkers of Long-COVID identified with targeted proteomics and machine learning. Molecular Medicine. 2023;29:26. – https://doi.org/10.1186/s10020-023-00610-z
  30. Pennestrì F., Cabitza F., Picerno N., Banfi G. Sharing reliable information worldwide: Healthcare strategies based on artificial intelligence need external validation. Position paper. BMC Medical Informatics and Decision Making. 2025;25(1). – https://doi.org/10.1186/s12911-025-02883-2
  31. Rasheed J., Jamil A., Hameed A.A., Al-Turjman F., Rasheed A. COVID-19 in the age of artificial intelligence: A comprehensive review. Interdisciplinary Sciences: Computational Life Sciences. 2021;13(2):153–175. – https://doi.org/10.1007/s12539-021-00431-w
  32. Rasheed J., Jamil A., Hameed A.A., Al-Turjman F., Rasheed A. COVID-19 in the age of artificial intelligence: A comprehensive review. Interdisciplinary Sciences: Computational Life Sciences. 2021;13(2):153–175. – https://doi.org/10.1007/s12539-021-00431-w
  33. Reiner R.C., Barber R.M., Collins J.K., Zheng P., Adolph C., Albright J., Antony C.M., et al. Modeling COVID-19 scenarios for the United States. Nature Medicine. 2020;27(1):94–105. – https://doi.org/10.1038/s41591-020-1132-9
  34. Roberts M., Driggs D., Thorpe M., Gilbey J., Yeung M., Ursprung S., Aviles-Rivero A.I., et al. Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans. Nature Machine Intelligence. 2021;3(3):199–217. – https://doi.org/10.1038/s42256-021-00307-0
  35. Sattler F., Ma J., Wagner P., Neumann D., Wenzel M., Schäfer R., Samek W., et al. Risk estimation of SARS-CoV-2 transmission from Bluetooth low energy measurements. NPJ Digital Medicine. 2020;3:129. – https://doi.org/10.1038/s41746-020-00340-0
  36. Shi F., Wang J., Shi J., Wu Z., Wang Q., Tang Z., He K., Shi Y., Shen D. Review of artificial intelligence techniques in imaging data acquisition, segmentation, and diagnosis for COVID-19. IEEE Reviews in Biomedical Engineering. 2020;14:4–15. – https://doi.org/10.1109/rbme.2020.2987975
  37. Soltan A.A.S., Yang J., Pattanshetty R., Novak A., Yang Y., Rohanian O., Beer S., et al. Real-world evaluation of rapid and laboratory-free COVID-19 triage for emergency care: External validation and pilot deployment of artificial intelligence–driven screening. The Lancet Digital Health. 2022;4(4):e266–e278. – https://doi.org/10.1016/s2589-7500(21)00272-7
  38. Syrowatka A., Kuznetsova M., Alsubai A., Beckman A.L., Bain P.A., Craig K.J.T., Hu J., et al. Leveraging artificial intelligence for pandemic preparedness and response: A scoping review to identify key use cases. NPJ Digital Medicine. 2021;4(1):96. – https://doi.org/10.1038/s41746-021-00459-8
  39. Vaidyanathan A., Guiot J., Zerka F., Belmans F., Van Peufflik I., Deprez L., Danthine D., et al. An externally validated fully automated deep learning algorithm to classify COVID-19 and other pneumonias on chest computed tomography. ERJ Open Research. 2022;8(2):00579–2021. – https://doi.org/10.1183/23120541.00579-2021
  40. Vaishya R., Javaid M., Khan I.H., Haleem A. Artificial intelligence (AI) applications for COVID-19 pandemic. Diabetes & Metabolic Syndrome: Clinical Research & Reviews. 2020;14(4):337–339. – https://doi.org/10.1016/j.dsx.2020.04.012
  41. Valdez D., Ten Thij M., Bathina K., Rutter L.A., Bollen J. Social media insights into US mental health during the COVID-19 pandemic: Longitudinal analysis of Twitter data. Journal of Medical Internet Research. 2020;22(12):e21418. – https://doi.org/10.2196/21418
  42. Vogt F., Haire B., Selvey L., Katelaris A.L., Kaldor J. Effectiveness evaluation of digital contact tracing for COVID-19 in New South Wales, Australia. The Lancet Public Health. 2022;7(3):e250–e258. – https://doi.org/10.1016/s2468-2667(22)00010-x
  43. Wang L., Lin Z.Q., Wong A. COVID-Net: A tailored deep convolutional neural network design for detection of COVID-19 cases from chest X-ray images. Scientific Reports. 2020;10(1):19549. – https://doi.org/10.1038/s41598-020-76550-z
  44. Whitelaw S., Mamas M.A., Topol E., Van Spall H.G.C. Applications of digital technology in COVID-19 pandemic planning and response. The Lancet Digital Health. 2020;2(8):e435–e440. – https://doi.org/10.1016/S2589-7500(20)30120-0
  45. Wollenstein-Betech S., Cassandras C.G., Paschalidis I.Ch. Personalized predictive models for COVID-19 outcomes: Machine learning models for mortality, ICU admission, and mechanical ventilation risk prediction. PLOS ONE. 2021;16(3):e0249285. – https://doi.org/10.1371/journal.pone.0249285
  46. Wymant C., Ferretti L., Tsallis D., Charalambides M., Abeler-Dörner L., Bonsall D., Hinch R., Fraser C., et al. The epidemiological impact of the NHS COVID-19 app. Nature. 2021;594(7863):408–412. – https://doi.org/10.1038/s41586-021-03606-z
  47. Wynants L., Van Calster B., Collins G.S., Riley R.D., Heinze G., Schuit E., Bonten M.M.J., et al. Prediction models for diagnosis and prognosis of COVID-19: Systematic review and critical appraisal. BMJ. 2020;369:m1328. – https://doi.org/10.1136/bmj.m1328
  48. Yuan Q., Nsoesie E.O., Lv B., Peng G., Chunara R., Brownstein J.S., Zhang Q. Monitoring influenza epidemics in China with search queries from Baidu. PLOS ONE. 2013;8(5):e64323. – https://doi.org/10.1371/journal.pone.0064323
  49. Zang S., Zhang X., Xing Y., Chen J., Lin L., Hou Z. Applications of social media and digital technologies in COVID-19 vaccination: Scoping review. Journal of Medical Internet Research. 2023;25:e40057. – https://doi.org/10.2196/40057
  50. Zhang K., Liu X., Shen J., Li Z., Sang Y., Wu X., Zha Y., Liang W., Wang C., Wang K., et al. Clinically applicable AI system for accurate diagnosis, quantitative measurements, and prognosis of COVID-19 pneumonia using computed tomography. Cell. 2020;182(5):1360–1375.e13. – https://doi.org/10.1016/j.cell.2020.08.029

Supplementary files

Supplementary Files
Action
1. JATS XML
Download

Copyright (c) Eskandar K.

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.
СМИ зарегистрировано Федеральной службой по надзору в сфере связи, информационных технологий и массовых коммуникаций (Роскомнадзор).
Регистрационный номер и дата принятия решения о регистрации СМИ: серия ПИ № ФС 77 - 64788 от 02.02.2016.