Evaluating Usability of IRAIVI Pregnancy Prediction Model Using System Usability Scale

Rahul Shrivastava; Manmohan Singhal; Ashish Joshi; Nivedita Mishra; Neeraj Kashyap; Rohit Shrivastava; Eshwar Tipirisetty; Avisha Soni

doi:10.55489/njcm.141220233423

Authors

Rahul Shrivastava School of Pharmaceutical & Population Health Informatics, Faculty of Pharmacy, DIT University, Dehradun; Ministry of Health & Family Welfare, Nirman Bhawan, New Delhi, India
Manmohan Singhal School of Pharmaceutical & Population Health Informatics, Faculty of Pharmacy, DIT University, Dehradun, India
Ashish Joshi The University of Memphis, The University of Texas Health Science Center at Houston, Memphis, Tennessee, USA
Nivedita Mishra Sambodhi Research & Communication Pvt Ltd, Noida, India
Neeraj Kashyap The INCLEN Trust International, New Delhi, India
Rohit Shrivastava Ekdanta Dental Care, Bhopal, India
Eshwar Tipirisetty Indian Institute of Public Health, Gandhinagar, India
Avisha Soni Indian Institute of Public Health, Hyderabad, India

DOI:

https://doi.org/10.55489/njcm.141220233423

Keywords:

IRAIVI, SUS, citizen centric, urban slum, prediction, model, usability

Abstract

Background: Barriers to utilize maternal healthcare services amongst pregnant women at community level differs substantial variation among slums and urban areas which is essential to recognize and resolve these issues within framework of district-level policy planning. IRAIVI pregnancy prediction model is developed to enhance optimal utilization of maternal healthcare services. The study was conducted to evaluate usability of IRAIVI using System Usability Scale amongst healthcare workforce.

Methods: This model was developed with a set of predictors based on data collected during baseline and follow up visits from currently pregnant women as per study protocol. For evaluating efficiency of this model, System Usability Scale (SUS) was adopted and shared with 25 randomly selected experts in the field of public health. The questionnaire was shared via google form and responses recorded on Likert’s scale were then analysed using SPSS.

Results: IRAIVI model's usability assessed using SUS found to be user-friendly, best learning curve, adaptable to new system and highly acceptance by healthcare workforce. The SUS score averaging at 84 demonstrates favourable usability of the model.

Conclusion: This model has capability to accentuate maternal health services which in-turn can contribute for better ANC services in resource-constrained settings. Additionally, future opportunities can be explored through field studies in different settings.

References

Khan Z, Mehnaz S, Siddiqui AR, Ansari A, Khalil S, Sachdeva S. All Slums are Not Equal: Maternal Health Conditions Among Two Urban Slum Dwellers. Indian J Community Med. 2012 Jan;37(1):50-6. Doi: https://doi.org/10.4103/0970-0218.94027 PMid:22529541 PMCid:PMC3326809

Institute of Medicine (US) Committee on Improving Birth Outcomes; Bale JR, Stoll BJ, Lucas AO, editors. Improving Birth Outcomes: Meeting the Challenge in the Developing World. Washington (DC): National Academies Press (US); 2003. 2, Reducing Maternal Mor-tality and Morbidity. Available from: https://www.ncbi.nlm.nih.gov/books/NBK222105/ (Last accessed on 31 August 2023)

Sun H, Depraetere K, Meesseman L, Cabanillas Silva P, Szymanowsky R, Fliegenschmidt J, Hulde N, von Dossow V, Vanbiervliet M, De Baerdemaeker J, Roccaro-Waldmeyer DM, Stieg J, Domínguez Hidalgo M, Dahlweid FM. Machine Learning-Based Prediction Models for Different Clinical Risks in Different Hospitals: Evaluation of Live Performance. J Med Internet Res. 2022 Jun 7;24(6):e34295. Doi: https://doi.org/10.2196/34295 PMid:35502887 PMCid:PMC9214618

Shrivastava R, Singhal M, Gupta M, Joshi A. Development of an Artificial Intelligence-Guided Citizen-Centric Predictive Model for the Uptake of Maternal Health Services Among Pregnant Women Living in Urban Slum Settings in India: Protocol for a Cross-sectional Study With a Mixed Methods Design. JMIR Res Protoc. 2023 Jan 27;12:e35452. Doi: https://doi.org/10.2196/35452 PMid:36705968 PMCid:PMC9919485

Slobbe LCJ, Füssenich K, Wong A, Boshuizen HC, Nielen MMJ, Polder JJ, Feenstra TL, van Oers HAM. Estimating disease prevalence from drug utilization data using the Random Forest algorithm. Eur J Public Health. 2019 Aug 1;29(4):615-621. Doi: https://doi.org/10.1093/eurpub/cky270 PMid:30608539 PMCid:PMC6660107

Jacobsen LM, Larsson HE, Tamura RN, Vehik K, Clasen J, Sosenko J, Hagopian WA, She JX, Steck AK, Rewers M, Simell O, Toppari J, Veijola R, Ziegler AG, Krischer JP, Akolkar B, Haller MJ; TEDDY Study Group. Predicting progression to type 1 diabetes from ages 3 to 6 in islet autoantibody positive TEDDY children. Pediatr Diabetes. 2019 May;20(3):263-270. Doi: https://doi.org/10.1111/pedi.12812 PMid:30628751 PMCid:PMC6456374

AlKaabi LA, Ahmed LS, Al Attiyah MF, Abdel-Rahman ME. Predicting hypertension using machine learning: Findings from Qatar Bi-obank Study. PLoS One. 2020 Oct 16;15(10):e0240370. Doi: https://doi.org/10.1371/journal.pone.0240370 PMid:33064740 PMCid:PMC7567367

Weng SF, Vaz L, Qureshi N, Kai J. Prediction of premature all-cause mortality: A prospective general population cohort study com-paring machine-learning and standard epidemiological approaches. PLoS One. 2019 Mar 27;14(3):e0214365. Doi: https://doi.org/10.1371/journal.pone.0214365 PMid:30917171 PMCid:PMC6436798

Chaganti R, Rustam F, De La Torre Díez I, Mazón JLV, Rodríguez CL, Ashraf I. Thyroid Disease Prediction Using Selective Features and Machine Learning Techniques. Cancers (Basel). 2022 Aug 13;14(16):3914. Doi: https://doi.org/10.3390/cancers14163914 PMid:36010907 PMCid:PMC9405591

Bangor A, Kortum P, Miller J. Determining what individual SUS scores mean: adding an adjective rating scale. J Usability Stud. 2009;4(3):114-23. https://uxpajournal.org/determining-what-individual-sus-scores-mean-adding-an-adjective-rating-scale/

Kortum PT, Bangor A. Usability ratings for everyday products measured with the system usability scale. Int J Hum-Comput Inter-act. 2013 Jan;29(2):67-76. Doi: https://doi.org/10.1080/10447318.2012.681221

Geroldinger, A., Lusa, L., Nold, M. et al. Leave-one-out cross-validation, penalization, and differential bias of some prediction model performance measures-a simulation study. Diagn Progn Res 7, 9 (2023). Doi: https://doi.org/10.1186/s41512-023-00146-0 PMid:37127679 PMCid:PMC10152625

James R. Lewis (2018) The System Usability Scale: Past, Present, and Future, International Journal of Human-Computer Interac-tion, 34:7, 577-590, Doi: https://doi.org/10.1080/10447318.2018.1455307

Khalifa M, Magrabi F, Gallego Luxan B. Evaluating the Impact of the Grading and Assessment of Predictive Tools Framework on Cli-nicians and Health Care Professionals' Decisions in Selecting Clinical Predictive Tools: Randomized Controlled Trial. J Med Internet Res. 2020 Jul 9;22(7):e15770. Doi: https://doi.org/10.2196/15770 PMid:32673228 PMCid:PMC7381257

Batko K, Ślęzak A. The use of Big Data Analytics in healthcare. J Big Data. 2022;9(1):3. Doi: https://doi.org/10.1186/s40537-021-00553-4 PMid:35013701 PMCid:PMC8733917

Usability.Gov System Usability Scale (SUS). 2020. Available online: https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html (last accessed on 30 August 2023).

Wichmann RM, Fagundes TP, de Oliveira TA, Batista AFdM, Chiavegatto Filho ADP (2022) Physician preference for receiving ma-chine learning predictive results: A cross-sectional multicentric study. PLOS ONE 17(12):e0278397. Doi: https://doi.org/10.1371/journal.pone.0278397 PMid:36516134 PMCid:PMC9749966