Electronic Health Records (EHR) Usability and User Experience Evaluation: A Case Study
Abstract
Manual medical record documents have a high risk of leakage and loss. This is because the storage is only in the cupboard. In Indonesia, Electronic Health Records (EHR) have been widely adopted. Clinics, the Rumah Keluarga Sehat Clinic located in Jember Regency. The implemented EHR is a complement to the manual medical record. To obtain a successful EHR implementation, it is necessary to evaluate the usability and user experience. Usability testing is an evaluation method that involves end users in the system development process. This study aims to evaluate the usability and user experience of the EHR developed at the Rumah Keluarga Sehat Clinic. Tests were carried out using the System Usability Scale (SUS) and User Experience Questionnaire (UEQ) instruments. Instruments were given to 7 respondents consisting of doctors, medical recorders, admins, and pharmacists at Rumah Keluarga Sehat Clinic, Jember. The results of the usability evaluation with SUS showed a score of 77.14 which indicated the Good category. And the UEQ results show above-average results on a scale (perspicuity and dependability), and excellent on 4 scales (attractiveness, efficiency, stimulation, and novelty). The results of the evaluation with 2 instruments show that the developed EHR has met the user's needs.
Downloads
References
M. Hyzy et al., “System Usability Scale Benchmarking for Digital Health Apps: Meta-analysis.,” JMIR mHealth uHealth, vol. 10, no. 8, p. e37290, Aug. 2022, doi: 10.2196/37290.
D. Spatar, O. Kok, N. Basoglu, and T. Daim, “Adoption factors of electronic health record systems,” Technol. Soc., vol. 58, p. 101144, 2019.
A. Hennington and B. D. Janz, “Information systems and healthcare XVI: physician adoption of electronic medical records: applying the UTAUT model in a healthcare context,” Commun. Assoc. Inf. Syst., vol. 19, no. 1, p. 5, 2007.
S. Khairat, C. Coleman, P. Ottmar, T. Bice, and S. S. Carson, “Evaluation of Physicians’ Electronic Health Records Experience Using Actual and Perceived Measures,” Perspect. Heal. Inf. Manag., vol. 19, no. 1, 2022.
K. Andersen and C. Nøhrd, “Development of methods for usability evaluations of EHR systems,” in Ubiquity: Technologies for Better Health in Aging Societies: Proceedings of MIE2006, 2006, vol. 124, p. 341.
N. Bevan, J. Carter, and S. Harker, “ISO 9241-11 revised: What have we learnt about usability since 1998?,” in International conference on human-computer interaction, 2015, pp. 143–151.
S. Kumar, W. Nilsen, M. Pavel, and M. Srivastava, “Mobile health: Revolutionizing healthcare through transdisciplinary research,” Computer (Long. Beach. Calif)., vol. 46, no. 1, pp. 28–35, 2012.
J. Brooke, “SUS A Quick and Dirty Usability Scale,” Usability Eval. Ind., vol. 194, pp. 4–7, 1996.
M. Schrepp, “User experience questionnaire handbook,” All you need to know to apply UEQ successfully your Proj., 2015.
H. B. Santoso, M. Schrepp, R. Yugo Kartono Isal, Y. Utomo, and B. Priyogi, “Measuring User Experience of the Student-Centered e-Learning Environment,” J. Educ. Online-JEO, vol. 13, no. 1, pp. 58–79, 2016, doi: 10.9743/JEO.2016.1.5.
M. A. Tutty, L. E. Carlasare, S. Lloyd, and C. A. Sinsky, “The complex case of EHRs: examining the factors impacting the EHR user experience,” J. Am. Med. Informatics Assoc., vol. 26, no. 7, pp. 673–677, 2019.
U. U. Rehman, D. J. Chang, Y. Jung, U. Akhtar, M. A. Razzaq, and S. Lee, “Medical instructed real-time assistant for patient with glaucoma and diabetic conditions,” Appl. Sci., vol. 10, no. 7, p. 2216, 2020.
M. A. Kushendriawan, H. B. Santoso, P. O. H. Putra, and M. Schrepp, “Evaluating User Experience of a Mobile Health Application ‘Halodoc’using User Experience Questionnaire and Usability Testing,” J. Sist. Inf., vol. 17, no. 1, pp. 58–71, 2021.
M. Hägglund and I. Scandurra, “User evaluation of the swedish patient accessible electronic health record: system usability scale,” JMIR Hum. factors, vol. 8, no. 3, p. e24927, 2021.
E. R. Melnick et al., “The Association Between Perceived Electronic Health Record Usability and Professional Burnout Among US Physicians,” Mayo Clin. Proc., vol. 95, no. 3, pp. 476–487, 2020, doi: https://doi.org/10.1016/j.mayocp.2019.09.024.
B. Laugwitz, T. Held, and M. Schrepp, “Construction and Evaluation of a User Experience Questionnaire BT - HCI and Usability for Education and Work,” 2008, pp. 63–76.
A. Bangor, P. T. Kortum, and J. T. Miller, “An empirical evaluation of the system usability scale,” Int. J. Hum. Comput. Interact., 2008, doi: 10.1080/10447310802205776.
J. Brooke, “SUS: A Retrospective,” J. Usability Stud., vol. 8, no. 2, pp. 29–40, 2013.
A. Bangor, T. Staff, P. Kortum, J. Miller, and T. Staff, “Determining What Individual SUS Scores Mean : Adding an Adjective Rating Scale,” J. Usability Scales, vol. 4, no. 3, pp. 114–123, 2009.
S. Holmes, A. Moorhead, R. Bond, H. Zheng, V. Coates, and M. McTear, “Usability testing of a healthcare chatbot: Can we use conventional methods to assess conversational user interfaces?,” in Proceedings of the 31st European Conference on Cognitive Ergonomics, 2019, pp. 207–214.
V. Blanes-Selva, S. Asensio-Cuesta, A. Doñate-Martínez, F. P. Mesquita, and J. M. García-Gómez, “Validating a Clinical Decision Support System for Palliative Care using healthcare professionals’ insights,” medRxiv, 2022.
M. Sevilla-Gonzalez et al., “System Usability Scale Questionnaire for the assessment of electronic tools: Development and validation of a Spanish version (Preprint),” JMIR Hum. Factors, vol. 7, Jun. 2020, doi: 10.2196/21161.
I. D. Sabukunze and A. Arakaza, “User Experience Analysis on Mobile Application Design Using User Experience Questionnaire,” Indones. J. Inf. Syst., vol. 4, no. 1, pp. 15–26, 2021.
A. HESHMATI, N. Moftian, P. Rezaei-Hachesu, and T. Samad-Soltani, “Simple and efficient measurement of the user experience in health information systems: A persian version,” Iran. J. Public Health, vol. 50, no. 1, p. 217, 2021.
S. Stumpp, T. Knopf, and D. Michelis, “User experience design with augmented reality (AR),” 2019.
L. Mlekus, D. Bentler, A. Paruzel, A.-L. Kato-Beiderwieden, and G. W. Maier, “How to raise technology acceptance: user experience characteristics as technology-inherent determinants,” Grup. Interaktion. Organ. Zeitschrift für Angew. Organ., vol. 51, no. 3, pp. 273–283, 2020.
M. Schrepp, A. Hinderks, and J. Thomaschewski, “Applying the User Experience Questionnaire (UEQ) in Different Evaluation Scenarios BT - Design, User Experience, and Usability. Theories, Methods, and Tools for Designing the User Experience,” 2014, pp. 383–392.
Copyright (c) 2023 Muhammad Yunus, Intan Sulistyaningrum Sakkinah, Ulfa Emi Rahmawati, Atma Deharja, Maya Weka Santi
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
1. Copyright on any article is retained by the author(s).
2. The author grants the journal, right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work’s authorship and initial publication in this journal.
3. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal’s published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
4. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
5. The article and any associated published material is distributed under the Creative Commons Attribution-ShareAlike 4.0 International License
