Evaluating Usability And Customer Satisfaction in  E-Marketplaces Using System Usability Scale and Customer Satisfaction Score

Nova Christina Sari; Yusa Putra; Alfa Hikmatun Nabilah; Revania Jeni Puspitasari

Authors

Nova Christina Sari Universitas Muhammadiyah Semarang
Yusa Putra Universitas Riau
Alfa Hikmatun Nabilah Universitas Muhammadiyah Semarang
Revania Jeni Puspitasari Universitas Muhammadiyah Semarang

Keywords:

SUS, CSAT, User Experience, Usability, Customer Satisfaction

Abstract

In the digital era, user experience (UX) plays a critical role in determining the success of online platforms, particularly in the highly competitive e-commerce industry. This study presents a comparative evaluation of three major e-commerce applications in Indonesia namely Shopee, Tokopedia, and Lazada using two standardized instruments: the System Usability Scale (SUS) and the Customer Satisfaction Score (CSAT). A total of 100 respondents participated in the survey, which included 10 questions mapped to SUS and 10 questions mapped to CSAT. The results revealed that all three platforms scored poorly in usability and customer satisfaction metrics. Shopee achieved the highest SUS score 51, categorized as marginal, while Tokopedia 50 and Lazada 49.55. CSAT scores followed a similar pattern, with Shopee 52%, Tokopedia 55%, and Lazada 50% falling into low or very low satisfaction categories. These findings highlight the need for substantial improvements in both usability and service quality to enhance overall user experience and engagement. This study emphasizes the importance of integrating both SUS and CSAT methods to obtain a holistic understanding of user perceptions

References

M. Rohandi, N. Husain, and I. W. Bay, “Usability testing of intensive course mobile application using the usability scale system,” ILKOM Jurnal Ilmiah, vol. 13, no. 3, pp. 252–258, Dec. 2021, doi: 10.33096/ilkom.v13i3.821.252-258.

A. Pailina and E. P. Agustini, “Evaluasi Usability Aplikasi BSB Mobile Menggunakan System Usability Scale,” Jurnal Teknologi Informatika dan Komputer, vol. 9, no. 2, pp. 1259–1267, Sep. 2023, doi: 10.37012/jtik.v9i2.1815.

H. Åvik Persson, C. Castor, N. Andersson, and M. Hylén, “Swedish Version of the System Usability Scale: Translation, Adaption, and Psychometric Evaluation,” JMIR Hum Factors, vol. 12, pp. e64210–e64210, Jan. 2025, doi: 10.2196/64210.

Bagas Yana Prayoga, “Usability Analysis Of Camscanner Applications Using The System Usability Scale (Sus) Method,” Jurnal Perangkat Lunak, vol. 5, no. 2, pp. 177–186, Jun. 2023, doi: 10.32520/jupel.v5i2.2616.

N. Hasibuan and R. A. Putri, “Usability Evaluation of Wedding Administrative Information System using System Usability Scale,” Sinkron, vol. 7, no. 4, pp. 2198–2207, Oct. 2022, doi: 10.33395/sinkron.v7i3.11749.

I. D. Maramba and A. Chatterjee, “Continuous User Experience Monitoring of a Patient-Completed Preoperative Assessment System in the United Kingdom: Cross-sectional Study,” JMIRx Med, vol. 3, no. 1, p. e31679, Jan. 2022, doi: 10.2196/31679.

W. A. Pratama, F. Fatmasari, E. Supratman, and T. Oktarina, “Evaluasi Usability Website Polrestabes Palembang Menggunakan System Usability Scale (SUS),” Jurnal Teknologi Informatika dan Komputer, vol. 9, no. 2, pp. 704–715, Sep. 2023, doi: 10.37012/jtik.v9i2.1745.

G. E. Saputra, R. Khalida, and R. Nurmalika, “Evaluation Of User Experience Tlx Training Gate For Competitive Programming Learning Using User Experience Questionnaire And System Usability Scale,” no. 2, 2022, [Online]. Available: https://tlx.toki.id/training

A. A. Amrullah, A. Anisya, A. Syahrani, I. Warman, and M. Minarni, “Survei Pengalaman Pengguna Dompet Digital Menggunakan Metode Kuesioner,” Device, vol. 13, no. 2, pp. 156–161, Nov. 2023, doi: 10.32699/device.v13i2.5526.

K. Wongso and W. P. Sari, “Analisa UX Writing terhadap User Experience pada Pengguna Aplikasi Grab,” Prologia, vol. 4, no. 1, p. 1, Feb. 2020, doi: 10.24912/pr.v4i1.6415.

M. Hyzy et al., “System Usability Scale Benchmarking for Digital Health Apps: Meta-analysis,” JMIR Mhealth Uhealth, vol. 10, no. 8, p. e37290, Aug. 2022, doi: 10.2196/37290.

S. Akter, S. F. Wamba, A. Gunasekaran, R. Dubey, and S. J. Childe, “How to improve firm performance using big data analytics capability and business strategy alignment?,” Int J Prod Econ, vol. 182, pp. 113–131, Dec. 2016, doi: 10.1016/j.ijpe.2016.08.018.

J. Brooke, “SUS: A quick and dirty usability scale,” in Usability Evaluation in Industry, P. W. Jordan, B. Thomas, I. L. McClelland, and B. A. Weerdmeester, Eds., London: Taylor & Francis, 1996, pp. 189–194.

B. E. Hayes, Measuring Customer Satisfaction and Loyalty: Survey Design, Use, and Statistical Analysis Methods, 3rd ed. Milwaukee: ASQ Quality Press, 2008.

A. Beleza, S. Nascimento da Mota, M. Lima, and T. Conte, “UX4ALL: A Repository of User Experience Evaluation Methods,” in Proceedings of the 27th International Conference on Enterprise Information Systems, SCITEPRESS - Science and Technology Publications, 2025, pp. 614–621. doi: 10.5220/0013435100003929.

S. J. Oudbier et al., “Patients’ Experienced Usability and Satisfaction With Digital Health Solutions in a Home Setting: Instrument Validation Study,” JMIR Med Inform, vol. 13, pp. e63703–e63703, Jan. 2025, doi: 10.2196/63703.

N. Setiyawati and D. H. Bangkalang, “The Comparison of Evaluation on User Experience and Usability of Mobile Banking Applications Using User Experience Questionnaire and System Usability Scale,” in International Academic Symposium of Social Science 2022, Basel Switzerland: MDPI, Sep. 2022, p. 87. doi: 10.3390/proceedings2022082087.