When fluent advice becomes unsafe: evaluating accuracy, uncertainty, and communicative risk in Indonesian AI-generated health responses
DOI:
https://doi.org/10.67490/ijml.v1i2.943Keywords:
AI-generated health advice, communicative risk, health communication, Indonesian digital health, uncertaintyAbstract
Background: AI-generated health advice is increasingly encountered by Indonesian users as a convenient source of explanation, reassurance, and self-care guidance, yet its linguistic fluency may conceal biomedical incompleteness, weak uncertainty marking, and unsafe action cues. Objective: This study aims to evaluate Indonesian AI-generated health responses by examining how accuracy, uncertainty communication, and communicative risk interact across safety-sensitive health domains. Method: Using a qualitative-dominant corpus design, this study analysed 15 Indonesian prompt-response units linked to official public-health references and coded each response for accuracy, completeness, disclaimer relevance, referral specificity, directive force, tone, and risk level. Results: Findings show that only 4 responses were both accurate and complete, whereas 5 were accurate but incomplete, 4 were partially aligned with weak red-flag articulation, and 2 were misleading or unsafe. Uncertainty was often present as generic disclaimer language, but it was not consistently translated into specific referral advice or usable escalation thresholds. Implication: Communicative risk emerged when empathetic, calm, or accessible responses softened urgency, normalised self-management, or implied authority beyond available clinical information. Novelty: This study contributes a linguistic-pragmatic model for AI-health evaluation by demonstrating that safe advice requires alignment between factual accuracy, explicit uncertainty, and responsible communicative force.
Downloads
References
[1] DataReportal, “Digital 2025: Indonesia,” 2025. [Online]. Available: https://datareportal.com/reports/digital-2025-indonesia
[2] L. De Angelis, F. Baglivo, G. Arzilli, G. P. Privitera, P. Ferragina, A. Tozzi, and C. Rizzo, “ChatGPT and the rise of large language models: The new AI-driven infodemic threat in public health,” Frontiers in Public Health, vol. 11, 2023, doi: 10.3389/fpubh.2023.1166120.
[3] L. Weidinger, J. F. J. Mellor, M. Rauh, C. Griffin, J. Uesato, P.-S. Huang, et al., “Ethical and social risks of harm from language models,” arXiv, abs/2112.04359, 2021.
[4] N. Laila, “Medical jargon and patient comprehension: A linguistic analysis of informed consent practices in Indonesian hospitals,” Indonesian Journal of Medical Linguistics, vol. 1, no. 1, pp. 13-25, 2026.
[5] M. Nurtyas, “Language barriers in healthcare delivery: A sociolinguistic case study of multilingual patients in Indonesian urban clinics,” Indonesian Journal of Medical Linguistics, vol. 1, no. 1, pp. 26-38, 2026.
[6] F. Busch, L. Hoffmann, C. Rueger, E. V. Van Dijk, R. Kader, E. Ortiz-Prado, et al., “Current applications and challenges in large language models for patient care: A systematic review,” Communications Medicine, vol. 5, 2025, doi: 10.1038/s43856-024-00717-2.
[7] D. Wang and S. Zhang, “Large language models in medical and healthcare fields: Applications, advances, and challenges,” Artificial Intelligence Review, vol. 57, 2024, doi: 10.1007/s10462-024-10921-0.
[8] R. K. Arora, J. Wei, R. S. Hicks, P. Bowman, J. Q. Candela, F. Tsimpourlas, et al., “HealthBench: Evaluating large language models towards improved human health,” arXiv, abs/2505.08775, 2025, doi: 10.48550/arxiv.2505.08775.
[9] Y. Hua, W. Xia, D. W. Bates, G. L. Hartstein, H. Kim, M. Li, et al., “Standardizing and scaffolding health care AI-chatbot evaluation: Systematic review,” JMIR AI, vol. 4, 2025, doi: 10.2196/69006.
[10] M. Neary, E. Fulton, V. Rogers, J. Wilson, Z. Griffiths, R. Chuttani, and P. M. Sacher, “Think FAST: A novel framework to evaluate fidelity, accuracy, safety, and tone in conversational AI health coach dialogues,” Frontiers in Digital Health, vol. 7, 2025, doi: 10.3389/fdgth.2025.1460236.
[11] B. De Busser, L. Roth, and H. De Loof, “The role of large language models in self-care: A study and benchmark on medicines and supplement guidance accuracy,” International Journal of Clinical Pharmacy, vol. 47, pp. 1001-1010, 2024, doi: 10.1007/s11096-024-01839-2.
[12] S. Giorgi, K. Isman, T. Liu, Z. Fried, J. Sedoc, and B. Curtis, “Evaluating generative AI responses to real-world drug-related questions,” Psychiatry Research, vol. 339, p. 116058, 2024, doi: 10.1016/j.psychres.2024.116058.
[13] S. Sharma, A. M. Alaa, and R. Daneshjou, “A longitudinal analysis of declining medical safety messaging in generative AI models,” NPJ Digital Medicine, vol. 8, 2025, doi: 10.1038/s41746-025-01943-1.
[14] J. Yau, S. Saadat, E. Hsu, L. Murphy, J. S. Roh, J. Suchard, et al., “Accuracy of prospective assessments of 4 large language model chatbot responses to patient questions about emergency care: Experimental comparative study,” Journal of Medical Internet Research, vol. 26, 2024, doi: 10.2196/60291.
[15] S. Shekar, P. Pataranutaporn, C. Sarabu, G. A. Cecchi, and P. Maes, “People over trust AI-generated medical responses and view them to be as valid as doctors, despite low accuracy,” arXiv, abs/2408.15266, 2024, doi: 10.48550/arxiv.2408.15266.
[16] J. Zhang and Z.-M. Zhang, “Ethics and governance of trustworthy medical artificial intelligence,” BMC Medical Informatics and Decision Making, vol. 23, 2023, doi: 10.1186/s12911-023-02103-9.
[17] A. Tilton, B. E. Caplan, and B. J. Cole, “Generative AI in consumer health: Leveraging large language models for health literacy and clinical safety with a digital health framework,” Frontiers in Digital Health, vol. 7, 2025, doi: 10.3389/fdgth.2025.1616488.
[18] J. C. L. Chow and K. Li, “Large language models in medical chatbots: Opportunities, challenges, and the need to address AI risks,” Information, vol. 16, no. 7, p. 549, 2025, doi: 10.3390/info16070549.
[19] G. Deiana, M. Dettori, A. Arghittu, A. Azara, G. Gabutti, and P. Castiglia, “Artificial intelligence and public health: Evaluating ChatGPT responses to vaccination myths and misconceptions,” Vaccines, vol. 11, no. 7, 2023, doi: 10.3390/vaccines11071217.
[20] I. S. Schwartz, K. E. Link, R. Daneshjou, and N. W. Cortés-Penfield, “Black box warning: Large language models and the future of infectious diseases consultation,” Clinical Infectious Diseases, vol. 78, no. 3, pp. 860-866, 2023, doi: 10.1093/cid/ciad633.
[21] L. Balcombe, “AI chatbots in digital mental health,” Informatics, vol. 10, no. 4, p. 82, 2023, doi: 10.3390/informatics10040082.
[22] L. Hipgrave, J. Goldie, S. Dennis, and A. Coleman, “Balancing risks and benefits: Clinicians’ perspectives on the use of generative AI chatbots in mental healthcare,” Frontiers in Digital Health, vol. 7, 2025, doi: 10.3389/fdgth.2025.1606291.
[23] S. Yim, D. W. Yoo, A. Polymerou, Y. Liu, and K. Saha, “Generative AI for eating disorders: Linguistic comparison with online support and qualitative analysis of harms,” International Journal of Eating Disorders, 2025, doi: 10.1002/eat.24604.
[24] P. Festor, M. Nagendran, A. Gordon, A. A. Faisal, and M. Komorowski, “Safety of human-AI cooperative decision-making within intensive care: A physical simulation study,” PLOS Digital Health, vol. 4, p. e0000726, 2025, doi: 10.1371/journal.pdig.0000726.
[25] J. Zhou, Y. Zhang, Q. Luo, A. G. Parker, and M. De Choudhury, “Synthetic lies: Understanding AI-generated misinformation and evaluating algorithmic and human solutions,” Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, 2023, doi: 10.1145/3544548.3581318.
[26] I. L. Nazulpa, “Doctor–patient communication in rural Indonesia: A pragmatic study of directive speech acts in clinical consultations,” Indonesian Journal of Medical Linguistics, vol. 1, no. 1, pp. 1-12, 2026.
[27] Y. D. Rusita, “Constructing empathy through language: A linguistic perspective on compassionate communication in mental health services,” Indonesian Journal of Medical Linguistics, vol. 1, no. 1, pp. 67-81, 2026.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Novela Eka Candra Dewi (Author)
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Creative Commons Attribution 4.0 International License