Automatic readability assessment of Indonesian educational texts using hybrid NLP approaches

Tifani Yuliyana

doi:10.67490/ijcl.v1i1.552

Authors

Tifani Yuliyana STKIP Taman Siswa Bima Author

DOI:

https://doi.org/10.67490/ijcl.v1i1.552

Keywords:

educational texts, hybrid NLP, Indonesian language, machine learning, readability assessment

Abstract

Background: The increasing complexity of Indonesian educational texts across print and digital platforms raises concerns about mismatches with students’ reading capacities, while existing readability formulas remain largely English-centric. Objective: This study develops and evaluates an automatic readability assessment framework for Indonesian texts integrating surface metrics, linguistic features, and machine learning. Method: A stratified corpus of textbooks and digital materials was analyzed using readability indices, lexical–morphological–syntactic features, and supervised models with cross-validation. Results: Surface complexity rises across levels but shows overlap, indicating limits of traditional metrics; linguistic features such as lexical density, nominalization, and morphological complexity strongly predict readability, while hybrid ensemble models achieve the highest accuracy and lowest misclassification. Implication: These findings support the need for language-specific, multidimensional readability tools to improve text design and educational alignment in Indonesian contexts. Novelty: This study proposes a linguistically grounded hybrid NLP framework that reconceptualizes readability and enables scalable, automated evaluation of Indonesian educational texts.

References

1] S. Uçar, I. Aldabe, N. Aranberri, and A. Arruarte, “Exploring Automatic Readability Assessment for Science Documents within a Multilingual Educational Context,” International Journal of Artificial Intelligence in Education, vol. 34, pp. 1417–1459, Feb. 2024, doi: 10.1007/s40593-024-00393-2.

[2] F. Az-Zahra and A. Romadhony, “Text Readability Analysis for Indonesian School Textbook,” 2023 International Conference on Data Science and Its Applications (ICoDSA), pp. 30–35, Aug. 2023, doi: 10.1109/icodsa58501.2023.10277302.

[3] A. A. Hakim, E. Setyaningsih, and D. Cahyaningrum, “Examining the Readability Level of Reading Texts in English Textbook for Indonesian Senior High School,” Journal of English Language Studies, Mar. 2021, doi: 10.30870/jels.v6i1.8898.

[4] F. Azima, M. Ping, and A. K. Amarullah, “Are these texts readable? An Analysis on the Readability Level of English Textbooks for Indonesian High Schools,” Borneo Educational Journal (Borju), Nov. 2022, doi: 10.24903/bej.v5i1.1095.

[5] S. Crossley, “Developing Linguistic Constructs of Text Readability Using Natural Language Processing,” Scientific Studies of Reading, vol. 29, pp. 138–160, Nov. 2024, doi: 10.1080/10888438.2024.2422365.

[6] J. Zeng, X. Yu, X. Tong, and W. Xiao, “Self-Supervised Collaborative Information Bottleneck for Text Readability Assessment,” pp. 25814–25822, Apr. 2025, doi: 10.1609/aaai.v39i24.34774.

[7] J. M. Imperial and E. Ong, “Diverse Linguistic Features for Assessing Reading Difficulty of Educational Filipino Texts,” ArXiv, vol. abs/2108.00241, Jul. 2021.

[8] T. M. Siregar and W. Purbani, “Prominent linguistic features of pedagogical texts to provide consideration for authentic text simplification,” Studies in English Language and Education, Jan. 2024, doi: 10.24815/siele.v11i1.30815.

[9] S. Maqsood et al., “Assessing English language sentences readability using machine learning models,” PeerJ Computer Science, vol. 8, Jan. 2022, doi: 10.7717/peerj-cs.818.

[10] R. Wilkens, P. Watrin, R. Cardon, A. Pintard, I. Gribomont, and T. François, “Exploring hybrid approaches to readability: experiments on the complementarity between linguistic features and transformers,” pp. 2316–2331, Jan. 2024, doi: 10.18653/v1/2024.findings-eacl.153.

[11] D. Palma and C. Soto, “Combining large language models with linguistic features for the readability complexity assessment of texts,” AHFE International, Jan. 2025, doi: 10.54941/ahfe1007028.

[12] E. Gourlay et al., “Readability and complexity of written information presented to hospitalised patients for trial consent during the COVID-19 pandemic in the UK: a retrospective document analysis,” BMJ Open, vol. 15, Mar. 2025, doi: 10.1136/bmjopen-2024-089447.

[13] F. Indrawan, S. Ma’rifatulloh, and E. Hardinanto, “The Correlation Between Indonesian Senior High School Students’ Reading Comprehension and Text Readability,” Ed-Humanistics : Jurnal Ilmu Pendidikan, Nov. 2024, doi: 10.33752/ed-humanistics.v9i02.8259.

[14] R. Adyanthaya and R. S. P, “YenLP_CS@DravidianLangTech 2025: Sentiment Analysis on Code-Mixed Tamil-Tulu Data Using Machine Learning and Deep Learning Models,” Proceedings of the Fifth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages, Jan. 2025, doi: 10.18653/v1/2025.dravidianlangtech-1.50.

[15] S. M. Hassanin, E. M. Al Bayomy, and M. A. Eleleidy, “Leveraging Machine Learning and Natural Language Processing for Emotional and Thematic Analysis in Three Selected Contemporary English Novels,” Theory and Practice in Language Studies, vol. 15, no. 12, pp. 3833–3840, 2025, doi: 10.17507/tpls.1512.03.

[16] A. Fawaid, P. Handayani, and Y. A. Abdillah, “E-Portofolio in Improving Critical Thinking and Self-Management through Lesson Study: A Study on Writing Pedagogy in Higher Education,” in 2024 10th International Conference on Education and Technology (ICET), IEEE, Oct. 2024, pp. 149–154. doi: 10.1109/icet64717.2024.10778453.

[17] A. Fawaid, L. Y. Solehah, and R. Assya’bani, “Structured Infographic Worksheet through Lesson Study in Improving Madrasa Students’ Writing Skills in Indonesian Language Subject,” Tarbiyah: Jurnal Ilmiah Kependidikan, vol. 13, no. 2, pp. 273–290, 2024, doi: 10.18592/tarbiyah.v13i2.13849.

[18] M. Pradeep, T. Sasivardhan, G. Bodana, K. Shilpa, K. Savalapurapu, and G. C. Babu, “Natural Language Processing for Literacy Text Mining: Extracting Knowledge From British National Corpus,” presented at the Proceedings of the 6th International Conference on Inventive Research in Computing Applications, ICIRCA 2025, 2025, pp. 1816–1821. doi: 10.1109/ICIRCA65293.2025.11089848.

[19] N. Varghese and M. Punithavalli, “Lexical and semantic analysis of sacred texts using machine learning and natural language processing,” International Journal of Scientific and Technology Research, vol. 8, no. 12, pp. 3133–3140, 2019.

[20] P. Ardi, Y. D. Oktafiani, N. Widianingtyas, O. Dekhnich, and U. Widiati, “Lexical Bundles in Indonesian EFL Textbooks: A Corpus Analysis,” Journal of Language and Education, Jun. 2023, doi: 10.17323/jle.2023.16305.

[21] F. Etfita, S. Wahyuni, A. Ahmad, W. Sudusinghe, and C. K. W. Gamage, “Revealing lexical bundles of non-native English essay at a private Islamic university: A corpus-based evaluation,” English Learning Innovation, Aug. 2025, doi: 10.22219/englie.v6i2.40821.

[22] S. Azalia, P. Widodo, P. Wiedarti, and T. Sudartinah, “Lexical Bundle Structure and Function of ChatGPT-Generated Essay: Corpus-Based Study of Advanced Learners in Indonesia,” JOALL (Journal of Applied Linguistics and Literature), Aug. 2025, doi: 10.33369/joall.v10i2.39244.

[23] C. Anderson, M. Nguyen, and R. Coto-Solano, “Unsupervised, Semi-Supervised and LLM-Based Morphological Segmentation for Bribri,” Proceedings of the Fifth Workshop on NLP for Indigenous Languages of the Americas (AmericasNLP), Jan. 2025, doi: 10.18653/v1/2025.americasnlp-1.7.

[24] W. Chen and B. Fazio, “Morphologically-Guided Segmentation For Translation of Agglutinative Low-Resource Languages,” 2021, [Online]. Available: https://aclanthology.org/2021.mtsummit-loresmt.3/

[25] V. Demberg, “A Language-Independent Unsupervised Model for Morphological Segmentation,” pp. 920–927, Jun. 2007.

[26] A. Alhazmi, R. Mahmud, N. Idris, M. E. M. Abo, and C. Eke, “Code-mixing unveiled: Enhancing the hate speech detection in Arabic dialect tweets using machine learning models,” PLOS ONE, vol. 19, Jul. 2024, doi: 10.1371/journal.pone.0305657.

[27] G. A. Aranda-Corral, J. Borrego-Díaz, and J. Galán-Páez, “Concept learning consistency under three-way decision paradigm,” International Journal of Machine Learning and Cybernetics, vol. 13, no. 10, pp. 2977–2999, 2022, doi: 10.1007/s13042-022-01576-w.

[28] G. Diri, P. Obiorah, and H. Du, “Comparative Study of Sentiment Analysis Techniques: Traditional Machine Learning vs. Deep Learning Approaches,” InSITE Conference, Jan. 2025, doi: 10.28945/5490.