Category-Dependent Effectiveness of Web-Based and AI-Generated Usage Examples in Modern Greek Pedagogical Lexicography
Abstract
Background. Usage examples are an important feature of pedagogical dictionaries, as they help learners understand meaning, context, and lexical use. Although web-based resources and large language models offer new possibilities for generating dictionary examples, their pedagogical suitability has not yet been sufficiently examined in Modern Greek.
Purpose. To address this gap, this exploratory study compares Web-based and AI-generated usage examples across three lexically challenging categories in Modern Greek: polysemous verbs and nouns, and idiomatic expressions. It examines whether and how the pedagogical suitability of each source varies according to lexical complexity for learners aged 11-15 and explores their potential integration in pedagogical lexicography.
Materials and Methods. A small-scale exploratory dataset of 81 usage examples was collected from Web sources and generated by GPT-5.3 Instant and DeepSeek-V3 and was evaluated using a combined framework drawing on GDEX criteria and pedagogical appropriateness indicators on a three-point ordinal scale. A learner validation phase with 41 students (aged 11-15) was also conducted.
Results. Within the analysed dataset, the findings suggest category-related differences across lexical categories. Web-derived examples showed greater semantic diversity and higher correct meaning identification rates in the learner evaluation (56.50%), particularly for figurative and idiomatic uses. AI-generated examples, especially those produced by GPT-5.3, showed higher intelligibility scores and performed more consistently for semantically stable items such as polysemous nouns, but exhibited limitations in idiomatic language, with correct meaning identification for GPT-5.3 dropping to 32.37% for specific idiomatic items. DeepSeek-generated examples showed the most pronounced limitations within the analysed categories, particularly in typicality and pedagogical appropriateness.
Conclusion. The findings suggest that, within the analysed dataset, the pedagogical value of usage examples depends not only on their source, but also on the type of lexical complexity involved. Neither Web-based nor AI-generated examples can independently meet the requirements of a learner-oriented dictionary. Instead, the findings support a hybrid category-sensitive approach that combines Web-based authenticity, expert curation, selective AI assistance, and attention to learners’ actual comprehension. The methodological framework developed here may offer a basis for future comparative work in other morphologically complex and idiomatically rich languages, though transferability remains to be tested.
Downloads
References
Abdelrahman, M. (2024). Hallucination in low-resource languages: Amplified risks and mitigation strategies for multilingual LLMs. Journal of Applied Big Data Analytics, Decision-Making, and Predictive Modelling Systems, 8(12), 17-24. https://polarpublications.com/index.php/JABADP/article/view/2024-12-10
Alexandri, K. (2025). Τεχνητή Νοημοσύνη και Λεξικογραφία: Μπορεί ένα διαλογικό ρομπότ να αντικαταστήσει το παιδαγωγικό λεξικό; [Artificial intelligence and lexicography: Can a chatbot replace the pedagogical dictionary?]. In I. Spantidakis, K. Ntinas, V. Chatzinikita, & E. Griva (Eds.), Language, education and artificial intelligence (pp. 159-174). University of Crete.
Alexandri, K., & Iordanidou, A. (2025). Τεχνητή Νοημοσύνη και λεξικογραφία: Η συμβολή της ΤΝ στην επίλυση γλωσσικών αποριών [Artificial intelligence and lexicography: The contribution of AI to solving language-related queries]. Studies in Greek Linguistics, 44, 33-44. https://www.eshop.ins-auth.gr/images/companies/1/archive/MEG_PLIRI/MEG_44_33_44.pdf
Almeman, F. Y., Schockaert, S., & Espinosa Anke, L. (2024). WordNet under scrutiny: Dictionary examples in the era of large language models. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024) (pp. 17683-17695). European Language Resources Association and ICCL. http://dx.doi.org/10.63317/4skerayorcxh.
Arkhangelskiy, T. (2019). Corpus of usage examples: What is it good for? In A. Arppe, J. Good, M. Hulden, J. Lachler, A. Palmer, L. Schwartz, & M. Silfverberg (Eds.), Proceedings of the 3rd Workshop on the Use of Computational Methods in the Study of Endangered Languages Volume 1 (Papers) (pp. 56-63). Association for Computational Linguistics. http://dx.doi.org/10.33011/computel.v1i.411
Arnett, C., & Bergen, B. (2025). Why do language models perform worse for morphologically complex languages? In O. Rambow, L. Wanner, M. Apidianaki, H. Al-Khalifa, B. D. Eugenio, & S. Schockaert (Eds.), Proceedings of the 31st International Conference on Computational Linguistics (pp. 6607-6623). Association for Computational Linguistics. https://aclanthology.org/2025.coling-main.441/
Atkins, B. S., & Rundell, M. (2008). The Oxford guide to practical lexicography. Oxford University Press. http://dx.doi.org/10.1093/oso/9780199277704.001.0001
Beliga, S., & Filipović Petrović, I. (2024). Large language models supporting lexicography: Conceptual organization of Croatian idioms. In Š. Arhar Holdt & T. Erjavec (Eds.), Proceedings of the Conference on Language Technologies and Digital Humanities (pp. 23-46). Ljubljana University Press. http://dx.doi.org/10.5281/zenodo.13912515
Cai, B., Clarence, N., Liang, D., & Hotama, S. (2024). Low-cost generation and evaluation of dictionary example sentences. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1: Long Papers (pp. 3538-3549). Association for Computational Linguistics. https://aclanthology.org/2024.naacl-long.194.pdf
Chi, A. (2022). Researching pedagogical lexicography. In H. Jackson (Ed.), The Bloomsbury Handbook of Lexicography (2nd ed., pp. 145-164). Bloomsbury Publishing. https://hdl.handle.net/1783.1/116725
Crosthwaite, P., & Baisa, V. (2023). Generative AI and the end of corpus-assisted data-driven learning? Not so fast! Applied Corpus Linguistics, 3(3), Article 100066. http://dx.doi.org/10.1016/j.acorp.2023.100066
Davies, M. (2018). Corpus-based studies of lexical and semantic variation: The importance of both corpus size and corpus design. In A. Meurman-Solin, C. Seoane, & M. José López-Couso (Eds.), From data to evidence in English language research (pp. 66-87). Brill. http://dx.doi.org/10.1163/9789004390652_004
de Schryver, G.-M. (2024). The road towards fine-tuned LLMs for lexicography. In S. Krek (Ed.), Book of abstracts of the Workshop “Large Language Models and Lexicography”, 8 October 2024, Cavtat, Croatia (pp. 6-11). ELEXIS Association. http://hdl.handle.net/1854/LU-01JVRF7H9DE5TTXV4GPNEW30PC
Durward, M., & Thomson, C. (2024). Evaluating vocabulary usage in LLMs. In E. Kochmar, M. Bexte, J. Burstein, A. Horbach, R. Laarmann-Quante, A. Tack, V. Yaneva, & Z. Yuan (Eds.), Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024) (pp. 266-282). Association for Computational Linguistics. https://aclanthology.org/2024.bea-1.22/
Fuertes-Olivera, P. A. (2012). Lexicography and the Internet as a (re-)source. Lexicographica, 28, 49-70. http://dx.doi.org/10.1515/lexi.2012-0005
Garipov, T., Morozov, D., Gubarkova, Y., Kozerenko, A., & Glazkova, A. (2026). An experimental study of automating explanatory dictionary compilation with language models. In M. Bakaev et al. (Eds.), Internet and modern society: IMS 2025 (Communications in Computer and Information Science, Vol. 2671, pp. 117-131). Springer. http://dx.doi.org/10.1007/978-3-032-04958-2_9
Gatto, M. (2025). Web as corpus for data-driven learning. In The Palgrave Encyclopedia of Computer-Assisted Language Learning (pp. 1-8). Springer Nature Switzerland. http://dx.doi.org/10.1007/978-3-031-51447-0_66-1
Gavriilidou, M., Lampropoulou, P., & Angelakos, K. (2007). Ερμηνευτικό λεξικό νέας ελληνικής Α΄, Β΄, Γ΄ Γυμνασίου [Explanatory dictionary of modern Greek for grades 7, 8, and 9 of secondary school]. OEDB.
Hoenen, A., Koc, C., & Rahn, M. D. (2020). A manual for Web corpus crawling of low resource languages. Umanistica Digitale, 4(8). http://dx.doi.org/10.6092/issn.2532-8816/9931
Ide, Y., Nohejl, A., Tanner, J., Yanaka, H., Lindsay, C., & Watanabe, T. (2026). Towards automated lexicography: Generating and evaluating definitions for learner’s dictionaries. arXiv. http://dx.doi.org/10.48550/arXiv.2601.01842
Iordanidou, A., Mantzari, E., & Pantazara, M. (Eds). (2007). Λεξικό της ελληνικής ως ξένης γλώσσας για μαθητές της δευτεροβάθμιας εκπαίδευσης [Dictionary of Greek as a foreign language for secondary school students]. National and Kapodistrian University of Athens.
Jakubíček, M., & Rundell, M. (2023). The end of lexicography? Can ChatGPT outperform current tools for postediting lexicography? In M. Medveď, M. Měchura, I. Kosem, J. Kallas, C. Tiberius, & M. Jakubíček (Eds.), Electronic lexicography in the 21st century (eLex 2023): Invisible Lexicography. Proceedings of the eLex 2023 Conference (pp. 518-533). Lexical Computing CZ s.r.o. https://elex.link/ojs/index.php/elex/article/view/46/30
Kapsalis, G., Paskhalis, A., Tsialos, S., & Tsioulis, K. (2007). Ορθογραφικό-ερμηνευτικό λεξικό Δ΄, Ε΄, ΣΤ΄ Δημοτικού [Orthographic and explanatory dictionary for grades 4, 5, and 6 of primary school: Our dictionary]. OEDB.
Kilgarriff, A. (2007). Googleology is bad science. Computational Linguistics, 33(1), 147-151. http://dx.doi.org/10.1162/coli.2007.33.1.147
Kilgarriff, A., & Grefenstette, G. (2003). Introduction to the special issue on the web as corpus. Computational Linguistics, 29(3), 333-347. http://dx.doi.org/10.1162/089120103322711569
Kilgarriff, A., Husák, M., McAdam, K., Rundell, M., & Rychlý, P. (2008). GDEX: Automatically finding good dictionary examples in a corpus. In A. DeCesaris & E. Bernal (Eds.), Proceedings of the XIII EURALEX International Congress (pp. 425-432). Institut Universitari de Lingüística Aplicada, Universitat Pompeu Fabra. https://euralex.org/elx_proceedings/Euralex2008/
Kosem, I., Gantar, P., Holdt, Š. A., Gapsa, M., Zgaga, K., & Krek, S. (2024). AI in lexicography at the University of Ljubljana. In S. Krek (Ed.), Book of abstracts of the workshop “Large Language Models and Lexicography”, 8 October 2024, Cavtat, Croatia (pp. 29-32). ELEXIS Association. https://www.cjvt.si/wp-content/uploads/2024/10/LLM-Lex_2024_Book-of-Abstracts.pdf#page=11
Kosem, I., Koppel, K., Zingano Kuhn, T., Michelfeit, J., & Tiberius, C. (2019). Identification and automatic extraction of good dictionary examples: Τhe case(s) of GDEX. International Journal of Lexicography, 32(2), 119-137. http://dx.doi.org/10.1093/ijl/ecy014
Krstev, C., & Stanković, R. (2023). Language report Serbian. In G. Rehm & A. Way (Eds.), European language equality. (pp. 203-206). Springer. http://dx.doi.org/10.1007/978-3-031-28819-7_32
Laippala, V., Egbert, J., Biber, D., & Kyröläinen, A.-J. (2021). Exploring the role of lexis and grammar for the stable identification of register in an unrestricted corpus of web documents. Language Resources and Evaluation, 55(3), 757-788. http://dx.doi.org/10.1007/s10579-020-09519-z
Lew, R. (2015). Research into the use of online dictionaries. International Journal of Lexicography, 28(2), 232-253. http://dx.doi.org/10.1093/ijl/ecv010
Lew, R. (2023). ChatGPT as a COBUILD lexicographer. Humanities and Social Sciences Communications, 10, Article 704. http://dx.doi.org/10.1057/s41599-023-02119-6
Lew, R. (2024). Dictionaries and lexicography in the AI era. Humanities and Social Sciences Communications, 11, Article 426. http://dx.doi.org/10.1057/s41599-024-02889-7
Lewandowska-Tomaszczyk, B., & Pawłowski, G. (2025). Testing ChatGPT on terminology generation, definitions: Translation, and ontology creation in German, English and Polish. Research in Language, 23, 320-340. http://dx.doi.org/10.18778/1731-7533.23.20
Marković, A. & Stanković, R. (2025). So close but still far: Case study on application of LLMs in idioms identification, definition, and generation of illustrative examples. In I. Kosem, M. Jakubícek, M. Medved’, K. Zgaga, Š. Arhar Holdt, T. Munda, & A. Salgado (Eds.), Electronic lexicography in the 21st Century (eLex 2025): Intelligent lexicography. Proceedings of the eLex 2025 Conference (pp. 79-94). Lexical Computing.
Merx, R., Vylomova, E., & Kurniawan, K. (2024). Generating bilingual example sentences with large language models as lexicography assistants. In T. Baldwin, S. J. Rodríguez Méndez, & N. Kuo (Eds.), Proceedings of the 22nd Annual Workshop of the Australasian Language Technology Association (pp. 64-74). Association for Computational Linguistics. https://aclanthology.org/2024.alta-1.5/
Nasution, A. H., & Onan, A. (2024). ChatGPT label: Comparing the quality of human-generated and LLM-generated annotations in low-resource language NLP tasks. IEEE Access, 12, 71876-71900. http://dx.doi.org/10.1109/access.2024.3402809
Phoodai, C., & Rikk, R. (2023). Exploring the capabilities of ChatGPT for lexicographical purposes: A comparison with Oxford Advanced Learner’s Dictionary within the microstructural framework. In I. Kosem, M. C. Culy, M. Gantar, J. Kallas, S. Krek, & H. Tuul (Eds.), Electronic lexicography in the 21st Century (eLex 2023): Proceedings of the eLex 2023 Conference (pp. 345-375). Lexical Computing CZ. https://elex.link/ojs/index.php/elex/article/view/35
Rundell, M. (2024). Automating the creation of dictionaries: Are we nearly there? Humanising Language Teaching, 26(1).
Rundell, M., Jakubíček, M., Kovář, V., Matuška, O., & Cukr, M. (2025). Lexicom at 25: Reflections on the changing world of lexicography and language technology. In I. Kosem, M. Jakubíček, M. Medveď, K. Zgaga, Š. Arhar Holdt, T. Munda, & A. Salgado (Eds.), Electronic lexicography in the 21st century (eLex 2025): Intelligent lexicography. Proceedings of the eLex 2025 Conference (pp. 136-149). Lexical Computing CZ. https://elex.link/elex2025/wp-content/uploads/eLex2025-09-Rundell_etal.pdf
Tarp, S. (2014). Dictionaries in the internet era: Innovation or business as usual? Enrique Alcaraz Memorial Lecture 2014. Revista Alicantina de Estudios Ingleses, 27, 233. http://dx.doi.org/10.14198/raei.2014.27.13
Tarp, S., & Gouws, R. (2020). Reference skills or human-centered design: Towards a new lexicographical culture. Lexikos, 30, 470-498. http://dx.doi.org/10.5788/30-1-1600
Vakalopoulou, A. (2000). Το πρώτο μου λεξικό για το Δημοτικό [My first dictionary for primary school]. Patakis Editions.
Vakalopoulou, A., & Iordanidou, A. (2001). Το λεξικό του Δημοτικού [The dictionary of primary school]. Patakis Editions.
Verlinde, S., & Binon, J. (2009). Pedagogical lexicography revisited. In H. Bergenholtz, S. Nielsen, & S. Tarp (Eds.), Lexicography at a crossroads: Dictionaries and encyclopedias today, lexicographical tools tomorrow (pp. 69-89). Peter Lang.
Zhong, A. (2025). Prompts for language learners: A practical guide to using DeepSeek as a dictionary. Lexikos, 35(1), 274-285. http://dx.doi.org/10.5788/35-1-2037.
Zhong, T., Yang, Z., Liu, Z., Zhang, R., You, W., Liu, Y., Sun, H., Pan, Y., Li, Y., Zhou, Y., Jiang, H., Chen, J., Li, X., & Liu, T. (2026). Opportunities and challenges of large language models for low-resource languages in humanities research. arXiv. http://dx.doi.org/10.48550/arXiv.2412.04497
Copyright (c) 2026 National Research University Higher School of Economics
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the Copyright Notice.
