Synchronic and Diachronic Predictors of Socialness Ratings of Words
Abstract
In recent works, a new psycholinguistic concept has been introduced and studied that is the socialness of a word. In particular, Diveica et al., 2022 presents a dictionary with socialness ratings obtained using a survey method. The socialness rating reflects word social significance. Unfortunately, the size of the existed dictionary with word socialness ratings is relatively small. In this paper, we propose linear and neural network predictors of socialness ratings by using pre-trained fasttext vectors as input. The obtained Spearman`s correlation coefficient between human socialness ratings and machine ones is 0.869. The trained models allowed obtaining socialness ratings for 2 million English words, as well as a wide range of words in 43 other languages. An unexpected result is that the linear model provides highly accurate estimate of the socialness ratings, which can be hardly further improved. Apparently, this is due to the fact that in the space of vectors representing words there is a selected direction responsible for meanings associated with socialness driven by of social factors influencing word representation and use. The article also presents a diachronic neural network predictor of concreteness ratings using word co-occurrence vectors as input data. It is shown that using a one-year data from a large diachronic corpus Google Books Ngram one can obtain accuracy comparable to the accuracy of synchronic estimates. We study some examples of words words that are characterised by significant changes in socialness ratings over the past 150 years. It is concluded that changes in socialness ratings can serve as a marker of word meaning change.
Downloads
References
Basile, P., & McGillivray, B. (2018). Exploiting the web for semantic change detection. Lecture Notes in Computer Science, 11198, 194-208. https://doi.org/10.1007/978-3-030-01771-2_13
Binder, J.R., Conant, L.L., Humphries, C.J., Fernandino, L., Simons, S.B., Aguilar, M., & Desai, R.H. (2016). Toward a brain-based componential semantic representation. Cognitive Neuropsychology, 33(3-4), 130-74. https://doi.org/10.1080/02643294.2016.1147426
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017). Enriching Word Vectors with Subword Information. Transactions of the Association for Computational Linguistics, 5, 135–146. https://doi.org/10.1162/tacl_a_00051
Bochkarev, V. V., Savinkov, A. V., & Shevlyakova, A. V. (2021). Estimation of imageability ratings of English words using neural networks. In I. Batyrshin, A. Gelbukh, & G. Sidorov (Eds.), Advances in Soft Computing. MICAI 2021. Lecture Notes in Computer Science (Vol. 13068, pp. 59–69). Springer. https://doi.org/10.1007/978-3-030-89820-5_5
Bochkarev, V. V., Khristoforov, S. V., Shevlyakova, A. V., & Solovyev, V. D. (2022). Neural network algorithm for detection of new word meanings denoting named entities. IEEE Access, 10, 68499–68512. https://doi.org/10.1109/ACCESS.2022.3186681
Bochkarev, V., Khristoforov, S., Shevlyakova, A., & Solovyev, V. (2024). Diachronic analysis of a word concreteness rating: Impact of semantic change. Lobachevskii Journal of Mathematics, 45, 961–971. https://doi.org/10.1134/S1995080224600559
Bochkarev, V., Solovyev, V., Nestik, T., & Shevlyakova, A. (2024). Variations in average word valence of Russian books over a century and social change. Journal of Mathematical Sciences, 285, 14–27. https://doi.org/10.1007/s10958-024-07419-z
Buechel, S., & Hahn, U. (2018). Word emotion induction for multiple languages as a deep multi-task learning problem. In Proceedings of NAACL-HLT 2018 (pp. 1907–1918). Association for Computational Linguistics. https://doi.org/10.18653/v1/N18-1173
Bullinaria, J., & Levy, J. (2007). Extracting semantic representations from word co-occurrence statistics: A computational study. Behavior Research Methods, 39, 510–526. https://doi.org/10.3758/BF03193020
Bullinaria, J. A., & Levy, J. P. (2012). Extracting semantic representations from word co-occurrence statistics: Stop-lists, stemming, and SVD. Behavior Research Methods, 44(3), 890–907. https://doi.org/10.3758/s13428-011-0183-8
Charbonnier, J., & Wartena, C. (2019). Predicting word concreteness and imagery. In Proceedings of the 13th International Conference on Computational Semantics - Long Papers (pp. 176–187). Association for Computational Linguistics. https://doi.org/10.18653/v1/W19-0415
Conneau, A., Lample, G., Denoyer, L., Ranzato, M. A., & Jégou, H. (2017). Word translation without parallel data. arXiv preprint arXiv:1710.04087.
Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) (pp. 4171–4186). Association for Computational Linguistics.
Diveica, V., Pexman, P. M., & Binney, R. J. (2023). Quantifying social semantics: An inclusive definition of socialness and ratings for 8388 English words. Behavior Research Methods, 55, 461–473. https://doi.org/10.3758/s13428-022-01810-x
Firth, J. (1957). A Synopsis of Linguistic Theory, 1930-55. In Studies in linguistic analysis (Special Volume of the Philological Society, pp. 1-31). Blackwell.
Grave, E., Bojanowski, P., Gupta, P., Joulin, A., & Mikolov, T. (2018). Learning word vectors for 157 languages. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). ELRA.
Gulordava, K., & Baroni, M. (2011). A distributional similarity approach to the detection of semantic change in the Google Books Ngram corpus. In Proceedings of the GEMS 2011 Workshop on Geometrical Models of Natural Language Semantics (pp. 67–71). Association for Computational Linguistics.
Harris, Z. (1970). Papers in Structural and Transformational Linguistics. Dordrecht: Reidel.
Hengchen, S., Tahmasebi, N., Schlechtweg, D., & Dubossarsky, H. (2021). Challenges for computational lexical semantic change. In N. Tahmasebi, L. Borin, A. Jatowt, Y. Xu, & S. Hengchen (Eds.), Computational approaches to semantic change (pp. 341–372). Language Science Press.
Joulin, A., Bojanowski, P., Mikolov, T., Jégou, H., & Grave, E. (2018). Loss in translation: Learning bilingual word mapping with a retrieval criterion. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (pp. 2979–2984). Association for Computational Linguistics.
Khristoforov, S., Bochkarev, V., & Shevlyakova, A. (2020). Recognition of parts of speech using the vector of bigram frequencies. In W. van der Aalst et al. (Eds.), Analysis of Images, Social Networks and Texts. AIST 2019, Communications in Computer and Information Science (Vol. 1086, pp. 132–142). Springer. https://doi.org/10.1007/978-3-030-39575-9_13
Koper, M., & Schulte im Walde, S. (2016). Automatically generated affective norms of abstractness, arousal, imageability and valence for 350,000 German lemmas. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16) (pp. 2595–2598). ELRA.
Lin, Y., Michel, J.-B., Aiden, E. L., Orwant, J., Brockman, W., & Petrov, S. (2012). Syntactic annotations for the Google Books Ngram Corpus. In H. Li, C.-Y. Lin, M. Osborne, G. G. Lee, & J. C. Park (Eds.), 50th Annual Meeting of the Association for Computational Linguistics 2012: Proceedings of the Conference (Vol. 2, pp. 238–242). Association for Computational Linguistics.
Linzmayer, O. (2004). Apple Confidential 2.0: The Definitive History of the World's Most Colorful Company (2nd ed.). No Starch Press.
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013). Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems (Vol. 26, pp. 3111–3119). Curran Associates, Inc.
Mohammad, S., Kiritchenko, S., & Zhu, X. (2013). NRC-Canada: Building the state-of-the-art in sentiment analysis of tweets. In Second Joint Conference on Lexical and Computational Semantics (SEM) (Vol. 2: Proceedings of the Seventh International Workshop on Semantic Evaluation, pp. 321–327). Association for Computational Linguistics.
Pantel, P. (2005). Inducing ontological co-occurrence vectors. In Proceedings of the 43rd Conference of the Association for Computational Linguistics (pp. 125–132). Association for Computational Linguistics. https://doi.org/10.3115/1219840.1219856
Pennington, J., Socher, R., & Manning, C. (2014). GloVe: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 1532–1543). Association for Computational Linguistics. https://doi.org/10.3115/v1/D14-1162
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018). Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) (pp. 2227–2237). Association for Computational Linguistics. https://doi.org/10.18653/v1/N18-1202
Pexman, P. M., Diveica, V., & Binney, R. J. (2022). Social semantics: The organization and grounding of abstract concepts. Philosophical Transactions of the Royal Society B: Biological Sciences, 378(1870), 20210363. https://doi.org/10.1098/rstb.2021.0363
Pilehvar, M. T., & Camacho-Collados, J. (2020). Embeddings in natural language processing: Theory and advances in vector representations of meaning. Morgan & Claypool Publishers.
Rubenstein, H., & Goodenough, J. (1965). Contextual correlates of synonymy. Communications of the ACM, 8(10), 627–633. https://doi.org/10.1145/365628.365657
Ryzhova, A., Ryzhova, D., & Sochenkov, I. (2021). Detection of semantic changes in Russian nouns with distributional models and grammatical features. Komp'juternaja Lingvistika i Intellektual'nye Tehnologii, 20, 597–606.
Schweizer, P., & Schweizer, R. (2004). The Bushes: Portrait of a dynasty (1st ed.). Doubleday.
Tang, X. (2018). A state-of-the-art of semantic change computation. Natural Language Engineering, 24(5), 649–676. https://doi.org/10.1017/S1351324918000220
Turney, P. D., & Pantel, P. (2010). From frequency to meaning: Vector space models of semantics. Journal of Artificial Intelligence Research, 37(1), 141–188. https://doi.org/10.1613/jair.2934
Wang, S., Zhang, Y., Shi, W., et al. (2023). A large dataset of semantic ratings and its computational extension. Scientific Data, 10, 106. https://doi.org/10.1038/s41597-023-01995-6
Weeds, J., Weir, D., & McCarthy, D. (2004). Characterising measures of lexical distributional similarity. In Proceedings of the 20th International Conference on Computational Linguistics (pp. 1015–1021). COLING.
Worth, P. (2023). Word embeddings and semantic spaces in natural language processing. International Journal of Intelligence Science, 13, 1–21.
Xu, Y., & Kemp, C. (2015). A computational evaluation of two laws of semantic change. In Proceedings of the 37th Annual Meeting of the Cognitive Science Society, CogSci 2015. Association for Computational Linguistics.
Copyright (c) 2024 National Research University Higher School of Economics
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the Copyright Notice.
