Lesson 7
Computational Linguistics
What is it
Difficulties
Project Examples
Difficulties
Project Examples
What is it
Computational linguistics works with unstructured linguistic data — text and speech.
A person can read a text and understand what it says
A computer can count what is written — it "understands" only binary code.
See Chinese room.
Language is a set of chains made of characters from some alphabet. Not all chains, but only those that satisfy some rules.
Text is a single chain constructed according to these rules.
A person can read a text and understand what it says
A computer can count what is written — it "understands" only binary code.
See Chinese room.
Language is a set of chains made of characters from some alphabet. Not all chains, but only those that satisfy some rules.
Text is a single chain constructed according to these rules.
Alphabet is a set of symbols that can be used to construct texts.
Natural language is the language that people speak to each other.
Text and speech are information encoded in natural language.
Theoretical linguistics tries to account for all the features of natural language.
Computer linguistics, on the other hand, simplifies language, looks for structures and patterns in it, and uses them to draw conclusions about the text.
Disciplines
- Computational linguistics — mathematical models applied to natural language.
- Natural language processing (NLP) — how to correctly analyse and generate natural language text; a discipline at the intersection of linguistics and artificial intelligence.
- Language technology — software for working with natural languages.
- Artificial Intelligence — other tasks related to language, such as language learning assistance or grammar checking.
Where is it used
1. Checking grammar, spelling, punctuation, stylistics, and readability:
2. Spam-filters:
3. Machine translation:
4. Speech recognition:
5. Chatbots and voice assistance:
6. Search Engines:
7. Optical character recognition (OCR):
8. Morphological analysis:
9. Syntactic analysis:
10. Sentiment analysis:
11. Genre analysis.
12. Plagiarism detection.
2. Spam-filters:
- Email spam folder.
3. Machine translation:
4. Speech recognition:
5. Chatbots and voice assistance:
- chatbots in banking apps;
- DeepSeek, ChatGPT, YandexGPT.
6. Search Engines:
- Baidu;
- Search engine optimization (SEO).
7. Optical character recognition (OCR):
- conversion of handwritten or printed text images into data;
- text recognition on pdf scans;
- Tabula — tutorial.
8. Morphological analysis:
- find all mentions of the verb "to be" in all forms in the text.
9. Syntactic analysis:
- select passive verbs and convert them to their active form.
10. Sentiment analysis:
- is there a positive or negative reaction on this news event.
11. Genre analysis.
12. Plagiarism detection.
13. Keywords extraction:
- if we need to quickly analyse a huge text and understand what is it about;
- QuitaUp.
14. Named entities recognition:
- to determine who's the most mentioned person in the news.
- Distant reading of literature;
- Project Gutenberg;
- Voyant Tools.
15. Bias analysis:
- describe and detect pro-government and contra-government news coverage bias.
16. Language documentation and systematisation:
- The Russian National Corpus;
- MURCO (search by Russian gestures);
- Spread the Sign (sign languages);
- British National Corpus.
17. Languages typologization and comparative linguistics. For instance, how Russian differs from English:
- inflection (word change) = the ability to change the form of a word depending on the situation;
- semantic homonymy (ambiguity);
- particle homonymy (ambiguity) — is "learning" a verb or a noun?
- word order = freedom of arrangement of words in a sentence.
18. Definition of authorship:
- An investigation by digital humanities scholar Daniil Skorinkin into who's really writing the denunciations ("The author of thousands of Z-denunciations, 'Anna Vasilyevna Korobkova,' is actually a young Stalinist Wikipedian and NKVD fan, Ivan Abaturov").
19. Websites with various tools for analysing Russian-language texts:
20. Interrelation of newsworthy events and publications and other media analysis.
Typical phishing email
Important terminology
Philology — studies the culture expressed in language and the literary creativity of peoples.
Linguistics — studies language itself as a functioning system.
Phonology — how the words sounds:
Phonetics — how we pronounce and hear words.
Linguistics — studies language itself as a functioning system.
Phonology — how the words sounds:
- the role (function) that phonemes fulfil in speech as components of more complex meaningful units, morphemes and words;
- phoneme is a sound represented by a symbol; the minimum meaning-distinguishing unit of a language;
- diphthongs are considered one phoneme — i in write — phonemes in English.
Phonetics — how we pronounce and hear words.
Grammar — studies parts of speech, verb uses (infinitive, gerund), sentence members, cases.
Spelling — making sure that speech is written in a consistent way.
Morphology — how a word is constructed; where the root is and where the ending is, etc.; how different parts of speech are formed.
- Morpheme — the meaningful part of a word: prefix, root, interfix (o, e), suffix, ending.
- Morphological parsing of the word.
- Stemming — shortening a word to its root: unfaithful → faith; instructor, destruction, structure → struct.
Lexicon
the vocabulary of a language, all the words in it
Word — a unit of language that is used to name something or serve some auxiliary purpose.
Lexeme — the "pure" form of a word; the way we see a word in a dictionary:
Word form — the chain of phonemes that forms a word:
Text form — words separated by spaces:
Lexeme — the "pure" form of a word; the way we see a word in a dictionary:
- found, finds, finding → find.
Word form — the chain of phonemes that forms a word:
- found, finds, finding → three word forms.
Text form — words separated by spaces:
- in front of → three text form.
Paradigm — a list united by a common feature.
Morphological paradigm — a list of all word forms of one lexeme that have different grammatical meanings.
Lemma — canonical, standardised form of a lexeme:
- Lemmatization — reduction to the dictionary form of a word.
- CST's online lemmatization tool (set ✅ tokenizer ✅ lemmatizer, show: only lemma).
Token — an identified token; punctuation marks and emojis are also identified as individual tokens.
What's important to remember about tokens and LLM
LLM performs tokenization, but if LLM isn't trained on texts from a specific language, it will incorrectly tokenize the text when parsing the prompt and generating the response text. Therefore, in DeepSeek, for example, you might see phrases like:
- «Просто允许йте себе наслаждаться этими моментами лёгкости»
- «Вы ничего не теряете, только gain-ите»
- «Теперь это аннотация, которая清晰地 отражает глубину и ценность вашего исследования»
In the Russian-speaking community, volunteer communities like Ruadaptnaya room and Vikhr models are responsible for adapting models to Russian. However, when retraining models to Russian, they almost inevitably lose some of their skills.
Lexical meaning
of a word
of a word
how to understand a word
Ηow the sound of a word relates to an object?
Homonyms — similar in spelling and sound but different in meaning — ball (sphere and dance).
Homophones — similar in sound but different in spelling and meaning — air and heir.
Homonyms — similar in spelling and sound but different in meaning — ball (sphere and dance).
Homophones — similar in sound but different in spelling and meaning — air and heir.
Homographs — similar in spelling but different in sound and meaning — present.
Homoforms — different words that coincide in some grammatical forms — found (establish and find).
Paronyms — similar in sound and morphemes, but different in meaning — affect and effect.
Several words together
Syntax — how to compose a sentence and how to combine words together correctly.
Semiotics — what signs and sign systems mean and how meaning is related to object:
Semantics — what words mean and how the meaning of words relates to an object; semiotics applied to language and a section of semiotics in general; broader than the lexical meaning of a word.
Semantic field — words with similar meaning:
Pragmatics — how the addressee interprets the signs seen; a section of semiotics.
Syntagma — a word or a group of words, an intonationally complete thought
Semiotics — what signs and sign systems mean and how meaning is related to object:
- what do the icons at the airport mean.
Semantics — what words mean and how the meaning of words relates to an object; semiotics applied to language and a section of semiotics in general; broader than the lexical meaning of a word.
Semantic field — words with similar meaning:
- blue, celestial, azure, navy, ultramarine etc.
Pragmatics — how the addressee interprets the signs seen; a section of semiotics.
Syntagma — a word or a group of words, an intonationally complete thought
- Τhere, / where used to be a lonely rock, / was now a pile of rubble - three syntagmas.
- Казнить ? нельзя ? помиловать (Execution Impossible Mercy)
Word combination — 2 or more words that are connected in meaning and grammar and describe one conceptual unit (subject, quality, action):
Idioms — a stable word combination, the meaning of which is not determined by the meaning of individual words in its composition:
Collocations, N-grams — word combinations connected by grammar and meaning, where the choice of words is conditioned by the combination of the key word:
Clichés — constructions that are frequently found in a certain type of texts
- essay writing;
- beautiful lizard.
Idioms — a stable word combination, the meaning of which is not determined by the meaning of individual words in its composition:
- a blessing in disguise;
- wrap your head around;
- be all ears.
Collocations, N-grams — word combinations connected by grammar and meaning, where the choice of words is conditioned by the combination of the key word:
- play a role — the key word "role";
- deep shock — the key word "shock".
Clichés — constructions that are frequently found in a certain type of texts
- "Associated Press reports citing a source in the N"
- 200 journalism cliches by the Washington Post.
Difficulties
The main difficulty in dealing with natural languages is the removal of ambiguity.
1. You work at an anti-virus provider company and your programme bans websites with a hashtag #foodporn or with fire-proof (огнеупорные) objects. Why?
2. What is the difficulty with the word «apple»?
3. «This dog resembled the royal dogs» — how many dogs are there?
4. What is the ambiguity of the phrases:
5. How to interpret neologisms?
6. How to tokenize
7. How to process a text with different writing systems?
1. You work at an anti-virus provider company and your programme bans websites with a hashtag #foodporn or with fire-proof (огнеупорные) objects. Why?
2. What is the difficulty with the word «apple»?
3. «This dog resembled the royal dogs» — how many dogs are there?
4. What is the ambiguity of the phrases:
- flying planes can be dangerous;
- John saw the man on the mountain with a telescope.
- Learning is difficult VS I am still learning;
- We gave flowers to the girls because they were beautiful?
5. How to interpret neologisms?
- slay queen, she ate and left no crumbs
6. How to tokenize
- what’re, I’m, isn’t,
- 我二十四岁 — «I am 24 years old»?
- SegmentAnt — tool that tokenizes Chinese and Japanese texts.
- More on working with Chinese characters: one, two, three.
7. How to process a text with different writing systems?
- 我的hero! — My hero!
- Τι κάνεις; — How are you?
8. How to split this text into sentences?
- "Oils for petrol and diesel engines with extended service intervals. Alternative - VW 505.01, VW 506.00, VW 506.01. Deliveries from 12.02.2021"
- st., ave., e.g., etc.
9. How to separate emotion from a simple parenthesis in Russian correspondence «idk( » from «idk (for now)»; «We were late( but saw a squirrel instead)» от «We were late (but saw a squirrel instead)»?
10. Mondegreens — the words you hear incorrectly:
11. Should one split "state-of-the-art" and «Rimsky-Korsakov» into separate words?
12. How to unify the spellings of the United States? U.S.A. USA US us
13. One preposition — but a structural and semantic difference:
- You may fly to Tula from Moscow.
- As opposed to Tula, you may fly from Moscow to Novosibirsk directly.
14. How to give the right search engine results for queries formulated with different words and styles?
- Stomach aches.
- My tummy hurts.
Project Inspiration
- Interactive map of endangered languages in Peru — Intensidad y Altura
- Love at first sight — HSD Hochschule Düsseldorf
- Strong men, caring women — Pew Research Center.
- Literary Constellations — Nicholas Rougeux.
- Why do cats and dogs...? — Nadieh Bremer + Google Trends.
- What Do People Ask Most About Relationships In China? — ThePaper.cn (используйте браузерный плагин Google Translate).
7. What is this place called? — Research on names of settlements on the map of Russia — Yandex Research.
8. The Poet's Journey — Michela Lazzaroni.
9. On the same page — KTH Royal Institute of Technology.
10. Not my name — Liuhuaying Yang
Sometimes one picture is enough
Word of the month / of the year
Infographics about Homer's Iliad
Projects by graduates of the Data Journalism Master's program from previous years
Analysis of publications on the Chinese microblogging service Weibo 微博
Zhang Leshi
Major Grom and the Plague Doctor in fanfiction
Alisa Rodina and Yulia Sukhanova
Alisa Rodina and Yulia Sukhanova
Robots and artificial bodies in Soviet science fiction
Veronika Zaruba
Endangered Languages of Russia
Alexey Divnenko for TASS
Alexey Divnenko for TASS
Chart selection
We select the type of chart based on the goal.
What do I want to show? Why would a picture be more appropriate than text in this case?
What do I want to show? Why would a picture be more appropriate than text in this case?
- All frequent words in the text → word cloud.
- Frequency of individual words in the text → bar chart or column chart.
- Positive and negative emotions → divergening bar chart.
- Text distribution → barcode.
- Associations and N-grams → graph.
- MI and t-score ratio → scatter plot.
- Collocations with a specific word and detailed annotation of the association strength → diagram.
- Differences between word scores in two corpora and rank dynamics → slope.
- Predominance of one or two words in the collection → beeswarm.
- Assess emotions captured by sentiment libraries across all five categories → radar.
Senseless but grammatical
Grammatically correct nonsense
Natural language, unlike artificial language, can be understood even if the rules are broken. Conversely, following the rules does not guarantee meaningfulness.
Noam Chomsky:
Lev Shcherba:
The glocky kuzdra shteckly budled the bocker and is kurdyaking the bockerling.
Noam Chomsky:
- Furiously sleep ideas green colorless.
- Colorless green ideas sleep furiously.
Lev Shcherba:
The glocky kuzdra shteckly budled the bocker and is kurdyaking the bockerling.
Lewis Carroll:
'Beware the Jabberwock, my son!
The jaws that bite, the claws that catch!
Beware the Jubjub bird, and shun
The frumious Bandersnatch!'
