Alexander Panchenko

Below is a list of some selected research highlights in grouped by fields of interest. The full list of references can be found on the publication tab.

Lexical Semantics

Idea:	To induce sense inventories for 158 languages and design a word sense disambiguation (WSD) algorithm for them using only word embeddings with no training data. These way we enable WSD for low-resourced languages.
Paper:	Word Sense Disambiguation for 158 Languages using Word Embeddings Only (LREC-2020)
Illustration:
Ideas:	(1) To perform lexical substitution in context, integration of information about the target work is important; (2) The majority of the lexical substitutes are co-hyponyms and synonyms, but the distribution varies across parts of speeches and models.
Paper:	Always Keep your Target in Mind: Studying Semantics and Improving Performance of Neural Lexical Substitution (COLING-2020)
Illustration:
Idea:	Induce from text interpretable representations of word senses fitted with images, hypernyms, definitions, and so on in a completely unsupervised fashion and design a system for word sense disambiguation on the basis of this sense inventory.
Reference:	Unsupervised, Knowledge-Free, and Interpretable Word Sense Disambiguation (EMNLP-2017)
Illustration:
Idea:	Similarly to a social ego-network, where an individual has several circles of close groups that do not overlap, related words cluster in a similar way forming word senses. An ego-network based graph clustering can be used to automatically identify word senses based on any pre-trained word embedding model.
Paper:	Making Sense of Word Embeddings (ACL-2016)
Illustration:

Argument Mining

Idea:	It is possible to mine the Web for comparative statements to help to answer comparative questions (like “Is python better than Matlab for deep learning?”) and design a system that would fulfill the information needs of the users more efficiently than the usual web search.
Paper:	Answering Comparative Questions: Better than Ten-Blue-Links? (CHIIR-2019)
Illustration:
Idea:	Argument mining can be cast as a tagging task, we make available the recent neural models readily available for the integration into various NLP pipelines as well as for interactive analysis of user texts.
Paper:	TARGER: Neural Argument Mining at Your Fingertips (ACL-2019)
Illustration:

List of the Main Research Interests

A more complete list of research interests is listed below:

• Lexical semantics (especially word sense induction and disambiguation, frame induction and disambiguation, semantic similarity and relatedness, sense embeddings, automated construction and completion of lexical resources such as WordNet and FrameNet)
• Argument mining (especially comparative argument mining, and argument retrieval)
• Learning representations of linguistic symbolic structures (graphs) such as knowledge bases and lexical resource
• NLP for a better society: recognition of fake news, hate speech, and related phenomena
• Textual style transfer

• Chekalina, V., Panchenko (2021): Retrieving Comparative Arguments using Ensemble Methods and Neural Information Retrieval. Proceedings of the Working Notes of CLEF 2021 – Conference and Labs of the Evaluation Forum. Bucharest, Romania.
• Dementieva, D., Ustyantsev, S., Dale, D., Kozlova, O., Semenov, N., Panchenko, A., and Logacheva, V. (2021): Crowdsourcing of Parallel Corpora: the Case of Style Transfer for Detoxification. Proceedings of the 2nd Crowd Science Workshop: Trust, Ethics, and Excellence in Crowdsourced Data Management at Scale co-located with 47th International Conference on Very Large Data Bases (VLDB 2021) . Copenhagen, Denmark
• Razzhigaev, A., Arefyev, N., Panchenko, A. (2021): SkoltechNLP at SemEval-2021 Task 2: Generating Cross-Lingual Training Data for the Word-in-Context Task. In Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021). Association for Computational Linguistics. Bangkok, Thailand (Online)
• Dale, D., Markov, I., Logacheva, V., Kozlova, O., Semenov, N., Panchenko, A. (2021): SkoltechNLP at SemEval-2021 Task 5: Leveraging Sentence-level Pre-training for Toxic Span Detection. In Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021). Association for Computational Linguistics. Bangkok, Thailand (Online)
• Dementieva, D., Moskovskiy, D., Logacheva, V., Dale, D., Kozlova, O., Semenov, N., and Panchenko, A. (2021): Methods for Detoxification of Texts for the Russian Language. In Proceedings of the 1st Workshop on NLP for Positive Impact (non-archival). The Association for Computational Linguistics and The Asian Federation of Natural Language Processing. Bangkok, Thailand (online).
• Babakov N., Logacheva, V., Kozlova, O., Semenov, N., Panchenko, A. (2021): Detecting Inappropriate Messages on Sensitive Topics that Could Harm a Company’s Reputation. In Proceeding of the 8th Workshop on Balto-Slavic Natural Language Processing (BSNLP 2021). The 2021 Conference of the European Chapter of the Association for Computational Linguistics. Kyiv, Ukraine (Online)
• Bondarenko, A., Fröbe, M., Beloucif, M., Gienapp, L., Ajjour, Y., Panchenko, A., Biemann, C., Stein, B., Wachsmuth, H., Potthast, M. Hagen, M. (2020): Overview of Touché 2020: Argument Retrieval. Working Notes of CLEF 2020 – Conference and Labs of the Evaluation Forum. Thessaloniki, Greece.
• Chekalina, V. and Panchenko, A. (2020): Retrieving Comparative Arguments using Deep Pre-trained Language Models and NLU. Working Notes of CLEF 2020 – Conference and Labs of the Evaluation Forum. Thessaloniki, Greece.
• Bondarenko, A., Hagen, M., Beloucif, M., Biemann, C., Panchenko, A. (2020): Answering Comparative Questions withWeb-based Arguments. Proceedings of AAAI 2020 Workshop on Interactive and Conversational Recommendation Systems, New York, USA
• Panchenko, A., Bondarenko, A., Franzek, M., Hagen, M., Biemann, C. (2019): Categorizing Comparative Sentences. Proceedings of the 6th Workshop on Argument Mining, Florence, Italy.
• Puzyrev, D., Shelmanov, A., Panchenko, A. and Artemova, E., (2019): A Dataset for Noun Compositionality Detection for a Slavic Language. In Proceedings of the 7th Workshop on Balto-Slavic Natural Language Processing (pp. 56-62).
• Arefyev, N., Sheludko, B., Davletov, A., Kharchev, D., Nevidomsky, A., Panchenko, A. (2019): Neural GRANNy at SemEval 2019 Task 2: A combined approach for better modeling of semantic relationships in semantic frame induction. Proceedings of SemEval 2019, Minneapolis, MN, USA [Winner of Task 2 and selected for best of SemEval session]
• Anwar, S., Ustalov, D., Arefyev, N., Ponzetto, S.P., Biemann C. and Panchenko, A. (2019): HHMM at SemEval-2019 Task 2: Unsupervised Frame Induction using Contextualized Word Embeddings. Proceedings of SemEval 2019, Minneapolis, MN, USA
• Panchenko A., Faralli S., Ponzetto S. P., and Biemann C. (2017): Using Linked Disambiguated Distributional Networks for Word Sense Disambiguation. In Proceedings of the Workshop on Sense, Concept and Entity Representations and their Applications (SENSE) co-located with the 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL’2017). Valencia, Spain. Association for Computational Linguistics
• Pelevina M., Arefyev N., Biemann C., Panchenko A. (2016): Making Sense of Word Embeddings. In Proceedings of the 1st Workshop on Representation Learning for NLP co-located with the ACL conference. Berlin, Germany. Association for Computational Linguistics [Best paper award at the first workshop on representation learning for NLP]
• Ballweg K., Zouhar F., Wilhelmi-Dworski P., von Landesberger T., Fahrer U., Panchenko A., Yimam S.M. Biemann C., Regneri M., Ulrich H. (2016): new/s/leak – A Tool for Visual Exploration of Large Text Document Collections in the Journalistic Domain. Workshop on Visualisation in Practice co-located with the VIS conference, Baltimore, MD, USA
• Panchenko A., Faralli S., Ruppert E., Remus S., Naets H., Fairon C. Ponzetto S. P., and Biemann, C. (2016): TAXI at SemEval-2016 Task 13: a Taxonomy Induction Method based on Lexico-Syntactic Patterns, Substrings and Focused Crawling. In Proceedings of the 10th International Workshop on Semantic Evaluation. San Diego, CA, USA. Association for Computational Linguistics [Winner of SemEval-2016 Task 13 shared task]
• Kivimaki I., Panchenko A., Dessy A. Verdegem D., Francq P., Fairon C., Bersini H., Saerens M. (2013): A Graph-Based Approach to Skill Extraction from Text. In Proceedings of TextGraphs-8 Workshop co-located with the Conference on Empirical Methods for Natural Language Processing (EMNLP’2013). Seattle, USA. Association for Computational Linguistics
• Panchenko A. Morozova O. (2012): A Study of Hybrid Similarity Measures for Semantic Relation Extraction. In Proceedings of Workshop of Innovative Hybrid Approaches to the Processing of Textual Data Workshop co-located with the EACL’2012 conference, pp.10-18, Avignon, France. Association for Computational Linguistics
• Panchenko A, Beaufort R., Fairon C. (2012): Detection of Child Sexual Abuse Media on P2P Networks: Normalization and Classification of Associated Filenames. In Proceedings of Workshop on Language Resources for Public Security Applications of the 8th International Conference on Language Resources and Evaluation (LREC). European Language Resources Association (ELRA)
• Panchenko A., Adeykin S., Romanov P., Romanov A. (2012): Extraction of Semantic Relations between Concepts with KNN Algorithms on Wikipedia. In Proceedings of Concept Discovery in Unstructured Data (CDUD) workshop co-located with the International Conference On Formal Concept Analysis (ICFCA’2012), pp.78-88, Leuven, Belgium. CEUR-WS
• Panchenko A. (2012): Towards an Efficient Combination of Similarity Measures for Semantic Relation Extraction. Abstract in Proceedings of the 22nd Meeting of Computational Linguistics in the Netherlands (CLIN 22). Tilburg, The Netherlands. Tilgurg University
• Panchenko A. (2011): Comparison of the Knowledge-, Corpus-, and Web-based Similarity Measures for Semantic Relations Extraction. In Proceedings of the Workshop GEometrical Models of Natural Language Semantics (GEMS) co-located with the EMNLP’2011 conference. pp.10-18. Edinburgh, Scotland. Association for Computational Linguistics
• Panchenko A. (2010): Can We Automatically Reproduce Semantic Relations of an Information Retrieval Thesaurus?. In Proceedings of the Young Scientists Conference of the 4th Russian Summer School in Information Retrieval (RuSSIR’2010). pp.36–51. Voronezh, Russia. VSU

• Best paper award at the first workshop on representation learning for NLP (RepL4NLP) at the ACL 2016  conference in Berlin, Germany for the paper “Makings sense of word embeddings“, where an approach for inducing sense embeddings was presented.
• Best paper award of Fraunhofer IGD and Visual Computing Groups of TU Darmstadt (Darmstadt, Germany) in the category “Impact on the society” for the paper “new/s/leak – Information Extraction and Visualization for Investigative Data Journalists”. The paper presents an NLP system for investigative data journalism which was used by Der Spiegel journal.
• Best of SemEval-2019: our paper on unsupervised semantic frame induction using BERT was selected as one of the best submissions (with the possibility to present our work orally) at the 13th International Workshop on Semantic Evaluation (SemEval-2019) in  Minneapolis, USA.
• Best presentation according to attendees of the AIST’2014 conference on gender detection in social media (slideshare).

• Two papers are accepted to the special issue of Semantic Web Journal (Q1) on Deep Learning for Knowledge Graphs (DL4KG). One is on taxonomy enrichment and the second one is a survey on neural entity linking.
• An invited talk at YaC/E: Yet Another Conference on Education in Yandex. Participation in a discussion on teaching data science courses and sharing my experience with teaching NLP course at Skoltech. Video.
• An invited talk at AI Technology In Search And Recommendation Workshop of Huawei on neural entity linking.
• An invited talk at AI Journey conference by Sberbank.
• A paper on text detoxification has been accepted to EMNLP-2021 on a collaborative project with MTS AI. See also a post at MTS blog about this line of research (in Russian).
• A lab at CLEF-2022 is accepted: Conference and Labs of the Evaluation Forum Information Access Evaluation meets Multilinguality, Multimodality, and Visualization. Together with German colleagues I will be involved in the organisation of a shared task on argument retrieval.
• I chaired the NLP session at the Data Fusion conference. The video is available online.
• I received a DFG Merkator Fellowship (Visiting Professorships at German Universities) to perform a visit to Germany to work with researchers from Universities of Hamburg and Halle on comparative argument mining.
• An invited talk at the colloquium of Computer Science faculty of HSE on neural entity linking. The video is available online.
• Three papers has been accepted to the 16th conference of the European Chapter of the Association for Computational Linguistics (EACL) on active learning with pre-trained language models for sequence tagging, uncertainty estimation for transformer NLP models, and a demo paper on comparative question answering.
• A paper has been accepted to the 8th workshop on Balto-Slavic Natural Language Processing (BSNLP) co-located with the EACL-2021 conference on detection of inappropriate messages on sensitive topics that could harm a company’s reputation.
• An overview paper related to the CLEF laboratory on argument retrieval has been accepted to the 43rd European Conference on Information Retrieval (ECIR-2021).
• A paper has been accepted to the the 11th International Global Wordnet Conference (GWC2021) on graph-based representations for taxonomy enrichment (in collaboration with researchers from Moscow State University). Poster and video.
• Two papers have been accepted to the 28th International Conference on Computational Linguistics (COLING-2020) on neural lexical substitution (in collaboration with researchers from Samsung) and taxonomy enrichment.
• I am a co-organizer of the 9th international conference on Analysis of Images, Social Networks and Texts (AIST-2020).
• I am involved in the organization of the 14th workshop on graph-based natural language processing (TextGraphs-14) co-located with the 28th International Conference on Computational Linguistics (COLING’2020) in Barcelona, Spain.
• I co-organize a shared task on taxonomy enrichment at the Dialogue Evaluation campaign.
• A paper is accepted at the ECIR’2020 conference on Touché: The First Shared Task on Argument Retrieval.
• A paper accepted at the Web Search and Data Mining (WSDM’2020) conference on “Comparative Web Search Questions” in collaboration with several German researchers.
• I am involved in the organization of a CLEF 2020 lab on Argument Mining: consider participation in the shared tasks!
• Presentation at the panel discussion at the BUCC workshop at the RANLP’2019 conference.
• An invited talk at the AIST’2019 conference on several papers published in ACL and the associated workshops (conference web site).
• A paper is accepted at the international conference on Recent Advances in Natural Language Processing (RANLP’2019) in Varna, Bulgaria entitled “Combining Lexical Substitutes in Neural Word Sense Induction“.
• A paper is accepted at the 8th International Conference – Analysis of Images, Social networks and Texts (AIST’2019) in the main volume Springer LNCS on using distributional semantics for detecting noun compositionality.
• An invited talk at the Deep Pavlov lab at Moscow Institute of Physics and Technology.
• Seven papers accepted at ACL 2019 and associated workshops: three papers accepted to the main conference, one demo paper, a paper at the student research workshop (main conference) and two papers at regular workshops. This presentation provides summaries of all these papers and contains references to the pre-prints and codes.
• Two papers accepted at SemEval-2019 on unsupervised semantic frame induction in collaboration with Universities of Hamburg and Mannheim and in collaboration with Samsung research. In these two papers, ELMo and BERT contextualized word embedding models were probed in the task of semantic frame induction.
• A paper is accepted at the 8th Joint Conference on Lexical and Computational Semantics (*SEM) in Minneapolis, USA on learning graph embeddings based on graph similarity metrics.
• I co-organize a special issue in the Natural Language Engineering journal on “Informing Neural Architectures for NLP with Linguistic and Background Knowledge” together with Simone Paolo Ponzetto and Ivan Vulić.
• A keynote talk at the 24th International Conference on Computational Linguistics and Intellectual Technologies (Dialogue’2018): From unsupervised induction of linguistic structures to applications in deep learning.
• The best paper award in the category “Impact on Society” of Fraunhofer IGD and the Visual Computing Groups of TU Darmstadt for the paper “new/s/leak – Information Extraction and Visualization for Investigative Data Journalists“.
• I co-organize the 7th Conference on Analysis of Images, Social Networks, and Texts (AIST’2018). The selected papers will be published in the Springer LNCS series.
• An invited talk at the Global WordNet Conference (GWC’2018) in Singapore on inducing interpretable word senses for word sense disambiguation and enrichment of lexical resources.
• An article is accepted in the Natural Language Engineering article on graph-based distributional semantics.
• I co-organize a shared task on word sense induction for the Russian language. 18 teams participated in the task submitting 383 models. An overview of the results is available in this preprint.
• The release of a web-scale dependency-parsed corpus of English texts, based on the CommonCrawlweb crawls. The corpus features over 250 billion of tokens and is available at Amazon S3.
• The release of a web demo of the unsupervised, knowledge-free, and interpretable system for word sense disambiguation presented at EMNLP 2017 in Copenhagen.

Supervision of PhD Theses

• Özge Sevgili Ergüven (10.2018-…) is co-supervised with Chris Biemann (Germany). Özge is supported by DAAD (Deutscher Akademischer Austauschdienst) and is based at the University of Hamburg. Her PhD research is related to neural entity linking and representation learning on graphs.
• Saba Anwar (10.2018-…) is co-supervised with Chris Biemann (Germany). Saba is supported by DAAD (Deutscher Akademischer Austauschdiens) and the Higher Education Commission of Pakistan. Her PhD research is related to using of neural language models for unsupervised frame induction.
• Irina Nikishina (11.2019-…). Research of Irina is related to the development of methods for the automatic construction of lexical resources and the application of multilinear algebra to NLP.
• Daryna Dementieva (11.2019-…) Research of Daryna is related to the detection of fake news and textual style transfer.
• Victoriia Chekalina (11.2019-…) Research of Victoriia is related to comparative argument mining and multilinear algebra for NLP.
• Anton Razzhigaev (11.2020-…) Research of Anton is related to vectorization of knowledge graphs and question answering (KBQA).

Supervision of Master and Bachelor Theses

I supervised research-oriented Master theses, usually also aiming to publish a conference paper on the basis of the produced materials.

• Dmitry Puzyrev (2021, Skoltech).Policy-based strategies for active learning in a scalable setup. Co-supervised with Artem Shelmanov.
• Anton Voronov (2021, Skoltech-MIPT): Automatic Dialogue Censor – Style Transfer for Texts. Now with Sberbank AIR institute.
• Lyubov Kupriyanova  (2021, Skoltech): Uncertainty Estimation for Active Learning and Misclassification detection in NLP. Co-supervised with Artem Shelmanov.
• Vitaly Protasov (2021, Skoltech-MIPT): Cross-lingual lexical substitution. Now with Sberbank AIR institute.
• Denis Teslenko (2020, Ural Federal University): Multilingual Graph-based Word Sense Disambiguation with Word Embedding Only.
• Heike Heller (2019, University of Hamburg): Comparative Query Suggestion. Co-supervised with Chris Biemann.
• Dmitry Puzyrev (2019, Higher School of Economics, CS faculty): Supervised Approaches to Detection of Noun Compositionality. Co-supervised with Artem Shelmanov and Ekaterina Artemova.
• Matthias Schildwächter (2019, University of Hamburg): An Open-Domain System for Retrieval and Visualization of Comparative Arguments from Text.
• Alvin Rindra Fazrie (2019, University of Hamburg): Visual Information Management with Compound Graphs. Main supervisor: Steffen Remus.
• Dahmash Ibrahim (2018, University of Hamburg): Question Answering using Dynamic Neural Networks. Main supervisor: Benjamin Milde.
• Mirco Franzek (2018, University of Hamburg). Comparative Argument Mining. Co-supervised with Chris Biemann.
• Marten Fide (2017, TU Darmstadt). Predicting hypernyms in contexts with JoBimText. Co-supervised with Chris Biemann. Now at TU Darmstadt.
• Maria Pelevina (2016, TU Darmstadt). Unsupervised Word Sense Disambiguation with Sense Embeddings. Co-supervised with Chris Biemann. Now at Deutsche Bahn R&D.
• Simon Dif (2015, TU Darmstadt). Statistical Models of Semantics with Structured Topics. Co-supervised with Chris Biemann. A dual degree Masters program with ENSIMAG, Grenoble, France. Now at Altran R&D.
• Alexey Romanov (2012, Moscow State Technical University). Graph Algorithms in the Lexical Semantic Search Engine ‘Serelex’. Co-supervised with Andrew Philippovich. Now at the University of Massachusets Lowell.

Internships & Visiting Researchers

I help to write research proposals to funding organizations which let researchers visit our faculty and do interesting short-term research project together.

• Dmitry Puzyrev (2019): Using hyperbolic word embeddings for detection of noun compositionality. Partially funded by the University of Hamburg. Visit outcome: ACL publication.
• Shantanu Acharya (2018): Taxonomy induction using word sense representations. Funded by DAAD. Visit outcome: ACL publication.
• Andrey Kutuzov (2018): Learning graph embeddings via node similarities. Funded by the University of Oslo. Visit outcome: ACL and *SEM publications.
• Artem Chernodub (2017): Recurrent Neural Networks for Argument Mining. Funded by DAAD. Visit outcome: ACL demo paper.
• Dmitry Ustalov (2016): Graph Clustering for Word Sense Induction. Funded by DAAD. Visit outcome: ACL and EACL publications.

• Statistical Natural Language Processing (Spring, 2019). A course for Master students. Slides. The course is based on the classic textbook of Jurafsky & Martin and represents a set of topics on (mostly) pre-neural NLP.
• Neural Natural Language Processing (Winter, 2019). A course for Master students. Slides. This course is focusing mostly on neural NLP models.
• A guest lecture at the Deep Learning for Natural Language Processing course at Indian Institute of Technology (IIT) Patna, India on “Word and document embeddings” (Winter, 2020).

Organization of Events