Computational Linguistics
Department at Trier University

The Chair of Computational Linguistics at Trier University has been held since 2019 by Prof Dr Achim Rettinger, who heads the Computational Linguistics team. Computational Linguistics at Trier University focuses on the machine learning of knowledge representation, primarily from textual media. These are used to understand natural language, to analyse media and to integrate and provide knowledge.

The page is currently in development.
The following content is preliminary and will change.


Projects

  • Exploring the Interplay between Emotions and Distortion in Informal Science Discourse logo

    Exploring the Interplay between Emotions and Distortion in Informal Science Discourse

    Funding
    German Research Foundation (DFG) / French National Research Agency (ANR)
    Duration
    Mar 2026 – Feb 2029
    Contact
    Achim Rettinger

    More and more people learn about science informally, through social media, online forums and, increasingly, conversations with AI chatbots. In these settings, scientific findings are often simplified, exaggerated or distorted. Emotions play a key role: content that triggers fear, anger or excitement tends to spread further and can make distorted claims seem more convincing. How emotions and distorted science interact, and how AI systems affect this, is still poorly understood. EMO-SCI is a Franco-German project that studies these interactions in online science discourse. Partners from Heinrich Heine University Düsseldorf, Trier University, the i3S laboratory and the coordinating lab LIRMM are building new datasets, models and tools. These will be used to analyse how emotions drive the spread of scientific misinformation and disinformation, including in exchanges with AI chatbots. The goal is to better understand these effects and to provide tools that help counter scientific disinformation online.

  • TWin of Online Social Networks logo

    TWin of Online Social Networks

    Funding
    Horizon Europe Research and Innovation Actions
    Duration
    Apr 2023 – Mar 2026
    Contact
    Achim Rettinger

    It is a widely shared concern by experts that Online Social Networks (OSNs) have transformed public debate in harmful ways. Personalization algorithms can create filter bubbles and echo chambers which reinforces and polarizes the users’ opinions. Thus, OSNs have been blamed for spreading false information. To this point, tech-companies argue that there is no unequivocal empirical evidence and that they should not interfere with user behavior. The TWON project aims to provide scientific evidence on both counterarguments, whether social networks have undesirable effects on democratic debates and how they inferfere with user behavior. This will be done by creating digital twins of online social networks to estimate the effects of a change in the system.


Team

  • Portrait of Achim Rettinger

    Achim Rettinger

    Chair of Computational Linguistics

    Achim Rettinger is a full professor at Trier University, a director at FZI Forschungszentrum Informatik and supervising director at HoP House of Participation. He leads the krAil research group. Prof. Rettinger’s research spans machine learning, natural language understanding, knowledge representation, and the Web of Data. His group’s work has received multiple best paper, demo, challenge, and presentation awards. He has served as a senior Program Committee member of the International Semantic Web Conference (ISWC) and as a track chair and steering committee member of the European Semantic Web Conference (ESWC). He is also a member of the Board of Directors of the AI for Good Foundation.

  • Portrait of Christoph Hau

    Christoph Hau

    Research Fellow

    Christoph Hau is a Research Fellow at the Department of Computational Linguistics at Trier University, where he studies narrative construction across languages and cultures. His Ph.D. research in the BMBF-funded kuKI project examines Russian- and Turkish-language content and asks how language models can be adapted to recognize culturally specific narratives instead of defaulting to an English-centric view. He builds LLM-based annotation pipelines and classifiers, and probes the internal activations of language models to test whether they form stable, human-aligned concept representations. He also teaches in the Master’s program in Natural Language Processing and supervises student research.

  • Portrait of Nils Schwager

    Nils Schwager

    Research Fellow

    Nils Schwager is a Research Fellow at the Department of Computational Linguistics at Trier University, where he brings together computational social science and natural language processing. After completing a B.A. in Media and Communication Studies and Political Science, he recognized the potential of computational methods to investigate questions of polarization in societies fragmented by digital mass media and pursued an M.Sc. in Natural Language Processing. Most of his research is in computational social science, much of it together with his colleagues at EVAS.

  • Portrait of Simon Münker

    Simon Münker

    Research Fellow

    Simon Münker is a computational linguist and Research Fellow at the Department of Computational Linguistics at Trier University, where he studies whether language models can stand in for human participants in research. His Ph.D. thesis, “A Critical Examination of Large Language Models as Human Simulacra”, tested this across psychometrics and social media simulation. He builds metrics that expose where language models diverge from human behavior, and the frameworks researchers need to use them responsibly. He teaches lectures and seminars in natural language processing and linguistics.


Publications

2026

  • Can we use automated approaches to measure the quality of online political discussion? How to (not) measure interactivity, diversity, rationality, and incivility in online comments to the news

    Sjoerd B. Stolwijk, Mark Boukes, Wang Ngai Yeung, Yufang Liao, Simon Münker, Anne C. Kroon, Damian Trilling

    Communication Methods and Measures
    Abstract

    This article explores the (in)ability of automated tools to measure the deliberative quality of online user comments along the standards set out by Habermas: interactivity, diversity, rationality, and (in)civility. Utilizing a stratified sample of manually coded comments (n = 3,862) responding to news videos on YouTube and Twitter, we examined the performance of rule-based measures (i.e. dictionaries), machine-learning classifiers (conventional and transformer-based) and measurements by generative AI (Llama 3.1, GPT-4o, GPT-4T). We present results for over 50 metrics side-by-side to judge the opportunity costs of choosing one method over another. The results revealed strong variation across different groups of models. Overall, our expectation that more modern methods (transformers and generative AI) outperform the older, simpler ones was confirmed. However, the absolute differences between these model groups strongly depended on the measured concept, and we observed strong variance in performance among models of the same group. We provide recommendations for future research that balance ease of use with the performance of automated measurements, along with important cautions to consider.

  • Challenging the Myth: A Research Arc on LLMs as Human Simulacra

    Simon Münker, Achim Rettinger, Damian Trilling

    The Big Picture @ ACL
    Abstract

    When Large Language Models (LLMs) combined with prompt-based approaches as human simulacra emerged, they promised revolutionary shortcuts. Models trained on vast internet corpora may replicate human behavior and communication through text-based alignment. The initial optimism of the NLP community positioned LLMs as universal human proxies capable of replacing participants in surveys, generating authentic social media content, and simulating diverse cultural perspectives. We systematically dismantle this “myth of universal generalization” and document a shift toward methodological rigor. Our research reveals fundamental limitations: LLMs exhibit inhuman response patterns in psychometric assessments and produce detectable synthetic content. We analyze the difference between superficial linguistic fluency and genuine human-like representation, and reframe the current paradigm from asking “can LLMs replace humans?” to “under what validated conditions might LLMs serve as useful research components in social sciences?” Our work shows how interconnected research efforts challenge foundational assumptions and establishes best practices for deploying LLMs as human simulacra.

  • Don't Trust Generative Agents to Mimic Communication on Social Networks Unless You Benchmarked their Empirical Realism

    Simon Münker, Nils Schwager, Achim Rettinger

    EACLTWON
    Abstract

    The ability of Large Language Models (LLMs) to mimic human behavior triggered a plethora of computational social science research, assuming that empirical studies of humans can be conducted with AI agents instead. Since there have been conflicting research findings on whether and when this hypothesis holds, there is a need to better understand the differences in their experimental designs. We focus on replicating the behavior of social network users with the use of LLMs for the analysis of communication on social networks. First, we provide a formal framework for the simulation of social networks, before focusing on the sub-task of imitating user communication. We empirically test different approaches to imitate user behavior on X in English and German. Our findings suggest that social simulations should be validated by their empirical realism measured in the setting in which the simulation components were fitted. With this paper, we argue for more rigor when applying generative-agent-based modeling for social simulation.

  • Evaluating Token Probabilities as Confidence Estimators in Human Behavior Simulations: A Case Study in Luxembourgish Next Like Prediction

    Nils Schwager, Simon Münker, Alistair Plum, Achim Rettinger

    W-NUT @ EMNLPEMO-SCI
    Abstract

    As large language models (LLMs) are increasingly used to simulate human behavior, their operational validity is typically assessed through accuracy alone, implicitly treating human behavior as deterministic. We argue that on noisy digital trace data, a model’s performance depends not only on predicting the correct outcome but on its calibration. We present a case study testing whether the token probabilities of open-weight LLMs can serve as confidence estimates in a Next Like Prediction task: predicting how a user of the Luxembourgish news platform RTL.lu rates a comment, conditioned on their prior interactions. Evaluating Ministral 3 (3B, 8B), we find that, with in-context learning only, the tested models are overconfident and are outperformed on accuracy and calibration by a simple statistical baseline. Accuracy and calibration do not move together: scaling from 3B to 8B improves accuracy while degrading calibration, and longer behavioral histories raise accuracy while confidence declines. Task-specific supervised fine-tuning acts as an effective calibration intervention, reducing Expected Calibration Error from 0.2 to 0.006 while improving accuracy from 0.536 to 0.758. We conclude that confidence estimation should be evaluated alongside accuracy to improve validity when using LLM-based agents.

  • Guardrail Vulnerabilities in Open-Source Language Models: Implications for Democratic Discourse and Marginalized Communities

    Simon Münker, Fabio Sartori

    HICSS
    Abstract

    The proliferation of open-source Large Language Models (LLMs) presents a complex technological phenomenon with significant societal implications. While these models democratize access to advanced Natural Language Processing (NLP) capabilities, they simultaneously amplify risks for marginalized communities who often bear the disproportionate burden of technological misuse. Our research examines systematic vulnerabilities in guardrail mechanisms across seven prominent open-source LLMs, revealing patterns of harmful content generation that threaten democratic discourse and social cohesion. Through empirical analysis using advanced NLP classification methods, we demonstrate that popular open-source models consistently generate content classified as hateful or offensive when subjected to adversarial prompting techniques. These findings directly contradict the safety assurances provided by model developers, particularly Meta AI’s stated commitment that their systems should present balanced perspectives on debated policy issues rather than singular viewpoints.

  • Next Reply Prediction X (NRP-X) Dataset: Linguistic Discrepancies in Naively Generated Content

    Simon Münker, Nils Schwager, Kai Kugler, Michael Heseltine, Achim Rettinger

    LLMs4SSH @ LRECTWON
    Abstract

    The increasing use of Large Language Models (LLMs) as proxies for human participants in social science research presents a promising, yet methodologically risky, paradigm shift. While LLMs offer scalability and cost-efficiency, their “naive” application, where they are prompted to generate content without explicit behavioral constraints, introduces significant linguistic discrepancies that challenge the validity of research findings. This paper addresses these limitations by introducing a novel, history-conditioned reply prediction task on authentic X (formerly Twitter) data, to create a dataset designed to evaluate the linguistic output of LLMs against human-generated content. We analyze these discrepancies using stylistic and content-based metrics, providing a quantitative framework for researchers to assess the quality and authenticity of synthetic data. Our findings highlight the need for more sophisticated prompting techniques and specialized datasets to ensure that LLM-generated content accurately reflects the complex linguistic patterns of human communication, thereby improving the validity of computational social science studies.

  • On the Impact of Item Order and Questionnaire Popularity for Language Model Construct Validity and Cross-Human Alignment

    Simon Münker, Simon Werner, Achim Rettinger

    Discovery Science
    Abstract

    Large language models (LLMs) are increasingly deployed as synthetic survey respondents in psychological research, yet the conditions under which their responses achieve psychometric validity remain poorly understood. We investigate two factors hypothesized to systematically modulate LLM response quality on established questionnaires: item pre-sentation order and questionnaire popularity as an operationalization of likely training data coverage. We analyze whether grouping question-naire items by their measured construct, rather than cycling through constructs sequentially, improves internal consistency and cross-human structural alignment, and whether this effect is amplified for instruments more likely to be well-represented in LLM training corpora. We evaluate six open-source instruction-tuned LLMs spanning three model fami-lies on two instruments of markedly contrasting public prominence: the widely-known Big Five Personality Traits (Big 5) and the lesser-known Humor Styles Questionnaire (HSQ). Our results show that grouped item presentation consistently improves both metrics across all models and both instruments and instrument popularity reinforces this effect. Our findings indicate that LLM performance is partly driven by format-triggered pattern recognition within training data rather than generalizable construct understanding.

  • The Unsampled Truth: Quantifying Prompt Artifacts in LM Psychometrics

    Nils Schwager, Christoph Hau, Simon Münker, Achim Rettinger

    Findings of EMNLPEMO-SCI
    Abstract

    When prompting language models for psychometric assessment, researchers assume that the responses reflect the injected persona and the meaning of the survey item. We test this premise using a diagnostic design that crosses five semantically distinct baseline personas with five semantically equivalent variants of each of four prompt components (persona wording, task instruction, item wording, option symbol). Measuring the 1-Wasserstein distance between the resulting response distributions and partitioning the variation among the five components allows for the separation of target effects from prompt artifacts. We apply the framework to 13 open-weight small language models (0.6B–14B) on the Big Five Inventory and the Short Dark Triad. We find that in most models, the task instruction and option symbol displace response distributions further than paraphrasing the persona description or the item itself. For a substantial share of items, the artifact share of explained variation exceeds 50%; non-semantic changes of the prompt account for more response variation than the baseline personas. Our framework lets researchers quantify these prompt artifacts before interpreting psychometric output.

  • Towards Simulating Social Media Users with LLMs: Evaluating the Operational Validity of Conditioned Comment Prediction

    Nils Schwager, Simon Münker, Alistair Plum, Achim Rettinger

    WASSA @ EACLTWON
    Abstract

    The transition of Large Language Models (LLMs) from exploratory tools to active “silicon subjects” in social science lacks extensive validation of operational validity. This study introduces Conditioned Comment Prediction (CCP), a task in which a model predicts how a user would comment on a given stimulus by comparing generated outputs with authentic digital traces. This framework enables a rigorous evaluation of current LLM capabilities with respect to the simulation of social media user behavior. We evaluated open-weight 8B models (Llama-3.1, Qwen3, Ministral) in English, German, and Luxembourgish language scenarios. By systematically comparing prompting strategies (explicit vs. implicit) and the impact of Supervised Fine-Tuning (SFT), we identify a critical form vs. content decoupling in low-resource settings: while SFT aligns the surface structure of the text output (length and syntax), it degrades semantic grounding. Furthermore, we demonstrate that explicit conditioning (generated biographies) becomes redundant under fine-tuning, as models successfully perform latent inference directly from behavioral histories. Our findings challenge current “naive prompting” paradigms and offer operational guidelines prioritizing authentic behavioral traces over descriptive personas for high-fidelity simulation.

2025

  • Cultural Bias in Large Language Models: Evaluating AI Agents through Moral Questionnaires

    Simon Münker

    Moral and Legal AI Alignment Symposium @ IACAP/AISB
    Abstract

    Are AI systems truly representing human values, or merely averaging across them? Our study suggests a concerning reality: Large Language Models (LLMs) fail to represent diverse cultural moral frameworks despite their linguistic capabilities. We expose significant gaps between AI-generated and human moral intuitions by applying the Moral Foundations Questionnaire across 19 cultural contexts. Comparing multiple state-of-the-art LLMs’ origins against human baseline data, we find these models systematically homogenize moral diversity. Surprisingly, increased model size doesn’t consistently improve cultural representation fidelity. Our findings challenge the growing use of LLMs as synthetic populations in social science research and highlight a fundamental limitation in current AI alignment approaches. Without data-driven alignment beyond prompting, these systems cannot capture the nuanced, culturally-specific moral intuitions. Our results call for more grounded alignment objectives and evaluation metrics to ensure AI systems represent diverse human values rather than flattening the moral landscape.

  • Fingerprinting LLMs through Survey Item Factor Correlation: A Case Study on Humor Style Questionnaire

    Simon Münker

    EMNLP
    Abstract

    LLMs increasingly engage with psychological instruments, yet how they represent constructs internally remains poorly understood. We introduce a novel approach to “fingerprinting” LLMs through their factor correlation patterns on standardized psychological assessments to deepen the understanding of LLMs constructs representation. Using the Humor Style Questionnaire as a case study, we analyze how six LLMs represent and correlate humor-related constructs to survey participants. Our results show that they exhibit little similarity to human response patterns. In contrast, participants’ subsamples demonstrate remarkably high internal consistency. Exploratory graph analysis further confirms that no LLM successfully recovers the four constructs of the Humor Style Questionnaire. These findings suggest that despite advances in natural language capabilities, current LLMs represent psychological constructs in fundamentally different ways than humans, questioning the validity of application as human simulacra.

  • Political Bias in LLMs: Unaligned Moral Values in Agent-centric Simulations

    Simon Münker

    JLCL
    Abstract

    Contemporary research in social sciences increasingly utilizes state-of-the-art generative language models to annotate or generate content. While these models achieve benchmarkleading performance on common language tasks, their application to novel out-of domain tasks remains insufficiently explored. To address this gap, we investigate how personalized language models align with human responses on the Moral Foundation Theory Questionnaire. We adapt open-source generative language models to different political personas and repeatedly survey these models to generate synthetic data sets where model-persona combinations define our sub-populations. Our analysis reveals that models produce inconsistent results across multiple repetitions, yielding high response variance. Furthermore, the alignment between synthetic data and corresponding human data from psychological studies shows a weak correlation, with conservative persona-prompted models particularly failing to align with actual conservative populations. These results suggest that language models struggle to coherently represent ideologies through in-context prompting due to their alignment process. Thus, using language models to simulate social interactions requires measurable improvements in in-context optimization or parameter manipulation to align with psychological and sociological stereotypes properly.

  • twony: A Micro-Simulation of the Impact of OSN Mechanics on the Emotionality of Online Discourse

    Simon Münker, Achim Rettinger

    SemGenAge @ ESWCTWON
    Abstract

    We introduce twony, a micro-simulation prototype designed to show the impact of online social network (OSN) mechanics—particularly recommendation algorithms—on emotional contagion and discourse dynamics. By harnessing the capabilities of Large Language Models (LLMs), the system generates an ecosystem of synthetic, politically engaged digital personas that interact within a controlled social media environment. These autonomous agents emulate human behaviors through in-context prompting techniques, enabling users to observe emotional transmission patterns under systematically varied conditions while circumventing the constraints inherent to real-user experimentation. The prototype implements two distinct recommendation paradigms: a baseline chronological feed and an emotion-prioritizing ranking mechanism that amplifies content based on emotional intensity, allowing the examination of the formation and reinforcement of echo chambers. Emotional valence is quantified via a fine-tuned BERT model, while network-level and agent-level visualizations track emotional cascades and polarization dynamics throughout the lifecycle. The system is architected as a browser-based application leveraging modern web technologies and decentralized APIs. Twony emphasizes accessibility, customizability, and extensibility. This contribution advances the field by providing a scalable, open-source prototype for systematically investigating OSN dynamics, offering actionable insights for platform designers and policymakers seeking to mitigate harmful emotional contagion while fostering healthier online discourse environments.

  • Zero-shot prompt-based classification: topic labeling in times of foundation models in German Tweets

    Simon Münker, Achim Rettinger, Damian Trilling

    SRW @ ACL
    Abstract

    Filtering and annotating textual data are routine tasks in many areas, like social media or news analytics. Automating these tasks allows to scale the analyses wrt. speed and breadth of content covered and decreases the manual effort required. Due to technical advancements in Natural Language Processing, specifically the success of large foundation models, a new tool for automating such annotation processes by using a text-to-text interface given written guidelines without providing training samples has become available. In this work, we assess these advancements in-the-wild by empirically testing them in an annotation task on German Twitter data about social and political European crises. We compare the prompt-based results with our human annotation and preceding classification approaches, including Naive Bayes and a BERT-based fine-tuning/domain adaptation pipeline. Our results show that the prompt-based approach – despite being limited by local computation resources during the model selection – is comparable with the fine-tuned BERT but without any annotated training data. Our findings emphasize the ongoing paradigm shift in the NLP landscape, i.e., the unification of downstream tasks and elimination of the need for pre-labeled training data.

2024

  • InvBERT: Reconstructing Text from Contextualized Word Embeddings by inverting the BERT pipeline

    Kai Kugler, Simon Münker, Johannes Höhmann, Achim Rettinger

    JCLS
    Abstract

    Digital Humanities and Computational Literary Studies apply automated methods to enable research on large corpora which are not feasible by manual inspection alone. However, due to copyright restrictions, the availability of relevant digitized literary works is limited. Derived Text Formats (DTFs) have been proposed as a solution. Here, textual materials are transformed in such a way that copyright-critical features are removed, but that the use of certain analytical methods remains possible. Contextualized word embeddings produced by transformer-encoders are promising candidates for DTFs because they allow for state-of-the-art performance on analytical tasks. However, in this paper we demonstrate that under certain conditions the reconstruction of the original text from token representations becomes feasible. Our attempts to invert BERT suggest that publishing the encoder together with the contextualized embeddings is unsafe, since it allows to generate data to train a decoder with a reconstruction accuracy sufficient to violate copyright laws.