MUWS 2026 - The 5th International Workshop on Multimodal Human Understanding for the Web and Social Media

Chiao-I Tseng is an Associate Professor in Communication in the Department of Applied Information Technology at the University of Gothenburg. Her work integrates large-scale multimodal discourse analysis with computational and digital methods, as well as cognitive approaches such as eye-tracking and comprehension studies. Using these tools, she examines how film and video structures shape and influence viewers’ narrative interpretations.

Abstract: TikTok has become a primary news environment for digital audiences, and the rise of TikTok journalism raises pressing questions. How do mainstream news channels adapt to social media logics in order to stay relevant to younger audiences? How are alternative media and influencers reshaping the news landscape? And do platform-native actors rely on different narrative strategies to establish and maintain credibility? Addressing these questions empirically requires a framework for systematically analysing and comparing the news narrative approaches adopted across very different news actors, such as public service media, populist outlets, market-driven news institutions, and influencers who command young audiences’ trust through personal authenticity and charisma. Current scholarship on social media news remains largely text-centred, and the studies that do attend to more complex narrative strategies typically rest on close readings of a handful of video clips. This talk introduces a framework for studying multimodal news narrative strategies on TikTok at scale. We present a novel method that combines qualitative multimodal discourse analysis with large-scale computational analysis of news videos, together with a multimodal annotation scheme that supports systematic annotation across formal, semantic and narrative patterns. This combination allows us to move from illustrative examples to empirically robust comparative analysis, and to address the transformation and logics of TikTok journalism across the broad social media news landscape rather than within a single outlet. The work presented in this talk is carried out within TACTik: Transformation and Consequences of News on TikTok, a research project funded by the Swedish Research Council.

• 11:00 – 11:20: The Geometry of World-Event News: Cross-Day Drift and Within-Day Dispersion by Weijian Fan et al.
• 11:20 – 11:40: Who Judges the Frame? Auditing Multimodal LLM Judges for News Framing Across Event-Level Perspectives by Antonela Tommasel & Markus Schedl
• 11:40 – 12:00: When Model Outputs Become Social Evidence: Challenges for LLM-Mediated Web and Social Media Analysis by Antonela Tommasel & Markus Schedl

Dr. Veronika Solopova is a Senior Researcher at Johannes Gutenberg University Mainz and TU Berlin and co-lead of the XplaiNLP research group. Her research focuses on natural language processing and multimodal AI for social media analysis, with particular emphasis on disinformation, propaganda, harmful content, explainable AI, and human-centered content moderation. She received her doctorate in Computer Science from Freie Universität Berlin with summa cum laude for her work on hybrid AI systems for automated content moderation and analysis. Her doctoral research received the German Public Broadcasters’ ARD/ZDF Award “Women + Media Technology” (Frauen + Medientechnologie). Veronika is a Junior Fellow of the German Informatics Society (Gesellschaft für Informatik, GI) and has contributed to several national and European research projects on AI, disinformation, and fact-checking. Her work combines NLP, multimodal learning, and explainable AI to better understand and address complex information phenomena on the Web and social media.

Abstract: When we think about multimodal content on social media, we often think first of posts combining text and images. Yet the multimodal nature of the Web extends far beyond this familiar setting. Memes combine visual, textual with cultural signals; videos bring together speech, imagery, text overlays, and temporal information; and social media posts exist within networks of interactions, engagement patterns, and relationships that in computational terms represent multimodality. In this talk, we will explore the variety of multimodal information we encounter on social media, discuss how these heterogeneous signals can be connected and jointly modeled and why this is important to go beyond text. Drawing on examples from social media analysis, misinformation and manipulation detection, and multimodal content understanding, we will highlight both established approaches and emerging directions for moving beyond isolated modalities.

• 14:10 – 14:30: A Multimedia Perspective on the Study of Cognitive Biases in Large Language Models by Markus Schedl et al.
• 14:30 – 14:50: FashionTwin: A Holistic Benchmark for the Perception of AI-Generated Images in Fashion by Marina Paolanti et al.