I read studies on how sarcasm is perceived and produced by various LLMs. The research shows that AI has trouble with producing witty remarks but perception of sarcasm/irony has come a long way. Models such as LVLM (Large Vision Language Model) were used in some of these studies, showing that combining NLP and computer vision, i.e. multimodality, gives some of the best results.

When AI companions become witty: Can human brain recognize AI-generated irony?

This was a small study with a sample size of 59 participants (5 were excluded), that were Mandarin speaking. They gave participants either literal or ironic pieces of text. The aim of the study was to understand brain processes of AI generated irony in comparison to human generated irony.

Through EEG recordings, they found that humans didn’t fully engage in AI generated irony and most participants mismatched statements from AI to irony. When considering anthropomorphism (giving the AI human like characteristics), participants didn’t give the AI any credit for having the capacity to create intentional irony. The intention behind the irony proved to be more important than the semantics around how the irony was delivered.

Participants who perceived AI to be more human like tended to engage more with mechanisms that were similar to those of how humans interact with each other. This was also true when AI was perceived as trustworthy.

The study did find that anthropomorphism in human-ai interaction led to participants attributing human like characteristics to the AI, such as age, gender, perspective taking abilities and linguistic competence but they withheld deep meaningful conversation or genuine mental state with the AI.

This study found that participants engaged with AI generated irony as something that was quite mechanical and linguistically the AI might be skilled with advances in NLP but it lacks the ability to understand how humans perceive genuine intentional to something that is created artificially. The study spoke to some degree around intentional stance adoption and people’s mental models of AI and how this processing took place in their brains with P200 and P600 within EEGs. From this study it’s evident the AI cannot generate on the fly witty remarks that feel as though a human was in the conversation.

Sarc7: Evaluating Sarcasm Detection and Generation with Seven Types and Emotion-Informed Techniques

The data set MUStARD was used, also known as multimodal sarcasm detection dataset. They looked at seven types of sarcasm, self-deprecating, brooding, deadpan, polite, obnoxious, raging, and manic. Introducing Sarc7, a benchmark for fine-grained sarcasm classification and generation, they labelled every sarcastic utterance with one of these seven types of sarcasm.

They set out with two evaluation tasks, classification and generation. Generating specific types of sarcasm, they were scored on incongruity (semantic mismatch), shock value (intensity of sarcasm), context dependency (conversation history) and emotion.

The study included several models for the benchmark, these included GPT-4o, Claude 3.5 Sonnet, Gemini 2.5, Qwen 2.5 and Llama 4 Maverick. They found emotion based prompting led to higher F1 (precision and recall metric of classification) of scores for Claude, Gemini, Llama and Qwen. Emotions should moderate success when identifying sarcasm and struggled to categorise different types of sarcasm.

Sarc7 enhances the understanding of LLM’s ability to identify subtle cues in sarcastic statements.

They found a number of limitations within their methodology of Sarc7. During the process of annotating the MUStARD dataset they found many inconsistencies due to the the structure, even with peer-reviews. Second, the dataset introduced biases to the classification because it didn’t include enough different types of sarcasm. Singlecategory was also seen to be an issue with regards to skewed results due to a classification have the ability to have more than one label but only getting one classification due to the prompt.

Towards Multimodal Sarcasm Detection, (An Obviously Perfect Paper)

In the study they took YouTube videos from shows like The Big Bang Theory and Friends and got the system to label them to see which ones were sarcastic, alongside character’s characteristics. They then fed this into imageNet with Pool 5 level, to be able to understand the features of the characters. They also used text modality with Tweets from Twitter.

Video did better than text, primarily because of facial expressions when conveying sarcasm. It showed multi-modality (in this case, text, audio and visual) is important when looking into sarcasm. This study looked at AI being able to detect sarcasm, not produce sarcastic remarks.

World model inspired sarcasm reasoning with large language model agents

Looks at how agents interpret sarcasm with various reasoning. Here they computer vision and regression techniques on various data sets and came up with WM-SAR (World Model inspired Sarcasm Reasoning).

They set up five agents:

• Literal meaning: extracted the literal menaing of the input of text u and played the role of observation in WM-SAR
• Context constructor: constructed a background situation where utterance could occur and played the role of latent state inference
• Norm and expectation reasoner: estimated norm expected valence based on social norms inferred context and played the role of prediction
• Inconsistency detector: computed the consistency and the sign discrepancy indication and played the role of prediction error computation
• Mental state and intention reasoner: based on the text (u) and context C(u), performing ToM reasoning to infer speaker intentions and emotions to output a sarcasm intention score of Tsar(u, C(u)) and required for decision in the model

The final judgment is not delegated to an LLM. The signals extracted by the agents are integrated by LR, and the sarcasm probability is estimated as P(sarcasm = 1 | u).

The Inconsistency Detector computes (as a deterministic), the inconsistency between the observation output by the Literal Meaning Agent and the prediction output by the Norm and Expectation Reasoner.

Sarcasm does not arise solely from the structural condition of semantic inversion, but is often accompanied by social and interpersonal intentions such as anger, disappointment, contempt, or distancing. By explicitly modeling this aspect through ToM reasoning, the Mental State and Intention Reasoner complements pragmatic distinctions that cannot be captured by structural signals such as D and SD alone. In addition to T (u, C(u)) and Tsar(u, C(u)), the agent also generates an explanation, such that T (u, C(u)) = {intentions, emotions}.

Within this study, three sarcasm detection sets were used:

• IAC-V1: a dataset with comments collected from political forums, it contained sarcastic expressions in argumentative contexts
• IAC-V2: extending IAC-V1 to include more sarcastic and non-sarcastic instances to examine model generalisations
• SemEval-2018 Task 3: a sarcasm and irony detection dataset for English tweets on Twitter, it contains short and informal expressions including slang and abbreviated forms - which is helpful for intention reasoning

To verify the effectiveness of the proposed method, comparisons are conducted with the following representative baselines, ranging from conventional deep learning models to LLM-based reasoning approaches.

Baselines ranged from traditional deep learning models (BiLSTM, CNN-based classifiers) to LLM-based approaches like zero-shot GPT-4.1-mini and the multi-stage CAF-I framework.

WM-SAR average accuracy and Macro-F1 scored 0.750 across the three datasets vs 0.731 for zero-shot GPT-4.1-mini and 0.718 for BERT.

They also tested how much the backbone LLM matters. Swapping in GPT-4.1 instead of GPT-4.1-mini barely changed anything (0.751 vs 0.750) but GPT-4.1-nano dropped to 0.653, so the framework can handle different backbones but only down to a point. It still needs a model that’s decent at social reasoning.

Since the five agents run in parallel and only the LR step is sequential, WM-SAR took about 7.65s per sample compared to ~21.10s for CAF-I.

Ablation-wise, the Mental State and Intention Reasoner mattered most, pulling out its sarcasm-intention score hurt performance more than removing inconsistency magnitude or sign disagreement did.

The experiments suggest that intention is especially important. Removing the sarcasm-intention score causes the largest performance degradation, while removing inconsistency magnitude or sign disagreement produces smaller but consistent declines. Ablation Study.

The main future role of WM-SAR is as a portable experimental framework. Researchers can plug in new LLMs, add better context, alter the final learner, and inspect how each reasoning component behaves on new kinds of sarcasm. Its main weakness is that it is biased toward sarcasm expressed through a mismatch between literal meaning and normative expectation. It may therefore miss “positive sarcasm,” where the surface meaning and the expected evaluation appear aligned and the sarcasm is conveyed through subtler discourse or interpersonal cues.

This perspective is not limited to sarcasm, but also suggests that other NLP tasks involving implicature, euphemism, humor, offensive language, and more generally social reasoning and value judgment, may benefit from being reformulated as world model inspired structures to more systematically exploit LLM capabilities.

This study focused on the speakers intention and context in regards to sarcasm with modelled cognitive structure. Previous deep learning methods have focused on black boxes, make it hard to explain why a judgement of sarcasm was made in regards to a specific piece of text or utterance.

Can Large Vision-Language Models Understand Multimodal Sarcasm?

Looked at four LVLM models, LLaVA, MiniGPT, InstructBLIP and GPT-4o, with GPT-4o having the highest accuracy of 65.9%.

• LLaVA-v1.5: combines CLIP (visual encoder) with LLaMA (language model), instruction-tuned on GPT-4-generated image-based linguistic data. They evaluate the 7B variant.
• MiniGPT: pairs a frozen visual encoder with a frozen LLaMA backbone, aligned via instruction tuning on instruction datasets. They evaluate the MiniGPT-v2 LLaMA-7B variant.
• InstructBLIP: built on pre-trained BLIP-2, made multimodal through visual-language instruction tuning. They evaluate the Vicuna-7B variant.
• GPT-4o: the vision-optimized GPT-4 variant, refined through pre-training, instruction tuning, and RLHF. They evaluate the GPT-4o-mini version.

To extract fine-grained visual elements from images, they used Fast-RCNN for object recognition, which described objects with attributes such as shape and colour.

The MSD (multimodal sarcasm detection) task on GPT-4o achieved 75.3% accuracy and 14.2 improvement over the baseline. The method achieved the best improvement on the models Instruct BLIP and MiniGPT.

Seeing Sarcasm Through Different Eyes: Analyzing Multimodal Sarcasm Perception in Large Vision-Language Models

The premise of this research was to understand whether different LVLMs interpret sarcasm in different ways and whether models could demonstrate human-like thinking of parsing sarcastic and literal meanings. They set out a four task evaluation framework:

• Binary sarcasm classification (BSC)
• Ternary sarcasm classification (TSC)
• Sarcasm-centric scoring (SCS)
• Literal-centric scoring (LCS)

They set out four tasks designed to assess the model’s ability for multimodal sarcasm understanding from different perspectives.

Binary Sarcasm Classification (BSC): The BSC task requires the model to classify a given text-image pair (Ti, Vi) into sarcastic or non-sarcastic categories

Ternary Sarcasm Classification (TSC): To test whether LVLMs can directly identify neutral samples, we introduce the TSC task

Sarcasm-Centric Scoring (SCS): This task requires the model to score the degree to which a given text-image pair can be interpreted as sarcastic

Literal-Centric Scoring (LCS): Similar to SCS, this task requires the model to score how well a given text-image pair can be interpreted from a literal, non-sarcastic perspective

They found most models showed lower rationale consistency scores for SCS and LCS tasks in comparison to BSC and TSC tasks, with higher standard deviations. This shows SCS and LSCS tasks require more nuanced reasoning and subjective interpretation.

Across experiments of twelve open source models and benchmarks, they found two findings:

• LVLMs showed better confidence on literal interpretations rather than sarcastic ones
• SCS and LCS of interpretive prompts introduced more variety in response that in BSC and TSC of classification prompts (regardless of models)

Building my own sarcasm classifier

After reading the papers above, I chose to build a small sarcasm classifier, with the potential of looking into LLMs in the future. Taking data sets from Hugging Face which cover a sizeable set of tweets to understand whether the tweet would be considered sarcastic or not and a given score.

The code can be viewed here.

The tweets are scored based on “irony” from cardiffnlp/tweet_eval and TF-IDF enables the weighting alongside regression. As seen above, there are many sophisticated benchmarks and evals arising from research around sarcasm. With the rise of techniques within computer vision to map video to text to sarcasm to LVLM models, the way sarcasm is measured and produced varies significantly.

Once the work is done with TF-IDF, it’s then given a label based on the weighting, which enables a user to understand if it is perceived as sarcastic. This work could further be developed with different LLMs and datasets.

To develop this further, an LLM could act as the classification model. It would receive the user’s message and relevant conversation context through a prompt, then classify the text as sarcastic or non-sarcastic and provide a confidence score. The existing SemEval dataset could be retained as a test set to evaluate how accurately the LLM performs compared with the original TF-IDF classifier.