LLM testing paper accepted at NLDB 2025

Our paper “Test It Before You Trust It: Applying Software Testing for Trustworthy In-context Learning” was accepted at NLDB 2025, the 30th International Conference on Natural Language & Information Systems. The work is a collaboration with the ReaLearn lab of Dr. Teeradaj Racharak, at JAIST and now at Tohoku University. It applies metamorphic testing to large language models on two tasks, sentiment analysis and question answering. It changes the input in ways that keep its meaning, for example by swapping characters, replacing words with synonyms, and switching between active and passive sentences. Tests on OpenAI's GPT-4o and Google's Gemini-2.0-Flash found several cases where the models changed their answers on such inputs.