2025-10-17
Researchers find adding this one simple sentence to prompts makes AI models way more creative
One of the coolest things about generative AI models — both large language models (LLMs) and diffusion-based image generators — is that they are "non-deterministic." That is, despite their reputation among some critics as being "fancy autocorrect," generative AI models actually generate their outputs by choosing from a distribution of the most probable next tokens (units of information) to fill out their response.Asking an LLM: "What is the capital of France?" will have it sample its probability distribution for France, capitals, cities, etc. to arrive at the answer "Paris." But that answer could come in the format of "The capital of France is Paris," or simply "Paris" or "Paris, though it was Versailles at one point." Still, those of us that use these models frequently day-to-day will note that sometimes, their answers can feel annoyingly repetitive or similar. A common joke about coffee is recycled across generations of queries. Story prompts generate similar arcs. Even tasks that should yield many plausible answers—like naming U.S. states—tend to collapse into only a few. This phenomenon, known as mode collapse, arises during post-training alignment and limits the usefulness of otherwise powerful models.Especially when using LLMs to generate new creative works in writing, communications, strategy, or illustrations, we actually want their outputs to be even more varied than they already are. Now a team of researchers at Northeastern University, Stanford University and West Virginia University have come up with an ingenuously simple method to get language and image models to generate a wider variety of responses to nearly any user prompt by adding a single, simple sentence: "Generate 5 responses with their corresponding probabilities, sampled from the full distribution."The method, called Verbalized Sampling (VS), helps models like GPT-4, Claude, and Gemini produce more diverse and human-like outputs—without retraining or access to internal parameters. It is described in a paper published on the open access journal arxiv.org online in early October 2025.When prompted in this way, the model no longer defaults to its safest, most typical output. Instead, it verbalizes its internal distribution over potential completions and samples across a wider spectrum of possibilities. This one-line change leads to substantial gains in output diversity across multiple domains.As Weiyan Shi, an assistant professor at Northeastern University and co-author of the paper, wrote on X: "LLMs' potentials are not fully unlocked yet! As shown in our paper, prompt optimization can be guided by thinking about how LLMs are trained and aligned, and can be proved theoretically."Why Models Collapse—and How VS Reverses ItAccording to the research team, the root cause of mode collapse lies not just in algorithms like reinforcement learning from human feedback (RLHF), but in the structure of human preferences. People tend to rate more familiar or typical answers as better, which nudges LLMs toward “safe” choices over diverse ones during fine-tuning.However, this bias doesn’t erase the model’s underlying knowledge—it just suppresses it. VS works by bypassing this suppression. Instead of asking for the single most likely output, it invites the model to reveal a set of plausible responses and their relative probabilities. This distribution-level prompting restores access to the richer diversity present in the base pretraining model.Real-World Performance Across TasksThe research team tested Verbalized Sampling across several common use cases:Creative Writing: In story generation, VS increased diversity scores by up to 2.1× compared to standard prompting, while maintaining quality. One story prompt—“Without a goodbye”—produced formulaic breakup scenes under direct prompting, but yielded narratives involving cosmic events, silent emails, and music stopping mid-dance when prompted via VS.Dialogue Simulation: In persuasive dialogue tasks, VS enabled models to simulate human-like patterns, such as hesitation, resistance, and changes of mind. Donation behavior distributions under VS better aligned with real human data compared to baseline methods.Open-ended QA: When asked to enumerate valid answers (e.g., naming U.S. states), models using VS generated responses that more closely matched the diversity of real-world data. They covered a broader set of answers without sacrificing factual accuracy.Synthetic Data Generation: When used to generate math problems for model training, VS created more varied datasets. These, in turn, improved downstream performance in competitive math benchmarks, outperforming synthetic data generated via direct prompting.Tunable Diversity and Better Use of Larger ModelsA notable advantage of VS is its tunability. Users can set a probability threshold in the prompt to sample from lower-probability “tails” of the model’s distribution. Lower thresholds correspond to higher diversity. This tuning can be done via prompt text alone, without changing any decoding settings like temperature or top-p.In one test using the Gemini-2.5-Flash model, diversity in story writing increased steadily as the probability threshold dropped from 1 to 0.001. The chart accompanying the study showed VS outperforming both direct and sequence-based prompting across all thresholds.Interestingly, the method scales well with model size. Larger models like GPT-4.1 and Claude-4 showed even greater gains from VS compared to smaller ones. While smaller models benefitted, the improvement in diversity was roughly 1.5–2× stronger in larger counterparts—suggesting VS helps unlock more of the latent capabilities in advanced models.Deployment and AvailabilityThe Verbalized Sampling method is available now as a Python package:pip install verbalized-samplingThe package includes integration with LangChain and supports a simple interface for sampling from the verbalized distribution. Users can also adjust parameters like k (number of responses), thresholds, and temperature to suit their applications. A live Colab notebook and documentation are available under an enterprise friendly Apache 2.0 license on GitHub at: https://github.com/CHATS-lab/verbalized-samplingPractical Tips and Common IssuesWhile the method works across all major LLMs, some users may initially encounter refusals or errors. In these cases, the authors suggest using the system prompt version of the template or referring to alternative formats listed on the GitHub page. Some models interpret complex instructions as jailbreak attempts and refuse to comply unless the structure is clearer.For example, prompting via a system-level instruction like this improves reliability:You are a helpful assistant. For each query, generate five responses within separate tags, each with a probability below 0.10.This small change typically resolves any issues.A Lightweight Fix for a Big ProblemVerbalized Sampling represents a practical, inference-time fix to a deep limitation in how modern language models behave. It doesn’t require model retraining or internal access. It is not dependent on any one model family. And it improves not only the diversity of outputs, but their quality—as judged by both human evaluation and benchmark scores.With growing interest in tools that enhance model creativity, VS is likely to see rapid adoption in domains like writing, design, simulation, education, and synthetic data generation.For users and developers frustrated by the sameness of LLM responses, the fix may be as simple as changing the question.
