“Insanity is doing the same thing over and over and expecting different results.”
— Albert Einstein

At Voice of Reason AI, we’ve been exploring how the prompt structure influences the behavior of our language model (LLM). The prompt isn’t just a technical setup—it’s what shapes how the LLM perceives context and responds, almost like a guide that sets its “mindset” for each interaction.

For our work, two components define the prompt:

  1. System Prompt: This sets the tone, behavior, and approach for the LLM. It’s like a foundation that influences how the model interprets and responds to user inputs.
  2. User/Conversation Prompt: This is the dynamic, immediate part of the prompt, containing recent user input and relevant history, allowing the LLM to keep conversations coherent and contextual.

In our case, we’re modifying the system prompt with each inference request to keep interactions fresh. However, we quickly learned that including the LLM’s previous responses, especially if they include any negative or self-critical reasoning, can lead to a self-reinforcing feedback loop. The more the LLM sees its own past logic, the more likely it is to adopt those patterns, even if they’re not ideal. This mirrors human behavior—negativity can be subtle but powerful, slowly shifting one’s perspective.

Why Prompt Structure Matters to Us

In our journey, we noticed several issues that emerged when the LLM kept “seeing” its previous responses:

  1. Self-Reinforcement of Negative Bias: If the LLM reads its own critical reasoning repeatedly, it starts to perceive that reasoning as the norm. This subtly pushes it toward similar tones or patterns, even when it’s not helpful.
  2. Anchoring Effect: The model might get “stuck” on specific logic patterns, which limits its ability to explore diverse responses. This is especially challenging if the LLM interprets past reasoning as “correct” simply because it’s repeated.
  3. Reduced Exploration: When the LLM relies on its past responses, it reduces its flexibility. We want it to respond dynamically to each new interaction—not fall into repetitive logic.

How We Handle System Prompt Updates for Better Results

To counteract these issues, we’ve implemented several strategies to keep our LLM adaptive and focused on user needs:

1. Selective Exclusion of Past Responses

  • We limit the inclusion of the LLM’s past logic to avoid reinforcement of negative patterns, focusing on user inputs and relevant factual data. By stripping out unnecessary logic, we’re minimizing the risk of bias creeping in and keeping the LLM focused on the user.

2. Separate “Memory” Prompt Structure

  • We keep memory context separate from reasoning, storing user context and factual summaries in a dedicated “memory” space. This structure allows the LLM to stay informed without conflating it with its own reasoning, which could skew its responses.

3. Dynamic Rephrasing of Negative Feedback

  • If it’s necessary to include feedback, we rephrase it in neutral or constructive terms. For instance, instead of including self-critical statements, we focus on language that encourages improvement without self-reinforcement. This makes feedback constructive, guiding the LLM to build positively on past interactions.

4. Optimizing the System Prompt for Constructive Responses

  • We designed our system prompt to emphasize user-focused, adaptive responses. It’s specifically structured to avoid relying on past logic as a reference, so the LLM can stay flexible. For example, we prompt it to dynamically address the latest input and prioritize constructive insights.

5. Periodic System Prompt Refresh

  • To prevent outdated or irrelevant content from accumulating, we periodically review and refresh the system prompt. This keeps the LLM adaptable, giving it a “clean slate” for more objective responses.

Conclusion

By using these methods, we can maintain the balance between helpful context and unnecessary bias. Carefully managing the system prompt, structuring the input, and regularly refreshing the prompt help us guide the LLM toward responses that are accurate, flexible, and constructive. This makes it a far more effective tool for the individuals who rely on us, remaining adaptive to their needs. For us, it’s not just about a better AI—it’s about building a companion that genuinely supports and uplifts.