AI Agentic Patterns in ByteChef (Part 7): Chat Memory That Compacts by Turns

TL;DR: Simple chat memory drops the oldest messages to stay small, and it can drop a tool call while keeping its result. Chat Memory v2 in ByteChef stores the chat as a log of events and compacts it by whole turns, so a tool exchange is never split. Older turns are archived, not lost: a conversation search tool lets the agent look them up. To see it, import the Chat that compacts its history template. Its limits are small on purpose, so compaction shows up after a few messages.

Every chat agent has the same slow problem. Each message adds to the history, and the whole history goes into the next prompt. Long conversations get slower, cost more and eventually hit the model's context limit.

The usual fix is to keep only the last N messages. That works until a tool is involved. A tool call and its result are separate messages. Cut the history between them, and the model sees an answer with no question, or a question with no answer. It doesn't fail loudly. It just gets confused.

Chat Memory v2 fixes this by treating the conversation as turns instead of loose messages. This post shows how, using one template.

---

The Template: Chat That Compacts Its History

The Chat that compacts its history template has three steps:

1. New Chat Request: a message arrives in the workflow chat.
2. AI Agent: Claude Sonnet as the model, and Chat Memory (version 2) in its Memory slot.
3. Reply: the agent's answer goes back to the chat.

The Chat Memory element is set up like this:

Field	Value in the template
Conversation ID	${chat_1.conversationId}
Compaction Strategy	Turn window (by turns)
Max turns	3
Enable conversation search tool	on

The system prompt tells the agent what to expect: "Your visible history is short. If the user asks about something said earlier that you no longer see, use the conversation search tool to find it, and say that you searched."

Three turns is far too small for real use. It's set that low so you can watch compaction happen in a short demo.

---

Turns, Not Messages

Chat Memory v2 is built on the Spring AI community Session library. It stores a conversation as an ordered log of events. Each user message, assistant reply, tool call and tool result is one event.

A turn is one user message plus everything that follows it: replies, tool calls and tool results, up to the next user message. Compaction always works on whole turns. A turn is never cut in half, so a tool call and its result stay together or leave together.

With the Turn window strategy, two pieces work together:

• When to compact: as soon as the history has more turns than Max turns.
• How to compact: archive the oldest turns until only Max turns remain. The system prompt is always kept.

In the template, that means the prompt never carries more than three turns of history, no matter how long the chat runs.

---

Archived, Not Deleted

Compaction moves old events out of the prompt and into an archive. The agent no longer sees them, but they're still stored.

That's what Enable conversation search tool is for. It gives the agent a tool called conversation_search. The agent passes a keyword, and the tool searches the whole conversation, including archived messages that are still kept. Results come back in pages; Search page size sets how many per page, 10 by default.

So the agent's view is short, but its reach is long. It answers from the last three turns, and when you ask about something older, it can search for it. The template's system prompt asks it to say when it searched, so you can tell recall from guessing.

Two limits to know:

• Summaries are not searched. If you use Recursive summarization (below), the summaries the model writes don't show up in search results. The original messages do, while they're kept.
• The archive has a cap on some storage. Redis and S3 storage keep the 1000 most recent archived events. The built-in Chat Memory keeps the number set by bytechef.ai.memory.session-max-archived-events.

---

Four Ways to Compact

Compaction Strategy is a dropdown. Each option shows only the fields it needs:

Strategy	Fields	What it does
None		Never compacts. The default.
Sliding window (by events)	Max events	Once the history has more than Max events, keeps the most recent Max events.
Turn window (by turns)	Max turns	Once the history has more than Max turns, keeps the most recent Max turns.
Token count	Max tokens	Keeps the history within an estimated token budget.
Recursive summarization (LLM)	Max events, Max events to keep, Overlap size	Once the history has more than Max events, a model summarizes the older events and the most recent Max events to keep stay in full.

For Recursive summarization, ByteChef checks two rules when the run starts: Max events must be greater than Max events to keep, and Overlap size must be less than Max events to keep. Overlap size is how many of the kept events the summarizer also sees, so the summary connects to what follows.

The second model

Look at the template again. The Chat Memory element has its own model, Claude Haiku, separate from the agent's Claude Sonnet.

Only Recursive summarization uses that model. With Turn window, which the template uses, old turns are archived and no model is called. The Haiku model is there so you can switch the dropdown to Recursive summarization without adding anything. Summaries then come from a smaller, cheaper model than the one answering your users. If you pick Recursive summarization and there is no model on the Chat Memory element, the run fails with a clear error.

---

Where the History Lives

Chat Memory v2 comes in several components with the same fields. They differ only in storage:

• Chat Memory: the built-in option. Storage follows the server's bytechef.ai.memory.provider setting: JDBC, Redis, AWS S3 or in-memory.
• JDBC Chat Memory: a relational database.
• Redis Chat Memory: Redis, with its own connection.
• AWS S3 Chat Memory: JSON objects in an Amazon S3 bucket, with its own connection.
• In Memory Chat Memory: in the server's memory, so it's gone after a restart.

The template uses the built-in Chat Memory, so it needs no extra connection.

The S3 backend is ByteChef's own implementation of the Session library's storage interface. It stores each session as one JSON object and uses S3 conditional writes (If-Match on the ETag), so two writers can't silently overwrite each other's events.

---

Set Up the Template

1. Import the template. Open Chat that compacts its history and add it to a project.
2. Connect the models. Add your Anthropic connection to both models: the agent's and the one on Chat Memory. That's the only setup; there is no canvas note because nothing else is needed.
3. Have a short conversation. Open the chat and send four or five messages on different topics. Put something specific in the first one, like a name or a number.
4. Ask about the first message. By now the first turn has been archived. Ask the agent what you said at the start. It no longer sees that turn, so the system prompt tells it to use conversation_search, and to say that it searched.

This is a real test run. The code name was in the first message, which had already left the three-turn window, so the agent searched the archive to answer.

To watch it work, use the Agent Playbook in the editor. Its tool-call cards show conversation_search running, with the keyword the agent chose and the messages it found.

When you move past the demo, raise Max turns to something like 10 to 20, or switch to Token count or Recursive summarization. The search tool works the same with any strategy.

---

What You Get from Spring AI, and What ByteChef Adds

The compaction strategies, triggers and conversation_search come from the Spring AI community Session library. ByteChef runs it directly and adds:

• Fields instead of code. In Java you build the memory advisor, pick a trigger and strategy, and register the search tool. In ByteChef it's one dropdown, a few numbers and a toggle.
• A model picker for summaries. The summarizer is a child element on Chat Memory, so it can be any model ByteChef supports.
• Clear errors. Invalid Recursive summarization settings, or a missing summarizer model, stop the run with a message that names the problem.
• Storage choices. JDBC, Redis, in-memory and ByteChef's own S3 backend, as separate components with the same fields.
• A search tool Claude will call. The library's conversation_search also asks the model for a private "inner monologue", and Claude refuses the whole turn rather than fill it in. ByteChef registers the tool with only a keyword and a page number.

---

Short-Term and Long-Term Memory

Chat Memory keeps this conversation coherent. It doesn't carry facts from one conversation to the next. For that, ByteChef has the Auto Memory Tool, where the agent writes down what's worth keeping and reads it back in later chats. It works in deployed runs, not editor test runs. The two work side by side on the same agent: Chat Memory in the Memory slot, the Auto Memory Tool in the Tools slot.

The same rule as in Part 1 applies: put each kind of knowledge where it's loaded only when needed. Skills load procedures on demand. Compaction keeps old turns out of the prompt until the agent searches for them.

---

Wrapping Up

Chat history grows with every message, and dropping old messages one by one can break a tool call in half. Chat Memory v2 compacts by whole turns, archives what it removes and lets the agent search the archive. You choose the strategy and the storage on the canvas.

Try it: import the Chat that compacts its history template, connect Anthropic, chat for five messages and ask what you said first.