Chatbot with summary memory
Conversation compression · Progressive summary · MemorySaver

1Problem background: Why do long conversations cause amnesia?

In previous lessons, we have learned to useMessagesStateMaintain conversation history, also tried usingtrim_messagesandfilter_messagesto control the context length. But these methods have a common flaw: they aredisposable——Old messages are deleted directly, and that part of the conversation is permanently lost.

The challenges of real LLM conversations are:

• Limited context window: Even models such as GPT-4o and DeepSeek have a Token upper limit. After exceeding the limit, the request will fail or be truncated.
• Cost increases linearly with length: Each incoming historical message requires a Token fee. The longer the conversation, the more expensive each round of calls is.
• Increased latency: Longer context means LLM requires longer inference time
• Important information is truncated: Simply retain only the most recent N messages, possibly losing key context (user name, user preferences, task goals)

The image above shows how a long conversation is dangerously close to the context limit.trim directlyThe result is to make LLM forget the key information at the beginning of the conversation; and the plan for this lesson——Conversation Summarization——Use LLM to convert old messagesCompressed into a summary textKeep it instead of deleting it directly.

2Solution: Generate rolling summaries with LLM

The summary strategy implemented in this lesson is calledProgressive Rolling Summary. Here's how it works:

• 1

  Normal conversation accumulation

  The conversation proceeds normally, and messages are appended to State one by one.messagesin the list.

• 2

  Trigger threshold detection

  After each round of dialogue, checkmessagesThe length of the list. Once the set threshold is exceeded (6 items in this example), the summary process is entered.

• 3

  LLM generates summary

  Send all current messages + a prompt word "please generate summary" to LLM to get a summary text. If an old digest already exists, LLM is asked to merge the old digest and the new message into a new digest (rolling update).

• 4

  Clean up old messages

  use RemoveMessageDelete all but the most recent 2 old messages, keeping only the latest conversation snippets.

• 5

  Summary injected into next round

  The next time you call LLM, putsummaryThe contents of the field are wrapped intoSystemMessage, appended to the front of the message list to let LLM know the key points of the previous conversation.

This mechanism achieves realLossless long-term memory: The message list is always kept short, and the summary field continues to accumulate semantic information as the conversation progresses, becoming the "long-term memory carrier" of LLM.

3State extension: add summary field in MessagesState

The first step to implement the summary mechanism is to extend the State data structure. LangGraph provides a built-inMessagesState, which already containsmessagesfields (withadd_messagesreducer, supports append semantics). We just need to inherit it and add an additionalsummaryString field:

from langgraph.graph import MessagesState

class State(MessagesState):
    # MessagesState 已内置：
    #   messages: Annotated[list[AnyMessage], add_messages]
    # 我们新增一个 summary 字段，用于存储对话摘要
    summary: str

There are a few key points worth understanding here:

• MessagesStatewhat is: It is a LangGraph pairTypedDictpre-packaged, internalmessagesThe field is usedAnnotated[list, add_messages]Annotation means that messages returned multiple times willAppendinstead of covering
• summary: strbehavior: Ordinary string fields do not have reducer, the default isCoverage semantics--every timesummarize_conversationWhen the node returns a new summary, it directly replaces the old summary.
• initial value：summaryThe field does not exist when the graph is first started. You need to usestate.get("summary", "")Safe reading to avoid KeyError

4call_model node: Inject summary into conversation context

call_modelIt is the core node responsible for calling LLM. What's special about it is that before calling LLM, it willCheck if summary exists in State, if so, wrap the summary intoSystemMessageInsert it at the front of the message list so that LLM can "see" the background of the previous conversation.

from langchain_core.messages import SystemMessage, HumanMessage, RemoveMessage

def call_model(state: State):

    # ① 尝试获取现有摘要，不存在则默认为空字符串
    summary = state.get("summary", "")

    # ② 如果有摘要，把它包装成 SystemMessage 前置到消息列表
    if summary:
        system_message = f"Summary of conversation earlier: {summary}"
        # SystemMessage 放最前面，后面接最近的几条真实消息
        messages = [SystemMessage(content=system_message)] + state["messages"]
    else:
        # 没有摘要时直接使用原始消息列表
        messages = state["messages"]

    # ③ 调用 LLM，传入（可能已注入摘要的）消息列表
    response = model.invoke(messages)

    # ④ 返回 AI 的回复（add_messages Reducer 会自动追加）
    return {"messages": response}

Let’s walk through a conversational demonstration to intuitively understand what LLM actually receives:

Why use SystemMessage instead of HumanMessage?

In LangChain’s message system,SystemMessageRepresents system-level directives and is the most appropriate way to tell LLM "your current background/role/rules". The summary serves as a context for the conversation, with theSystemMessagePass in, semantically speaking, useHumanMessageMore accurate and more consistent with most LLM training conventions.

5summarize_conversation node: generate and update summaries

This is the core node of this lesson. It accomplishes three things:

1. Generate summary: Call LLM to compress all current messages into a summary text
2. rolling update: If an old digest already exists, LLM is required to merge the old digest with the new message instead of rewriting it from scratch.
3. Clean up old messages:useRemoveMessageDelete all old messages except the last 2

def summarize_conversation(state: State):

    # ① 获取已有摘要（可能为空字符串）
    summary = state.get("summary", "")

    # ② 根据是否已有摘要，构造不同的提示词
    if summary:
        # 已有摘要 → 要求"在现有摘要基础上延伸"，而不是从头写
        # 这就是"滚动摘要"的关键：避免重复计算旧内容
        summary_message = (
            f"This is summary of the conversation to date: {summary}\n\n"
            "Extend the summary by taking into account the new messages above:"
        )
    else:
        # 第一次生成摘要
        summary_message = "Create a summary of the conversation above:"

    # ③ 构造给 LLM 的完整消息列表（历史消息 + 摘要请求）
    messages = state["messages"] + [HumanMessage(content=summary_message)]
    response = model.invoke(messages)

    # ④ 生成 RemoveMessage 对象列表，标记需要删除的旧消息
    # state["messages"][:-2] = 除最后2条外的所有消息
    delete_messages = [RemoveMessage(id=m.id) for m in state["messages"][:-2]]

    # ⑤ 同时返回：新摘要（覆盖旧摘要） + 删除指令（清理旧消息）
    return {"summary": response.content, "messages": delete_messages}

How RemoveMessage works

RemoveMessageIt is a special message type provided by LangChain Core. when it is put inmessagesWhen the list returns,add_messagesThe reducer will recognize it andRemove from State the correspondingidnews, instead of appending RemoveMessage itself. This is an elegant "message deletion signal" mechanism.

Why keep the most recent 2 items instead of 0?

Keeping the last 2 messages (usually the last round of Human + AI message pairs) serves two purposes:

• dialogue coherence: When the next round of conversation begins, the most recent conversation snippet serves as immediate context, which is more natural than just a summary
• Summary Quality Assurance: If the summary is triggered next time, these 2 messages will be included in the new summary as "new messages" to ensure that no content is missed in the scrolling summary.

6should_continue: conditional edge that controls when the summary is triggered

Conditional edge routing functionshould_continueDetermines the direction of the graph after each round of dialogue: it ends directly (END), it is better to do the summary first and then end it.

from langgraph.graph import END
from typing_extensions import Literal

def should_continue(state: State) -> Literal["summarize_conversation", END]:

    """返回下一个要执行的节点名称。"""

    messages = state["messages"]

    # 关键阈值：超过 6 条消息就触发摘要
    if len(messages) > 6:
        return "summarize_conversation"

    # 消息数在阈值内，直接结束（等下一轮继续积累）
    return END

Meaning and adjustability of threshold 6

A threshold of 6 is an example value. In actual projects, you can adjust based on the following factors:

7Complete assembly diagram: MemorySaver, Thread, checkpointer

Now put all the pieces together to build a complete Chatbot graph with persistent memory.

from IPython.display import Image, display
from langgraph.checkpoint.memory import MemorySaver
from langgraph.graph import StateGraph, START

# ① 创建图的蓝图（使用自定义的 State）
workflow = StateGraph(State)

# ② 注册两个节点
workflow.add_node("conversation", call_model)       # LLM 对话节点
workflow.add_node(summarize_conversation)           # 摘要节点（函数名即节点名）

# ③ 设置入口：START → conversation
workflow.add_edge(START, "conversation")

# ④ 设置条件边：conversation 完成后，由 should_continue 决定走向
workflow.add_conditional_edges("conversation", should_continue)

# ⑤ 摘要节点完成后直接结束
workflow.add_edge("summarize_conversation", END)

# ⑥ 编译时注入 MemorySaver 作为 checkpointer
# MemorySaver 是内存中的 KV 存储，生产环境可换成 SqliteSaver / PostgresSaver
memory = MemorySaver()
graph = workflow.compile(checkpointer=memory)

# 可视化图结构
display(Image(graph.get_graph().draw_mermaid_png()))

The role of MemorySaver and Thread

Givecompile()incomingcheckpointer=memoryAfter that, the picture hasPersistence: After each step is executed, LangGraph will automatically save the current State snapshot to MemorySaver. The next time the same thread_id is called, the graph continues from the last saved state rather than starting from the beginning.

# 创建一个对话线程（Thread），通过 thread_id 标识
config = {"configurable": {"thread_id": "1"}}

# 第一轮对话
input_message = HumanMessage(content="hi! I'm Lance")
output = graph.invoke({"messages": [input_message]}, config)

# 第二轮对话（同一 thread_id，图从上次状态继续）
input_message = HumanMessage(content="what's my name?")
output = graph.invoke({"messages": [input_message]}, config)

# 读取当前 State 的摘要字段
current_summary = graph.get_state(config).values.get("summary", "")

Thread analogy: Thread can be understood as different channels of Slack - eachthread_idIt is an independent dialogue channel and does not interfere with each other. You can maintain a Thread for User A and User B at the same time, and their States (including summaries) are completely independent.

8Complete operation demonstration and summary of key technologies

Complete conversation flow in Notebook

Here's a complete running demo in Notebook, showing the complete process from a normal conversation to a triggered summary:

After the fourth round of dialogue,should_continuedetectedlen(messages) == 8 > 6, route tosummarize_conversationnode. LLM generates a summary, the old 6 messages are deleted and only the latest 2 are retained.

# 查看摘要结果
graph.get_state(config).values.get("summary", "")

# 输出：
# 'Lance introduced himself and mentioned that he is a fan of the
#  San Francisco 49ers, specifically highlighting his admiration for
#  Nick Bosa. The conversation noted that as of September 2023, Nick
#  Bosa became the highest-paid defensive player in NFL history with
#  a five-year, $170 million contract extension with the 49ers.'

Summary of key technical points

Next step: Module 3 Human-in-the-Loop

After mastering summary memory, your Chatbot has been able to handle infinitely long conversations without losing its memory. Module 3 will introduce on this basisManual intervention mechanism (Breakpoint): Pause at a specific node of graph execution and wait for human confirmation or modification of the State before continuing. This is a key technology for building a safe and controllable Agent system.