Message tailoring and filtering
Trim & Filter Messages

1Core Problem: Infinitely Growing Message History

When building a chatbot, we store conversation messagesMessagesStateHere, new messages will be added in each round of dialogue. It looks perfect on the surface - but hides a fundamental engineering challenge:

Let’s use an intuitive Token consumption diagram to understand this problem:

This notebook describes a solution to this problemthree strategies, corresponding to different usage scenarios and trade-offs:

2MessagesState and basic chat graph review

Before explaining the three strategies in depth, let’s first review theBasic chat diagram, which is the starting point for all subsequent variations.

Define initial message list

Notebook first created a simple message list to simulate a conversation about "marine mammals":

from langchain_core.messages import AIMessage, HumanMessage

# 构建一个两条消息的初始对话历史
messages = [AIMessage("So you said you were researching ocean mammals?", name="Bot")]
messages.append(HumanMessage("Yes, I know about whales. But what others should I learn about?", name="Lance"))

for m in messages:
    m.pretty_print()
# 输出：
# =================================== Ai Message ==================================
# Name: Bot
# So you said you were researching ocean mammals?
# ================================ Human Message =================================
# Name: Lance
# Yes, I know about whales. But what others should I learn about?

The simplest chat graph (without any message management)

Notebook first created a "streaking" chat graph: put it in StateallThe message was passed intact to LLM.

from langgraph.graph import MessagesState, StateGraph, START, END
from langchain_deepseek import ChatDeepSeek

llm = ChatDeepSeek(model="deepseek-v4-pro")

# 节点：直接把 state["messages"] 全部传给 LLM
def chat_model_node(state: MessagesState):
    return {"messages": llm.invoke(state["messages"])}
    # ↑ 无任何裁剪！所有历史都发给模型

builder = StateGraph(MessagesState)
builder.add_node("chat_model", chat_model_node)
builder.add_edge(START, "chat_model")
builder.add_edge("chat_model", END)
graph = builder.compile()

What's built into MessagesState?

MessagesStateIt is the built-in State class provided by LangGraph, which is equivalent to:

from langgraph.graph.message import add_messages
from typing_extensions import Annotated, TypedDict

class MessagesState(TypedDict):
    # Annotated + add_messages：新消息追加到列表，而非覆盖
    messages: Annotated[list, add_messages]

The core isadd_messagesThis reducer: it returns a new message every time the nodeAppendto the existing list instead of replacing. This is the root cause of message accumulation and why we need to proactively manage the number of messages.

3Option 1: Use RemoveMessage to delete messages in the State layer of the graph

The first strategy isInsert a dedicated filter node into the graph, which is responsible for extracting data from StateDelete permanentlyUnnecessary old messages are removed, and then the updated State is handed over to the LLM node.

What is RemoveMessage?

RemoveMessageIt is a special message type provided by LangGraph. when you put aRemoveMessage(id="某条消息的id")Return to State,add_messagesThe reducer will recognize that this is a "delete instruction" and remove it from the State's message list.Completely removeThe message corresponding to id.

from langchain_core.messages import RemoveMessage

# 过滤节点：只保留最近 2 条消息，删除其余所有旧消息
def filter_messages(state: MessagesState):
    # state["messages"][:-2] = 除最后2条外的所有消息
    delete_messages = [RemoveMessage(id=m.id) for m in state["messages"][:-2]]
    return {"messages": delete_messages}
    # ↑ 返回一组"删除指令"，Reducer 会执行实际删除

# 聊天节点：此时 State 里只剩最近 2 条，正常传给 LLM
def chat_model_node(state: MessagesState):
    return {"messages": [llm.invoke(state["messages"])]}

# 图：先过滤，再聊天
builder = StateGraph(MessagesState)
builder.add_node("filter", filter_messages)
builder.add_node("chat_model", chat_model_node)
builder.add_edge(START, "filter")   # 先过滤
builder.add_edge("filter", "chat_model")  # 再聊天
builder.add_edge("chat_model", END)
graph = builder.compile()

Practical running demonstration

Notebook constructs a list of 4 messages, andExplicitly specify the id of each message（id="1"、id="2"...)——This is becauseRemoveMessageYou need to find the target message by id:

# 构建 4 条消息的对话，显式指定 id 以便后续删除
messages = [AIMessage("Hi.", name="Bot", id="1")]
messages.append(HumanMessage("Hi.", name="Lance", id="2"))
messages.append(AIMessage("So you said you were researching ocean mammals?", name="Bot", id="3"))
messages.append(HumanMessage("Yes, I know about whales. But what others should I learn about?", name="Lance", id="4"))

# 运行：filter 节点会删除 id="1" 和 id="2" 的消息
output = graph.invoke({'messages': messages})

4Option 2: filter_messages - filter within the node and keep the State intact

The second strategy is more "mild" than the first:The message list in State remains intact and is only temporarily filtered within the node., what is passed to LLM is a filtered subset, but the State itself is not affected.

core idea

In the chat node, do not putstate["messages"]Pass them all to LLM, but do a simple slicing first and only take the last 1 (or N):

# 节点：只把最后 1 条消息传给 LLM
def chat_model_node(state: MessagesState):
    return {"messages": [llm.invoke(state["messages"][-1:])]}
    #                                            ↑ [-1:] 只取最后 1 条
    #   state 里仍然保存着全部历史，只是传给 LLM 的只有最后 1 条

builder = StateGraph(MessagesState)
builder.add_node("chat_model", chat_model_node)
builder.add_edge(START, "chat_model")
builder.add_edge("chat_model", END)
graph = builder.compile()

Separation of State and LLM input parameters

Here's the key to understanding this scenario:What is stored in State ≠ what is received by LLM。

Full demo in Notebook

After constructing the filter graph, Notebook further added new messages to test multiple rounds of dialogue:

# 将上一次 LLM 的回复追加到消息列表
messages.append(output['messages'][-1])
# 追加新的用户问题
messages.append(HumanMessage("Tell me more about Narwhals!", name="Lance"))

# 此时 messages 列表有多条消息
for m in messages:
    m.pretty_print()

# 再次调用图（State 里有全部历史，但 LLM 只收到最后 1 条）
output = graph.invoke({'messages': messages})

Filtering flexibility: not just slicing

Beyond simple list slicingmessages[-1:], you can also pressMessage typeFilter, e.g. keep onlyHumanMessage, or pressidPrecise filtering:

from langchain_core.messages import filter_messages

# 只保留 HumanMessage（过滤掉所有 AI 消息）
human_only = filter_messages(state["messages"], include_types=[HumanMessage])

# 只保留 AIMessage
ai_only = filter_messages(state["messages"], include_types=[AIMessage])

# 排除某些类型
no_system = filter_messages(state["messages"], exclude_types=["system"])

# 按 name 过滤（只取某个 agent 的消息）
lance_msgs = filter_messages(state["messages"], include_names=["Lance"])

5Option 3: trim_messages - precise trimming based on the number of Tokens

The third strategy is the most refined control method: usetrim_messagesfunction,Precisely specify the maximum number of tokens allowed to be sent to LLM, the function automatically determines which messages to keep.

How trim_messages works

Unlike manual slicing (messages[-1:]），trim_messagesUsethe tokenizer of the model itselfto calculate the number of tokens for each message, and then start from the specified direction (latest or oldest), within no more thanmax_tokensUnder the premise of restrictions, try to retain as many messages as possible.

from langchain_core.messages import trim_messages

# 节点：使用 trim_messages 裁剪后再传给 LLM
def chat_model_node(state: MessagesState):
    messages = trim_messages(
        state["messages"],
        max_tokens=100,          # 最多允许 100 个 token
        strategy="last",          # 优先保留最新的消息
        token_counter=ChatDeepSeek(model="deepseek-v4-pro"),
                                   # 用这个模型来计算 token 数
        allow_partial=False,      # 不允许截断单条消息（要么整条保留，要么整条丢弃）
    )
    return {"messages": [llm.invoke(messages)]}

Detailed explanation of key parameters

Behavioral visualization of strategy="last"

Assume there are 5 messages totaling 280 tokens,max_tokens=100，strategy="last"：

Call trim_messages directly to observe the results

Notebook also shows direct calls outside the diagramtrim_messagesTo preview the cropping effect:

# 直接调用 trim_messages 预览裁剪结果（不经过图）
trimmed = trim_messages(
    messages,
    max_tokens=100,
    strategy="last",
    token_counter=ChatDeepSeek(model="deepseek-v4-pro"),
    allow_partial=False
)

# 查看裁剪后的消息列表
# 会输出保留下来的 N 条消息，其余旧消息被排除在外
for m in trimmed:
    m.pretty_print()

Usage of strategy="first"

When your scenario requiresKeep conversation starter(such as important system instructions or background information), you can instead usestrategy="first"：

# strategy="first": 优先保留最旧的消息（从开头往后取）
trimmed = trim_messages(
    messages,
    max_tokens=100,
    strategy="first",  # ← 改为 first
    token_counter=llm,
    allow_partial=False,
    include_system=True  # 始终保留 SystemMessage
)

Integration with graphs: complete example

# 使用 trim 方案构建图
def chat_model_node(state: MessagesState):
    trimmed = trim_messages(
        state["messages"],
        max_tokens=100,
        strategy="last",
        token_counter=ChatDeepSeek(model="deepseek-v4-pro"),
        allow_partial=False,
    )
    return {"messages": [llm.invoke(trimmed)]}

builder = StateGraph(MessagesState)
builder.add_node("chat_model", chat_model_node)
builder.add_edge(START, "chat_model")
builder.add_edge("chat_model", END)
graph = builder.compile()

# 在消息列表上追加更多消息后再调用
messages.append(output['messages'][-1])
messages.append(HumanMessage("Tell me where Orcas live!", name="Lance"))

# 调用图（State 包含全部历史，LLM 只收到 ≤100 token 的子集）
messages_out = graph.invoke({'messages': messages})

6Comparison and selection guide of three options

Each of the three strategies has its own merits and is suitable for different scenarios. The following is a comparison from multiple dimensions:

Selection decision tree

• 1

  Do you need to keep the complete conversation history?

  Yes → SelectFilterorTrim;no (identifying history is no longer useful) → considerRemoveMessage

• 2

  Do your message lengths vary widely?

  Yes (it can be long or short, it’s hard to predict the token) → Selecttrim_messages; No (the messages are relatively even) →Filter sliceenough

• 3

  Are you doing a PoC or a production environment?

  PoC/rapid prototyping →Filter slice(Easiest); production environment →trim_messages(accurate, predictable costs)

• 4

  Need to deal with SystemMessage (system prompt word)?

  Yes → Usetrim_messages + include_system=True, ensuring that the system prompt words are always retained

7Complete process: message list evolves with conversation

The notebook does not demonstrate each scenario in isolation, but runs through aContinuously growing list of messages, allowing you to see how messages accumulate over multiple rounds of conversations, and at what point filtering/clipping intervenes.

Complete evolution timeline of message lists

Behavioral differences between filter and trim in LangSmith

Notebook provides two versions of LangSmith tracking links: filter and trim. Key observations:

8Summary of core concepts and best practices

Best Practice Checklist