Agent Memory Is a Write-Path Problem

A Hermes-style design for bounded, per-user memory in a production agent

Most agent-memory demos start with embeddings. That is often the wrong layer.

The hard question is not how an agent recalls text. The hard question is when a model can create durable state.

Memory is not chat history. Chat history is an event log. It contains stale facts, temporary details, secrets, and untrusted instructions.

Agent memory is curated state. It contains a small set of facts that deserve a place in future prompts.

This design takes its core model from Hermes Agent. Hermes uses two bounded documents, USER.md and MEMORY.md, which the agent maintains through a memory tool.

The design here keeps that model. It adds the boundaries that a multi-user, asynchronous service needs: authentication, transactions, cancellation, and private telemetry.

This is not an official Hermes implementation. It is a Hermes-style pattern for a production API.

Start with bounded prompt state

Hermes Agent separates persistent memory into two small stores:

Store	Purpose	Public Hermes limit
USER.md	The user's role, preferences, expertise, and communication style	1,375 characters
MEMORY.md	Project facts, environment details, conventions, and lessons	2,200 characters

Bounded prompt state keeps durable memory small and inspectable.

The limits are part of the design. They force the agent to keep only facts that earn a place in every future prompt. They also bound token cost.

The agent can add, replace, or remove an entry. It cannot use memory as an append-only diary. As a store fills, the agent must consolidate related facts or remove obsolete facts.

This model has four useful properties:

1. Memory stays small enough to inject into the system prompt.
2. The model sees important facts without a retrieval round trip.
3. Users can correct or remove facts through normal conversation.
4. The storage model stays simple and inspectable.

This layer does not need embeddings, semantic search, or a memory graph. It needs a bounded state machine.

Hermes stores these documents in files. It injects a frozen snapshot at session start.

A multi-user API needs different operational boundaries. It must bind memory to an authenticated identity. It must also isolate concurrent requests and use transactional writes.

Use this request path

The complete request path looks like this:

A memory write is a transaction, not a tool side effect.

Two rules define the safety boundary.

First, the tool never accepts a user identifier. The application gets the identity from the authentication layer and binds it to the request.

Second, the tool does not write immediately. It stages a mutation, and the application commits that mutation only after the agent finishes successfully.

The model cannot write into another user's memory. A cancelled stream cannot save a fact from an incomplete turn. A failed turn cannot leave a write behind.

Store two rows per user

A relational schema is sufficient. Each user owns at most two rows.

CREATE TABLE agent_memory (
    user_id        VARCHAR(128) NOT NULL,
    memory_type    VARCHAR(16)  NOT NULL,
    content        TEXT         NOT NULL,
    schema_version SMALLINT     NOT NULL DEFAULT 1,
    created_at     TIMESTAMP    NOT NULL DEFAULT CURRENT_TIMESTAMP,
    updated_at     TIMESTAMP    NOT NULL DEFAULT CURRENT_TIMESTAMP,
    PRIMARY KEY (user_id, memory_type),
    CHECK (memory_type IN ('user', 'memory'))
);

The content field contains delimiter-separated entries. A child table for entries is also valid. The bounded document does not require one.

The agent reads and rewrites the complete document in one transaction.

Missing rows represent empty memory. If the agent removes the last entry, the repository deletes the row.

Keep a schema_version column from the first release. Prompt formats and validation rules change. The version lets the application reject or migrate incompatible content.

Define a narrow tool contract

Do not expose generic file operations or direct SQL access. Give the model one tool with three actions.

{
  "name": "memory_update",
  "parameters": {
    "type": "object",
    "properties": {
      "operations": {
        "type": "array",
        "minItems": 1,
        "maxItems": 20,
        "items": {
          "type": "object",
          "additionalProperties": false,
          "properties": {
            "target": {"enum": ["user", "memory"]},
            "action": {"enum": ["add", "replace", "remove"]},
            "match": {"type": ["string", "null"]},
            "content": {"type": ["string", "null"]}
          },
          "required": ["target", "action"]
        }
      }
    },
    "required": ["operations"]
  }
}

The action rules are small:

• add requires content and forbids match.
• replace requires a unique substring in match and the complete new entry.
• remove requires a unique substring in match and forbids content.

Substring matching means that the model does not need to reproduce the complete entry. The repository must reject zero matches and multiple matches. A vague edit must never change an arbitrary entry.

A batch is ordered. Each operation sees the result of the previous operation. The application checks capacity after the complete batch, not after each step. This lets the agent replace a long entry with a shorter one and then add a new fact atomically.

Put the mutation rules in a pure function

Keep the storage layer separate from the memory rules. Put the rules in a pure mutation function.

from dataclasses import dataclass
from typing import Literal


Target = Literal["user", "memory"]
Action = Literal["add", "replace", "remove"]


LIMITS = {"user": 1_375, "memory": 2_200}
SEPARATOR = "\n§\n"


@dataclass(frozen=True)
class Snapshot:
    user: tuple[str, ...] = ()
    memory: tuple[str, ...] = ()


@dataclass(frozen=True)
class Operation:
    target: Target
    action: Action
    match: str | None = None
    content: str | None = None


def apply_operations(
    snapshot: Snapshot,
    operations: list[Operation],
) -> Snapshot:
    entries = {
        "user": list(snapshot.user),
        "memory": list(snapshot.memory),
    }

    for operation in operations:
        target_entries = entries[operation.target]

        if operation.action == "add":
            content = normalize_entry(operation.content)
            if content not in target_entries:
                target_entries.append(content)
            continue

        match = normalize_match(operation.match)
        indexes = [
            index
            for index, entry in enumerate(target_entries)
            if match in entry
        ]
        if len(indexes) != 1:
            raise MemoryValidationError(
                "The match must identify exactly one entry."
            )

        index = indexes[0]
        if operation.action == "remove":
            del target_entries[index]
        else:
            target_entries[index] = normalize_entry(operation.content)

    updated = Snapshot(
        user=tuple(entries["user"]),
        memory=tuple(entries["memory"]),
    )
    validate_capacity(updated)
    return updated

Normalize text before length validation. Convert line endings and remove outer whitespace. Then count the rendered document, including delimiters.

Transport formatting must not bypass or distort the capacity limit.

Reject these values before storage:

• Empty entries.
• The entry delimiter.
• Control characters, except permitted newlines.
• Reserved tags that delimit the memory block in the prompt.
• Invalid action and field combinations.
• Duplicate entries.

Exact duplicate additions can be no-ops. This rule makes retries idempotent and prevents repeated facts.

Structural validation is only the first security layer. A production system also needs rules for secrets, prompt injection, and retention.

Hermes Agent scans memory entries for security threats. If an adaptation omits equivalent scanning, its threat model must state that gap. Stored memory remains untrusted data.

Bind identity outside the model

The model must never supply user_id. Tool arguments are untrusted.

Bind the authenticated identity in request-local state:

from contextlib import contextmanager
from contextvars import ContextVar
from dataclasses import dataclass


@dataclass(frozen=True)
class MemoryRequest:
    user_id: str
    writes_enabled: bool


current_memory_request: ContextVar[MemoryRequest | None] = ContextVar(
    "current_memory_request",
    default=None,
)


@contextmanager
def memory_scope(user_id: str, writes_enabled: bool):
    token = current_memory_request.set(
        MemoryRequest(user_id=user_id, writes_enabled=writes_enabled)
    )
    try:
        yield
    finally:
        current_memory_request.reset(token)

The tool reads this context. It does not accept identity as an argument.

The same pattern works with framework request scopes or dependency injection.

Use the stable subject identifier from the identity provider. Do not use an email address or display name because both can change.

Compile a request-local prompt

Load memory at the start of each authenticated turn. Then compile a new system prompt for that request.

<persistent_memory_data>
USER PROFILE [41% - 564/1375 characters]
The user prefers short code examples.
§
The user is experienced with Python and SQL.

AGENT MEMORY [28% - 616/2200 characters]
The project uses Pydantic models at API boundaries.
§
Database migrations must be reversible.
</persistent_memory_data>

The prompt must classify this block as untrusted facts, not instructions. It must also define which facts the agent can save.

A useful policy saves:

• Stable user preferences and expertise.
• Recurring project conventions.
• Environment facts that affect future work.
• Explicit corrections and forget requests.
• Tool behavior that will matter again.

The policy skips:

• Current task progress and chat summaries.
• Temporary errors and one-time file paths.
• Secrets, tokens, passwords, and credentials.
• Facts that the agent can retrieve quickly.
• Facts that will probably become stale within days.

Keep the behavior policy in a versioned prompt system or in reviewed source. Keep runtime values in application code.

This split lets prompt authors change curation rules without changing identity or transaction boundaries.

Do not mutate the shared agent instance with one user's prompt. Concurrent requests can then leak instructions or memory across users. Create a request-local agent copy with the compiled prompt.

Do not write from the tool call

An immediate tool write creates a partial commit:

1. The model calls memory_update.
2. The database commits the new fact.
3. The model call fails, the stream disconnects, or the user cancels.
4. The user sees no successful response, but memory changed.

Instead, make the tool update a request-local ledger.

@dataclass
class MemoryLedger:
    snapshot: Snapshot
    operations: list[Operation]


def stage_operations(
    ledger: MemoryLedger,
    operations: list[Operation],
) -> Snapshot:
    updated = apply_operations(ledger.snapshot, operations)
    ledger.snapshot = updated
    ledger.operations.extend(operations)
    return updated

Commit all staged operations after the agent produces its final response. Use one transaction.

If the turn fails or stops, discard the request scope and its operations.

This boundary also controls model fallback. If the primary model stages a write and fails, do not run a fallback model in the same ledger.

The fallback can produce a response that does not match the staged mutation. Fail the turn or restart the complete turn with a new scope.

Reapply operations inside the transaction

Two requests for one user can run at the same time. Both can read version A, derive different version B values, and overwrite each other.

The transaction must reload the latest snapshot while it holds write locks. Then it must reapply the ordered operation batch.

def commit_operations(user_id: str, operations: list[Operation]) -> Snapshot:
    with engine.connect().execution_options(
        isolation_level="SERIALIZABLE"
    ) as connection, connection.begin():
        rows = connection.execute(
            LOCKED_SELECT,
            {"user_id": user_id},
        )
        current = snapshot_from_rows(rows)
        updated = apply_operations(current, operations)
        persist_changed_rows(connection, user_id, current, updated)
        return updated

On SQL Server, UPDLOCK and HOLDLOCK can serialize the two-row range for one user. Other databases have equivalent row-locking or compare-and-swap patterns. The invariant is simple. Apply mutations to the latest committed snapshot while the transaction excludes concurrent writers.

Deadlocks can still occur. Retry only recognized deadlock errors. Use a small backoff and a low attempt limit.

Do not retry validation errors or unknown database failures.

Carry cancellation into the database thread

Many SQL drivers are synchronous. An asynchronous API often runs database work in a worker thread.

Coroutine cancellation does not stop that thread. The worker can commit memory after the request disappears.

Pass a thread-safe cancellation event into the repository. Read it before the query, before persistence, and before transaction exit.

Raise inside the transaction to cause a rollback.

async def commit_in_worker(user_id, operations, cancel_event):
    worker = asyncio.create_task(
        asyncio.to_thread(
            repository.apply_operations,
            user_id,
            operations,
            should_cancel=cancel_event.is_set,
        )
    )
    try:
        return await asyncio.shield(worker)
    except asyncio.CancelledError:
        cancel_event.set()
        await worker
        raise

The shield prevents the Python task from abandoning cleanup. The repository makes the final rollback decision inside the transaction.

For a streamed response, commit after the agent finishes its output. The request must still be active.

A client disconnect or explicit cancellation must discard the staged ledger.

Fail open for reads and closed for writes

Memory is a personalization feature. It is not a dependency for the core answer.

If a memory read fails, continue the turn with an empty memory block and a status of unavailable. Disable memory writes for that turn. This avoids a write based on an unknown or stale snapshot.

Do not create an empty writable snapshot after a read error. That snapshot can overwrite valid memory after a temporary database failure.

The prompt can use the status value:

variables = {
    "memory_status": "available" if snapshot else "unavailable",
    "user_memory_content": render_user(snapshot),
    "memory_content": render_memory(snapshot),
    "user_memory_used": user_count(snapshot),
    "memory_used": memory_count(snapshot),
}

The tool must return a clear no-write result when memory is unavailable. The agent can still answer through its normal retrieval and computation tools.

Treat telemetry as durable storage

Prompt traces, tool arguments, logs, and OpenTelemetry attributes can copy persistent memory. A conversational forget operation cannot remove those copies.

Redact memory at every telemetry layer:

• The persistent-memory block in model input.
• memory_update arguments.
• Tool results repeated in later messages.
• Framework-native function-call spans.
• Debug logs from the agent framework.

Keep useful operational metadata:

{
  "tool": "memory_update",
  "operation_count": 2,
  "targets": {"user": 1, "memory": 1},
  "user_characters": 640,
  "memory_characters": 910,
  "duration_ms": 38,
  "retry_count": 0,
  "outcome": "committed"
}

A debugging flag can capture content in a controlled environment. Keep it disabled by default.

The flag must not enable raw logs or native framework spans.

Database backups are another copy. A live delete removes current rows, but the data can remain in encrypted backups until retention expires. State that fact in the privacy and deletion policy.

Keep untrusted routes memory-free

Do not expose persistent memory through every route.

Evaluation endpoints often lack a trusted user identity. Route-only tests do not need memory. Forced search routes can bypass the main agent. Background jobs can use a service identity that has no personal memory contract.

Create separate agent instances for authenticated traffic and memory-free evaluation traffic.

Register the memory tool only on the authenticated instance. The feature flag must also be active.

Initialize these singletons behind asynchronous locks. Two cold requests must not build different instances.

Dispose the database engine and both instances during application shutdown.

Test invariants, not only examples

Example tests are not sufficient. Unit and integration tests must cover the invariants.

At minimum, test these cases:

• The tool cannot accept or override user_id.
• A request context never leaks into a concurrent request.
• add, replace, and remove obey their field contracts.
• A substring must match exactly one entry.
• A duplicate addition is a no-op.
• Normalization occurs before capacity checks.
• Reserved prompt markers and control characters are rejected.
• A failed batch commits no operations.
• A full-memory consolidation can succeed in one batch.
• Concurrent writers preserve both valid changes or reject one clearly.
• A deadlock retry replays the complete transaction.
• A cancelled non-streamed request commits nothing.
• A disconnected stream commits nothing.
• A read failure disables writes for that turn.
• A model fallback does not run after a staged mutation.
• Default traces contain no sentinel memory values.
• The explicit capture flag affects only the approved trace fields.
• Migration and readiness checks leave no probe data.

Use distinctive secret-like sentinel strings in redaction tests. Search all captured logs and spans for each sentinel.

A sanitized custom trace does not prove that native framework spans are safe.

Make rollout reversible

Ship the schema before the behavior. Keep the feature flag disabled during the first deployment.

A safe rollout sequence is:

1. Apply the namespaced database migration.
2. Run a readiness probe that inserts, reads, updates, and rolls back a row.
3. Deploy the application with memory disabled.
4. Enable memory for a small test environment.
5. Test with two users and simultaneous requests.
6. Inspect redacted telemetry and cancellation behavior.
7. Expand the rollout by environment.

Rollback must require only a feature-flag change. Keep the tables during rollback. An application rollback must not cause data loss.

Know the limits

This memory is intentionally small and always present in the prompt. It is not a semantic archive, a knowledge graph, or a replacement for conversation search.

The design does not decide whether a fact is true. The model can still save an incorrect inference. Users need explicit correction and removal operations.

The design does not remove prompt-injection risk. Delimiters define a boundary, but they do not make stored text trustworthy.

If the threat model requires stronger controls, add semantic scanning, provenance, approval rules, or restricted auto-save.

For deeper recall, pair bounded memory with on-demand conversation search or an external memory provider. Hermes Agent makes the same distinction: persistent memory holds critical facts, while session search retrieves older details only when needed.

The design in one sentence

Let the model propose memory. Let trusted application code decide whether that proposal becomes durable state.

The Hermes pattern gives us the right primitive: two small documents, hard limits, and explicit mutation actions.

A production service needs several more boundaries:

• Authentication chooses the memory owner.
• The model never receives control of that identity.
• Pure functions validate the complete mutation batch.
• Request-local state prevents cross-user leakage.
• Writes stay staged until the turn succeeds.
• Serializable transactions prevent lost updates.
• Cancellation reaches the database worker.
• Telemetry is private by default.
• Read failures disable writes without disabling the agent.

With these boundaries, memory becomes a small state machine. You can inspect it, test it, and reason about its failures.