Building Customer-Support Memory That Survives an Audit

Watch the 3-minute demo

Every time a customer contacts support across separate sessions, they re-explain the same things: their account, their setup, who owns their escalations, what integration they're on, what level of urgency applies. The agent may be sharp inside a single conversation, but it has no continuity across conversations, so the burden of memory falls back on the customer. That's the wrong default.

There's a crowded answer to this now: bolt a vector store onto an LLM and call it memory. That produces recall. It does not produce something a support organization can stand behind. The question a support manager actually has to answer isn't "can the agent remember" — it's "why did the agent tell the customer that, and where did it come from." Most memory demos optimize for recall. Fewer show the evidence trail behind that recall. That's the gap Never Ask Twice is built to close: customer-support memory that removes repeat questions, and can show exactly what it remembered, where it came from, and when it should stop applying.

---

The Architecture Decision

The core choice was to give the agent durable, scoped memory of each customer rather than stuffing more transcript into the prompt. Those are not the same thing, and the difference is the whole design.

Memory lives in three tiers. Working memory is the current session. Episodic memory is the raw record of what happened in a conversation. Semantic memory is the small set of durable, distilled facts that survive across sessions: a customer's SLA tier, required configuration, escalation contact, integration.

When a session closes, the episodic record is distilled by Qwen into candidate semantic facts. Those candidates are validated against a constrained predicate vocabulary, scoped to the customer, assigned provenance, and written to the fact store. The deployed demo makes that write path visible: on session close, the memory trace shows the chain explicitly — supersession check, semantic facts written, Qwen distillation complete, session closed.

That visibility is the point. A memory system that silently accumulates facts is easy to demo and hard to trust. One that shows what it recalled, where it came from, and when it should stop applying is harder to build and much closer to production.

Built with Qwen for distillation, a scoped semantic fact store, and a live browser demo that exposes recall, write, and evaluation traces.

---

Why Context Windows Aren't Memory

A longer context window isn't memory. It's more text. Stuffing prior conversations into the prompt scales badly, carries stale claims forward, and gives you no clean way to answer "why did the agent believe that."

A scoped fact with a source and a validity window can be recalled precisely, superseded when it changes, and audited when a human asks why the agent behaved a certain way. The novel mechanism here isn't recall — it's the provenance chain: every fact knows where it came from and when it should stop applying.

The forgetting policy is a feature, not an afterthought. A fact can be durable without being eternal: it has provenance, it can expire, and a newer fact that contradicts an older one supersedes it. Memory that only grows is memory you cannot trust, because you can never tell whether what it recalls is still true.

---

What the Live Demo Proves

The video above follows the exact path described here: memory on, session close, memory off, and the semantic fact store. Watch the right-side trace, not just the reply. The product is not only the answer — it's the evidence trail behind the answer. In the demo, watch for three things:

1. The same vague support request behaves differently with memory on vs. off.
2. Recall chips show the exact facts the agent used, not a generic "I remember you."
3. Session close writes semantic facts through Qwen distillation, visible in the trace.

The demo centers on one deliberately vague request: "the integration is failing again, can you route this?"

With memory on, the default customer (Jason) has prior context, and the agent recalls the relevant facts — Salesforce integration, SSO configuration, Gold SLA, Priya as escalation contact — without asking Jason to restate anything. The response isn't just smart, it's inspectable: the remembered facts appear as recall chips in the reply, and the matching rows light up in the memory trace. The interface shows the evidence behind the answer.

Send the same request with memory off, and the agent asks for the missing details: SLA tier, configuration, integration, escalation contact. It has no prior facts to rely on, and the trace says so. Memory on turns a vague request into an action. Memory off turns it into another interview.

---

How It's Measured

With memory, the repeat-question rate is 0.00. Without it, 1.00. The same agent, evaluated both ways against a fixed set — every returning customer forced to re-explain in the no-memory condition, none in the memory condition.

In the product UI that number is presented as an evaluation snapshot derived from the current fact store, not a live tally of the open session. The distinction matters: a live counter would answer "what happened in this one conversation," while the ablation answers the product claim — does memory stop returning customers from being asked to repeat durable context. That's the right measurement, because it isolates the one thing the product claims to improve.

---

What Broke, and How We Caught It

The memory-write path passed every test and had never run against the real model.

That was the failure I'm most glad we caught. The eval harness reported a clean repeat-question rate. Unit tests were green. The demo recalled seeded facts. From the outside, everything looked right. But the mechanism that turns a real conversation into durable memory had never successfully written a usable fact in production.

Distillation is the step where a closed session's transcript gets compressed by Qwen into durable facts. Every test we wrote exercised that path against a fake client that returned a hand-shaped response. The fake always returned what the system expected. The real model didn't.

Running session close against real Qwen on the deployed path, three failures fell out in sequence:

• Write-path parsing. One bad Qwen candidate discarded the whole batch instead of just itself.
• Raw-response validation. An unguarded parse one layer earlier threw before the first fix ever ran.
• Prompt/schema mismatch. The real model was never told the allowed predicates or the expected output shape, so it returned zero usable facts even once both crashes were fixed.

In detail: Qwen returned a candidate with a predicate outside our approved set, the code parsed the whole batch eagerly, and one out-of-vocabulary predicate threw and discarded every fact in the session. The fix was to validate candidate-by-candidate, so one bad fact drops without taking the good ones. Then we hit the same class of bug one layer earlier — an unguarded parse where Qwen's raw response first enters the system; our first fix hadn't mattered because this one threw first. Finally, with both crashes fixed, the root cause: even running cleanly, Qwen returned zero usable facts, because the prompt told the model to use approved predicate names without listing them or specifying the output shape. The fake client had always known the shape because we wrote it to. The real model had no reason to.

The fix was to make the prompt explicit: enumerate the allowed predicates, specify the output structure, validate each candidate independently. The deployed trace now shows the full lifecycle on session close, so the write path isn't a claim, it's visible.

The lesson isn't "we had a bug." It's that a test double let the most important mechanism in the product stay green and hollow. Testing against a stand-in for the model tests your code against your own assumptions about the model. The only thing that catches the gap between those assumptions and reality is running the real thing and watching what it does.

Two more that mattered.

Tenant isolation: facts are scoped to one customer and must never cross that boundary; we found a path where the read boundary held but the write path didn't enforce the same invariant, and fixed it by enforcing customer scope on every path that touches the store, verified against the write path directly rather than assuming a safe read implied a safe write. A memory leak is worse than an ordinary data leak, because the agent doesn't just reveal a wrong fact, it may act on it.

Recall honesty: the interface signals recall by lighting up the facts the agent used, and we found a path where that signal could come from presentation state rather than the actually-cited facts. A memory system that fakes the appearance of recall is worse than one that doesn't recall, because it trains the user to trust evidence that isn't evidence. We tied the signal to the real cited-facts payload: if the data doesn't back the claim, the interface doesn't make it.

---

What Production Would Actually Require

What we built proves the mechanism. A real enterprise deployment would still have to solve four things we deliberately scoped out.

Production would require four upgrades:

1. Stronger forgetting. Facts here have provenance, validity, and supersession — enough to show memory can stay current. Production needs conflict resolution when two sources disagree with no clear recency winner, confidence decay for facts that were true once and quietly stopped being, and a human-auditable trail for why a fact was forgotten, because in support a wrongly-forgotten fact is a customer-facing failure someone has to explain.

2. Async promotion. We distill inline on session close, which is right for a demo and wrong for scale. Production distills asynchronously, batches across concurrent sessions, retries safely, and routes untrusted distillation output to a reviewable dead-letter path instead of dropping it.

3. Structural isolation. It's not enough to remember to check customer scope on every path. A boundary maintained by discipline is a boundary that eventually leaks. Production multi-tenancy should make cross-customer recall structurally impossible at the storage and query layer.

4. Operational telemetry. We measure the thing that proves the thesis. A real support org also needs recall precision and latency in the live path, distillation quality sampling, fact-drift detection, and alerts when a customer's memory no longer matches what they'd recognize as true. The measurement we have proves the claim. It's not enough to operate the system.

None of this is a disclaimer. It's the real-world shape of the problem, and naming it is the difference between a demo that claims to be production-ready and a system that knows what production would still cost.

---

Why This Matters Beyond Support

Never Ask Twice is a support demo, but the underlying pattern is broader: scoped facts, provenance, expiry, supersession, and visible recall for AI systems that act across time. That is the memory layer Marcelle Labs is exploring through Vreko — not agents that remember everything, but systems that remember responsibly.