Qwen MemoryAgent

The problem

• Most AI chat is stateless: every session starts from zero, so an assistant either re-explains itself constantly or loses useful context after a few hours.
• Naively storing every message solves recall but not usefulness: without a way to weigh, deduplicate, and expire memories, a memory store just becomes noise an LLM has to wade through.

The approach

• A full pipeline runs on every turn: semantic recall of relevant past memories, retrieval into an importance-ordered context block, Qwen extracting structured facts from the new turn, Qwen scoring each fact’s importance (0.0–1.0), tiered TTL expiry based on that score, and a separate Qwen-arbitrated "smart forget" pass that reviews expired memories instead of deleting on a blunt timer.
• Weighted duplicate/conflict detection before anything is stored: candidates are ranked by a blended 60% similarity + 40% importance score; above 0.96 similarity it’s rejected outright as a duplicate, above 0.82 Qwen arbitrates whether the new fact should UPDATE the old one or stand as a NEW independent fact.
• The importance-scoring prompt bakes in two rules added after testing surfaced real failure modes: a specific fact (e.g. "grows cherry tomatoes and basil") must score at least as high as the general category it belongs to (e.g. "is a gardener"), and a person’s own name is floored at 0.6+ importance so it can’t accidentally expire within a day.
• The extraction prompt explicitly refuses to generate memories from absence-of-information statements, so the store doesn’t fill up with facts like "doesn’t have a car."
• Extraction and scoring run as a background task after the chat reply is already sent, so a user isn’t kept waiting on the embed/dedup/arbitrate/score chain for facts that don’t affect the answer they’re currently reading.

What broke (and how it got fixed)

• Early scoring treated a vague statement and the specific fact behind it as equally important, which meant vague summaries could outscore, and outlive, the concrete facts that actually made recall useful. Fixed by explicitly ranking specific facts at or above their general category in the scoring prompt.
• Without a floor, a person’s own name could be scored as casual conversation and expire within a day of being mentioned; an identity fact disappearing is a worse failure than almost anything else the memory store could get wrong, so names are now floored at 0.6+ importance.
• Because stored memories get re-injected into the system prompt of unrelated future sessions, a malicious message stored as a "memory" could otherwise function as a persistent, cross-session jailbreak. Both the recall and extraction prompts now explicitly frame all memory content as untrusted data to read, never instructions to follow.

Outcome

• An end-to-end test suite makes live HTTP calls against a running instance (local or the real Alibaba Cloud deployment) rather than mocking anything, covering health checks, store/recall, the importance-scoring calibration rules above, deduplication and conflict arbitration, negative-fact filtering, cross-session recall, and smart forget.
• The same suite includes a structural validation pass over the frontend that catches regressions like a hardcoded API endpoint, a leaked default credential in the login form, or a dynamic value interpolated into an inline event handler, before any of them reach the live deployment.
• The README is explicit that the prompt-injection hardening is "prompt-level defense-in-depth, not a hard guarantee": LLM-based defenses reduce, but don’t eliminate, injection risk.

Stack

Honest scope

A hackathon submission deployed to a single Alibaba Cloud ECS instance for the Qwen Cloud Hackathon: real, working infrastructure (JWT auth, a real Postgres+pgvector store), but not a production, multi-tenant service.