Memory System
BioCortex uses a three-tier memory system so the agent can use the current conversation, recall past analyses, and store biological facts. Episodic recall is enhanced with Engram-inspired mechanisms for fast, relevant retrieval.Three Tiers
| Tier | Role | Content |
|---|---|---|
| Working memory | Current session | Conversation messages; hierarchical compression when over budget. |
| Episodic memory | Past analyses | Task, plan summary, tools used, domains, result summary, success, quality, lessons, bio-entities. |
| Semantic memory | Facts | (Subject, predicate, object) triples; integrated with the knowledge graph. |
Working Memory
- Holds the current session conversation (and optionally plan state).
- When the token estimate exceeds the configured (or auto-calibrated) maximum:
- The oldest portion is compressed (e.g. summarized or truncated).
- The most recent messages are kept intact.
- Auto-calibration: If
working_memory_max_tokensis set to-1, the framework sets it to 60% of the reasoning model’s input context (clamped between 16K and 600K). See Context Window and Budget.
Episodic Memory (Engram-Inspired)
Episodic memory stores past analysis sessions and provides recall for the Planner so it can reuse successful patterns and avoid past mistakes.Stored per episode
- Task description and plan summary
- Tools used, domains
- Result summary, success flag, quality score
- Lessons learned
- Biological entities mentioned
Engram-inspired retrieval
-
N-gram fingerprint index
- Task and result text are tokenized (lowercase, no punctuation, stop words removed; biological IDs preserved).
- For each episode, unigram, bigram, trigram hashes (e.g. 48-bit) are computed and stored in an inverted index: fingerprint → set of episode IDs.
- Query text is fingerprinted the same way; candidates are episodes sharing at least one fingerprint → O(1) lookups per query fingerprint.
-
Multi-signal index
- For each episode we store: N-gram fingerprint set, tool names, domains, biological entities.
- Query returns candidates with per-signal scores (e.g. Jaccard-like for fingerprints, exact match for tools/domains/entities).
- Scores are combined (e.g. weighted sum) into a single relevance score per episode.
-
Relevance gate
- A gated fusion function (with optional domain hint and success filter) produces a final relevance score.
- Episodes below a configurable min_relevance are dropped.
- Remaining episodes are sorted by score, truncated to top_k, and then to a token budget.
-
build_episodic_context
- Entry point used by the Orchestrator:
build_episodic_context(query, top_k=5, token_budget=2000, ...). - Returns a token-budgeted string of relevant past analyses (task, relevance, result summary, lessons, tools).
- This string is injected into the Planner prompt so the agent can reuse pipelines and avoid known failure modes.
- Entry point used by the Orchestrator:
Semantic Memory
- Stores facts as (subject, predicate, object, source, confidence, timestamp).
- Queryable by entity (e.g. substring match).
- Integrated with the BioKnowledgeGraph: facts can be added as graph nodes/edges and used for retrieval and KG-grounded verification.
Configuration
memory.working_memory_max_tokens— Max tokens for working memory;-1for auto-calibration.- Episodic:
top_k,min_relevance, token budget, and weights for multi-signal fusion. - Persistence paths for episodic and semantic stores.
biocortex.memory.engine.
Next Steps
- Context Window and Budget — How working memory size is calibrated and used.
- Knowledge Graph — Integration with semantic memory and retrieval.