Memory System

BioCortex uses a three-tier memory system so the agent can use the current conversation, recall past analyses, and store biological facts. Episodic recall is enhanced with Engram-inspired mechanisms for fast, relevant retrieval.

Three Tiers

TierRoleContent
Working memoryCurrent sessionConversation messages; hierarchical compression when over budget.
Episodic memoryPast analysesTask, plan summary, tools used, domains, result summary, success, quality, lessons, bio-entities.
Semantic memoryFacts(Subject, predicate, object) triples; integrated with the knowledge graph.

Working Memory

  • Holds the current session conversation (and optionally plan state).
  • When the token estimate exceeds the configured (or auto-calibrated) maximum:
    • The oldest portion is compressed (e.g. summarized or truncated).
    • The most recent messages are kept intact.
  • Auto-calibration: If working_memory_max_tokens is set to -1, the framework sets it to 60% of the reasoning model’s input context (clamped between 16K and 600K). See Context Window and Budget.

Episodic Memory (Engram-Inspired)

Episodic memory stores past analysis sessions and provides recall for the Planner so it can reuse successful patterns and avoid past mistakes.

Stored per episode

  • Task description and plan summary
  • Tools used, domains
  • Result summary, success flag, quality score
  • Lessons learned
  • Biological entities mentioned

Engram-inspired retrieval

  1. N-gram fingerprint index
    • Task and result text are tokenized (lowercase, no punctuation, stop words removed; biological IDs preserved).
    • For each episode, unigram, bigram, trigram hashes (e.g. 48-bit) are computed and stored in an inverted index: fingerprint → set of episode IDs.
    • Query text is fingerprinted the same way; candidates are episodes sharing at least one fingerprint → O(1) lookups per query fingerprint.
  2. Multi-signal index
    • For each episode we store: N-gram fingerprint set, tool names, domains, biological entities.
    • Query returns candidates with per-signal scores (e.g. Jaccard-like for fingerprints, exact match for tools/domains/entities).
    • Scores are combined (e.g. weighted sum) into a single relevance score per episode.
  3. Relevance gate
    • A gated fusion function (with optional domain hint and success filter) produces a final relevance score.
    • Episodes below a configurable min_relevance are dropped.
    • Remaining episodes are sorted by score, truncated to top_k, and then to a token budget.
  4. build_episodic_context
    • Entry point used by the Orchestrator: build_episodic_context(query, top_k=5, token_budget=2000, ...).
    • Returns a token-budgeted string of relevant past analyses (task, relevance, result summary, lessons, tools).
    • This string is injected into the Planner prompt so the agent can reuse pipelines and avoid known failure modes.

Semantic Memory

  • Stores facts as (subject, predicate, object, source, confidence, timestamp).
  • Queryable by entity (e.g. substring match).
  • Integrated with the BioKnowledgeGraph: facts can be added as graph nodes/edges and used for retrieval and KG-grounded verification.

Configuration

  • memory.working_memory_max_tokens — Max tokens for working memory; -1 for auto-calibration.
  • Episodic: top_k, min_relevance, token budget, and weights for multi-signal fusion.
  • Persistence paths for episodic and semantic stores.
See Configuration and biocortex.memory.engine.

Next Steps