Lighthouse Memory Concepts Explained for Developers

Lighthouse Memory has a small API and a handful of ideas behind it. Learn the ideas once and every tutorial, MCP tool and config option makes sense.

The public surface is three verbs: remember, recall and forget. Underneath are about a dozen concepts that decide where memories live, how they are found, how long they last and how you get them back. This is the glossary we wish every developer read before writing their first line.

---

The Big Picture: One API, Three Swappable Layers

Lighthouse Memory is built as one API over swappable layers. You choose three things per deploy, usually with environment variables, and your application code does not change.

The SDK is a set of MIT licensed npm packages under @lighthouse-ai/*. core holds the interfaces and factory and has zero network dependencies; each engine, storage and embedder is its own package that self-registers when imported.

import '@lighthouse-ai/store-lighthouse'   // registers lh-ipfs-filecoin and lh-ipfs-walrus
import '@lighthouse-ai/embed-local'        // registers the 'local' embedder
import { createStorage, createEmbedder } from '@lighthouse-ai/core'
import { BatchedEngine } from '@lighthouse-ai/engine-batched'

const storage = await createStorage('lh-ipfs-filecoin', { apiKey: process.env.LIGHTHOUSE_API_KEY })
const embedder = await createEmbedder('local')
const memory = new BatchedEngine(storage, { namespace: 'demo', embedder })

Forget the side effect import and you get Unknown storage "…". Registered: [...]. That error message is the registry telling you what it knows.

---

Concept 1: Engines

An engine decides who embeds, who searches, and whether writes are batched.

Batched engine. The do it yourself engine. You embed locally, memories buffer, and they upload as one blob per flush, on any storage. Search is fully local.

Memwal engine. The relayer engine. A relayer embeds, SEAL-encrypts, uploads to Walrus and runs vector search. Every write goes straight to the network. You keep tags, CIDs and blob IDs locally.

Both implement the same MemoryEngine interface: remember, recall, list, get, forget, status, and snapshot methods. Batched adds flush and rebuild. Memwal adds analyze, verify, blobIds, restore and repinPending.

Concept 2: Storages

Storages decide where the batched engine puts bytes.

Every adapter speaks the same interface, so code written against local-fs runs unchanged on S3 or Filecoin.

Concept 3: Embedders

Embedders decide how batched recall ranks.

• local runs all-MiniLM-L6-v2 in process. A roughly 25 MB model downloads once, then everything is offline and no content leaves the machine.
• keyword is word and tag match only, with zero dependencies. Passing embedder: null is the same thing.

Memwal needs no embedder; the relayer embeds server side.

Concept 4: Hybrid Scoring

With the local embedder, recall blends meaning and words:

score = 0.7 × cosine(query, memory) + 0.3 × min(1, keywordScore)
keywordScore = hits / √words + 0.5 per tag appearing in the query

Recall returns score, semanticScore and keywordScore, so you can debug ranking. Ties break by recency. Vectors carry their model ID, and recall only compares same model vectors, so switching models safely falls back to keyword until memories are re-embedded.

Concept 5: Namespace and Agent

A namespace isolates memories per agent, project or team. Snapshots, rebuilds and pointers are all namespace checked; a snapshot for alice will not load into bob. The agent field is recorded on every memory so you can attribute records later.

Concept 6: Pending and Flush

In the batched engine, remember embeds the memory and adds it to a pending queue. Pending memories are searchable immediately but exist only on the local machine. A flush uploads all pending memories as one blob.

Flush happens automatically every flushEvery memories (default 10, minimum 1), or when you call flush(). The tradeoff is simple: flushEvery: 1 gives maximum durability; higher values give maximum quota efficiency, which is critical on Walrus.

Concept 7: Batch Blobs and CIDs

A flush writes a batch blob, a JSON file named mem-batch.<namespace>.<batchId>.json containing the records. Blobs are content addressed with CIDv1, raw codec, sha2-256: same content, same CID. Every memory in the batch is assigned that CID. Anyone can fetch it from a gateway and verify it. See What Is Verifiable Memory for AI Agents.

Concept 8: The Local Index

Recall is fast because it searches a local index in .memory-sdk/<namespace>.index.json, holding content, tags and vectors. The index is a cache. The source of truth is on the network. Writes to it are atomic, and a missing index simply starts empty.

Concept 9: Snapshots

snapshotIndex() pins the entire local index as one blob and returns one CID. rebuildLocal(cid) merges it into another engine. Merges skip IDs already present. Snapshots are the fast recovery path and the easiest way to hand a full context to another agent.

Concept 10: Rebuild

rebuild() (batched) re-reads every batch blob for the namespace from storage. It needs only storage credentials and no pointer, but gets slower as history grows. On memwal, restore() asks the relayer to repair its vector index from onchain data.

Concept 11: The Pointer Service

@lighthouse-ai/cloud-sync maps each user, namespace and network to the latest snapshot CID, with a pointer key of <namespace>#<network>. backupIndex() snapshots and saves the pointer; restoreIndex() fetches it and rebuilds. A new laptop gets your memory without anyone copying a CID. See Recover Agent Memory on Any Machine with Snapshots and Pointers.

Concept 12: Forget

forget() is honest about each backend. Pending memories are dropped locally. Flushed memories leave the index; the blob is deleted only when no other memory references it. On S3 and local disk that is a hard delete. On Lighthouse storage it stops renewal, so content stays readable until the period expires. On memwal the local copy and mirror are removed and the encrypted blob lapses on its own.

Concept 13: MCP

A hosted Model Context Protocol server at memory-api.lighthouse.storage/mcp exposes all of this as tools. The tool list adapts to the engine: common tools on both, memory_flush and memory_rebuild_index on batched, memory_analyze, memory_verify and friends on memwal. With no Lighthouse key saved yet, only three key tools appear, so an agent can onboard itself. See Give Claude Code Persistent Memory with Lighthouse MCP.

---

The Durability Model on One Page

---

Frequently Asked Questions

What are the three layers of Lighthouse Memory? Engine (batched or memwal), storage (Filecoin, Walrus, S3 or local disk) and embedder (local MiniLM or keyword). Storage and embedder apply to the batched engine.

What is the difference between the batched and memwal engines? Batched embeds and searches locally and uploads memories in batches to any storage. Memwal uses a relayer that embeds, SEAL-encrypts and searches, and writes every memory immediately to Walrus.

What does flushEvery do? It sets how many pending memories accumulate before the batched engine uploads them as one blob. The default is 10 and the minimum is 1.

Is the local index the source of truth? No. It is a cache for fast recall. Memories live on the network and can be rebuilt from blobs or a snapshot CID.

Is the Memory SDK open source? The @lighthouse-ai/* packages are published on npm under the MIT license.

---

Get Started

• SDK overview
• Core and factory
• Configuration reference
• Offline quick start, no keys needed

---

Stay in Touch

Learn more at the website, docs, or GitHub. Join the community on Discord, X, Telegram, and LinkedIn.