AI Agent Memory: MemPalace vs Karpathy's Knowledge Bank (And How to Choose)

Every AI agent system hits the same wall. The agent starts fresh every session. It doesn’t remember what it did yesterday, what decisions were made last week, or what’s happening across the other twelve projects it’s supposed to be aware of.

The obvious answer is a memory system. The less obvious question is what kind.

There are two schools of thought right now. Andrej Karpathy published his LLM Knowledge Bank in April 2026 - plain text files, an index, and an AI acting as librarian. The post got over 14 million views for a reason. On the other end, systems like MemPalace - built by Milla Jovovich and Ben Sigman - use a structured database, a connection map, and a spatial metaphor to organise and retrieve agent memory.

I use MemPalace. I’ve landed there for specific reasons, but Karpathy’s system solves a different problem and the two aren’t really competing.

Karpathy’s Knowledge Bank

The Knowledge Bank has three layers: raw sources (the original documents you feed in, kept unchanged), a wiki layer of AI-generated summary pages, and a rules document defining how things get organised. Two special files do the heavy lifting - an index that catalogues everything and a log that tracks what’s been added.

There’s no infrastructure to maintain. Any AI can read the files cold, and you can open them in any text editor or track changes like any other file. Karpathy grew one research topic to roughly 100 articles and 400,000 words without writing a single word himself. The AI did all the summarising, cross-referencing, and filing.

For single-topic deep dives, any AI model can pick up the files and start working. If you’re building a knowledge base around one domain or running a single-project agent setup, the files will outlive whatever tool you’re using today.

Why I use MemPalace

My work is multi-project and multi-client. Multiple agents running across different projects at the same time. That’s the reason I ended up on MemPalace rather than plain text files.

MemPalace organises memory using a spatial metaphor - wings, rooms, drawers, and tunnels. SEO knowledge lives in one room, client history in another, and tunnels connect related rooms across projects. Nobody using Claude Code is typing technical commands anyway - you tell it what you need in plain English and it handles the rest. But the spatial structure means the person directing the system can picture where things live, which I find easier to work with than remembering folder names and file conventions.

MemPalace also has a connection map - a structured record of how entities relate to each other across the whole system. And it has built-in tools that let agents store facts, file documents, and create connections as part of doing their job, without switching modes or remembering a format.

The infrastructure trade-off

MemPalace comes with infrastructure that plain text doesn’t. A database, a search engine, a server running in the background. You’re maintaining backend software so your AI agent can remember things.

For a solo operator running one project, that overhead probably doesn’t make sense. Karpathy’s system has no infrastructure at all - text files in a folder. But for multi-project agent systems where agents work across multiple projects at the same time, the infrastructure is there for a reason. For my setup, that trade-off made sense.

Getting memory to actually stick

No matter which system you pick, none of it matters if the memory doesn’t get written in the first place.

If saving to memory is a separate step from doing the actual work, it doesn’t get done. The capture needs to be a side effect of working. Automatic saves at the end of each session that update project status. Scripts that log what changed and why after each piece of work. Agent routines that write their findings back as part of completing the task.

MemPalace handles this through its built-in tools - agents can write to the system as part of their normal workflow. Karpathy’s system handles it differently, with the AI itself acting as the librarian and doing the filing. Both approaches try to make the write path as frictionless as possible.

The question I’d ask about any memory system: what mechanism ensures it stays populated without someone actively maintaining it?

How to choose

Karpathy’s Knowledge Bank fits if you’re doing deep research on one topic, running a single-project agent, or you want something that works with any AI model and has zero infrastructure. The files are portable, any model can pick them up cold, and there’s nothing to maintain.

MemPalace fits if you’re running multiple agents across multiple projects and you need the memory to work across project boundaries.

I use MemPalace because my work is multi-project and multi-client. If it were one project going deep, I’d probably be running Karpathy’s system.

What is the best memory system for AI agents?

It depends on what you’re doing. For single-topic research or single-project setups, Karpathy’s plain text Knowledge Bank with an AI librarian has zero infrastructure and the files work with any model. For multi-project agent systems, a structured system like MemPalace does things that are harder to replicate with plain text files. Either way, the thing I’d pay most attention to is whether the memory capture is automated - if it’s a separate step from doing the work, it won’t get done. I build memory capture into my agent workflows as a side effect.

Should I use a vector database for AI agent memory?

For a single project, probably not. Search across well-structured text files handles most retrieval needs and you skip the infrastructure overhead. For multi-project systems where you need search that understands meaning across thousands of documents, plus cross-project relationship tracking, a database starts to make more sense. Start without one and add it when basic search stops being enough.