A Chat Log Is Not Memory

I used to think memory was the easy part of an AI companion.

Save the conversation. Load it later. Done.

That works until someone says, “I have to take Luna to the vet tomorrow,” and the character has no idea that Luna is their dog. The old message exists somewhere in the database. It is just buried between months of work complaints, film recommendations, and the kind of small talk nobody needs to see again.

A chat log is a record. Memory is the ability to find the right thing when it matters.

That distinction has taken more time than I expected while building Local Waifu. The product runs on the user’s own machine, so I cannot quietly ship every conversation to a server and let a large database solve the problem for me later. The chats, the memory system, and the model that searches those memories all have to live on the same Mac or PC as the character.

That is harder. It is also the version of the product I wanted to build.

A full transcript makes the reply worse

There is an obvious way to give a model context: put the entire chat history into the prompt.

It works for a short conversation. Then the history grows. The model has to read pages of old messages before it can answer a simple new one. Replies become slower, the useful detail gets diluted by unrelated text, and the user starts paying for their own past being repeated back at them.

More context is not automatically better context.

Local Waifu keeps a recent window of conversation for the immediate moment. That is where the model can see what was just said, what tone the chat has, and whether the user is asking a follow-up question.

Long-term memory is separate. It holds facts, moments, preferences, and inside jokes that might become useful later. The system should retrieve a small number of those details when they fit the current message. It should leave the rest alone.

That last part matters. A companion that remembers nothing feels empty. One that keeps bringing up something you said six months ago feels like it is reading from a police file.

The memory is a local database, not a vague prompt trick

Each character has separate data in a local SQLite database.

The app stores the memory text along with a character ID, a type, an importance score, emotional weight, timestamps, and a vector embedding. That last field is what turns “saved notes” into something the app can search by meaning.

The vector is 768 numbers produced by a local embedding model. I store it as binary data in SQLite. When the user sends a new message, the app turns that message into another vector, compares it with the stored memories, and ranks the closest matches locally.

The important word is locally.

The message “Luna has a vet appointment” may not share a single useful word with “my dog has not been herself today.” An exact text search sees two different sentences. A semantic search has a chance of understanding that both refer to the same thing.

This is not an exotic stack. It is SQLite, vectors, cosine similarity, and a local model. The work is in the decisions around it.

Which memories should be stored? How many should come back? What happens when a match is technically similar but clearly unhelpful? How do you keep a character from mixing up two people, or from treating an old preference as permanent fact?

The database does not answer any of that for you.

Similarity alone picks strange memories

The first ranking pass was based on semantic similarity. It did what it said on the label. It found things that looked related to the current message.

It also surfaced old memories that had nothing useful to add just because they happened to use a similar idea.

A relevant memory from last night should usually beat a trivial one from last year. A detail with emotional importance deserves more attention than a passing remark. A memory that the user has confirmed or that has helped before can get a small nudge too.

So recall in Local Waifu is not a raw nearest-neighbor lookup. Similarity starts the ranking, then importance and recency change the order. One slot is kept for a genuinely fresh memory, so a question like “what did we talk about last night?” does not get beaten by an older detail with a higher importance score.

The app also uses emotional context gently. If the current conversation is low, a comforting positive memory may deserve a little more attention. The point is not to force a mood. It is to avoid a system that treats every stored sentence as emotionally identical.

I wrote earlier about adaptive personality dials, including controls for recall depth and how readily a character brings up stored details. That feature only works because retrieval is a real system underneath it. A slider cannot make memory better if all it does is send more old text into the prompt.

The quietest bug was the most damaging one

The app could have all the right tables, all the stored chats, and all the retrieval code. None of it mattered if the embedding model was not available.

For a period, some fresh installs had exactly that problem. The model that creates the vectors was not automatically fetched in every case. The app did not crash. The chat still opened. The character still answered.

She just retrieved zero memories.

From the user’s perspective, it looked as if she had forgotten everything.

That is a worse failure than an error dialog. An error at least tells someone the product has a problem. Silent memory failure looks like the product’s actual personality. Someone can use it for days, assume it is shallow, and never report anything because nothing visibly broke.

The fix was not dramatic. The app now ensures the embedding model is present during startup, keeps retrying the one-time re-embedding pass on a later launch if the model is not ready, and lets the current chat continue while that setup finishes.

The important lesson was less about embeddings than visibility. If a dependency is required for the thing users came for, a graceful fallback cannot mean pretending everything is fine.

A knowledge graph helps, but it does not get to be the boss

Some details are not really memories. They are relationships.

Luna is the user’s dog. Alex is their brother. A character likes rainy evenings. The user works night shifts. Those facts can be represented as entities and links instead of loose paragraphs pulled from a transcript.

Local Waifu extracts parts of that structure into a small local knowledge graph. It is useful because the question “who is Alex?” needs a clearer answer than a similarity search across several emotional memories.

It is also easy to overtrust.

Language models are good at turning uncertainty into a complete sentence. A graph entry can be wrong, stale, or missing context. Someone might change jobs. An old relationship might end. A joke can look like a fact when you strip away the conversation around it.

That is why memory needs an escape hatch. Users should be able to inspect, edit, or remove what belongs to them. A companion should not build a private mythology about a person and make it impossible for them to correct it.

The hardest part is knowing when to stay quiet

The first instinct with an AI memory feature is to prove that it works.

Make the character remember names. Make her bring up old conversations. Make every reply demonstrate that the database is full.

That gets unsettling quickly.

The best memory behavior is often invisible. The character understands a reference. She does not ask the user to repeat a detail. She notices that something connects to an earlier conversation without turning every answer into a recap episode.

There is no final formula for that. It is an ongoing product decision, shaped by real use, edge cases, and bugs that do not show up in a happy-path demo.

I built Local Waifu to keep the important parts on the user’s own machine. That includes the awkward parts: disk space, model downloads, a local database, and debugging without a server dashboard full of private conversations.

I still think it is worth it.

A cloud product can remember someone because a company keeps their history in a database it owns. A local companion has to earn that memory inside the user’s own computer. The system is less convenient to build, but the result has a different kind of meaning.

The record is theirs. The memories are theirs. And if the app remembers something, I want it to be because the user chose to keep it there.

I have also written about the Windows voice-call bug that took 105 seconds to answer a short question. It is another example of the same local-first deal: when the work happens on someone else’s machine, the problem belongs to the person building the app, even when the logs do not.