Memory Ep. 2: feeding the brain back

In the first memory episode I described how I stopped fighting Claude’s amnesia by giving it a shared brain: an Obsidian vault that both of us read and write to, so the work accumulates instead of evaporating at the end of every session.

That post ended with a loose thread. The brain only grows if someone feeds it. And for a while, that someone was me, by hand.

The recap skill

My first solution was a skill I called /recap. At the end of a session I would run it, and Claude would go back over everything we had discussed, pull out what was worth keeping, and write it into the vault as a dated note. Decisions, conclusions, things to follow up on. A diary of the work, written for me.

It worked really well the times I actually used it. That last part is the problem.

Within a few days two cracks showed up.

The first one is embarrassing and obvious: I kept forgetting to run it. The whole point was to capture things so I would not lose them, and the capture itself depended on me remembering to trigger it. When I was deep in something, the last thing on my mind at 1am was “let me close this properly and recap.” I just closed the terminal.

The second one is subtler. I assumed work happened in neat sessions with a clear end. It does not. When I am building something over several days, I tend to keep reusing the same session. The context is already warm, so why throw it away. But that means “the end of the session” never really arrives. There is no natural moment to run /recap. The good stuff just keeps piling up in a conversation that never closes.

And there is a worse version of that problem. Long sessions get compacted: when the context window fills up, the older messages are summarized and squeezed down to make room, and detail is lost in the process. So even if I did remember to recap a marathon session, it might be too late. A recap can only work with what is still in context, and compaction has already quietly eaten part of it. The interesting bit I wanted to keep was flattened into a one-line summary hours ago.

So the manual approach was broken on both ends. I needed the feeding to happen on its own, whether I remembered or not, whether the session ended or not.

capture.py

The first cron job runs every ten minutes. Its only job is to notice what is new and write it down.

It walks through my Claude Code conversation logs and, for each one, reads only the part that has appeared since the last time it looked. It keeps a small state file with a byte offset per conversation, so it never re-reads what it already processed. If nothing new showed up, or the new chunk is tiny, it exits and costs me nothing.

When there is enough new material, it hands the fresh slice of conversation to a model along with today’s note as it stands so far, and the message it sends is narrow on purpose: “extract only what is worth remembering and is not already in today’s note: decisions, ideas, open threads, context. Discard command output, the minor technical back and forth, system noise. If everything relevant is already written down, reply with a single word and stop.”

Whatever comes back gets appended to the daily note under a timestamped heading. Over a day, the note fills up on its own with little blocks like “automatic capture, 21:10.”

A detail I like, and that surprised people when I mention it: this does not call the API directly. It shells out to Claude Code in headless mode and pipes the prompt in. The same tool I use interactively is doing the summarizing in the background. There is a lock so two runs never overlap, and a hard timeout so a stuck call cannot wedge the whole thing.

A real daily note. Each block is one run of capture.py, stamped with the time it ran. I did not type a word of this.

The result is that the daily note is no longer something I write. It is something that gets written, and I read it later to see what the week actually contained.

consolidator.py

Capture is cheap and a bit dumb on purpose. It produces a running log, not a tidy knowledge base. A daily note full of timestamped bullets is a better diary than my memory, but it is not the linked, structured vault I actually want to think with.

That is the second job, and it runs every three days.

It reads the last few days of captures and the message it sends sets the bar high: “only propose concepts that appear at least twice, or that are clearly persistent entities: a person, a tool I am building, a recurring place. Ignore generic terms and one-off mentions. For each one, draft a note ready to paste, with a suggested location in the vault.” So instead of one-off mentions it looks for things that come up again and again, and for each candidate it drafts a real note, with the right structure and a place to live, ready to drop in.

And then it stops. It does not touch the vault.

Instead it writes everything into a single review file as a list of proposals, each one with a checkbox. I open it when I feel like it, read the drafts, and tick accept or reject on each. Nothing becomes a permanent note until I say so. The next time the job runs, it reads my decisions first: the accepted ones get created as notes, the rejected ones are dropped, and anything I have not decided on yet gets carried forward to the next round so it does not get lost.

One proposal from the review file: the draft note the consolidator wrote, with an accept or reject checkbox at the bottom. Nothing reaches the vault until I tick it.

This is the part I am most deliberate about. It would be easy to let the model write straight into the vault and watch it fill up by itself. I do not want that. The capture layer can be noisy and automatic because it only feeds a daily log. The consolidation layer, the one that creates the notes I will actually rely on for months, keeps a human gate. Me. The model proposes, I decide.

Where this leaves me

So the loop closed. Conversations flow into daily notes without me lifting a finger. Every few days those notes get distilled into proposed notes. I spend two minutes ticking boxes, and the vault grows with things I approved rather than things a model decided were important.

It is not perfect. Capture still grabs noise sometimes, and the consolidator occasionally proposes something I would never keep. But the failure mode is harmless: the worst case is a slightly messy daily note and a proposal I reject in one click. Nothing bad reaches the part that matters.

The interesting shift, for me, is who does what now. The boring, relentless part, paying attention and writing things down, is automatic. The judgment, deciding what deserves to be remembered for real, stays with me. That split feels right, and it is the thread I want to pull on in the next one.

But this is nowhere near the end of it. I find this whole problem genuinely fascinating, and I already have a handful of ideas in mind for where to take it next.