My personal agent is a folder

Yesterday I was a bit tired and almost skipped the gym. Before heading out, I asked the AI offhand: “If I train back today, which exercises should I do?” It said that if I was tired I should go lighter, taking 20 to 30 percent off last time’s weights, then lined up four exercises, three sets each. With the plan in front of me, I actually wanted to go.

In the evening I told it, one by one, what I’d eaten at each meal and how much each cost, and finally asked: “How much did I spend today? Did I get enough protein?” It worked out how much protein I was still short, reminded me to have a scoop of protein powder, logged the spending in my ledger, and checked it against this month’s budget while it was at it.

I didn’t type a single word of this. I said it all into my phone’s voice input. Small things that used to take me ages to decide, I now say in a sentence and hand over.

What does these things for me is my personal agent. It isn’t an app, just a folder on my computer. I open it in Claude Code, and the agent works inside it.

What it looks like

Fitness is only one of its modules. Taking yesterday as an example, the whole folder looks roughly like this:

I described this diagram to it, and it drew it. The folder also opens directly in Obsidian: the AI writes in it, and I read in Obsidian. Files link to each other with wikilinks, so one click takes you to the source, and you can see everywhere a note is referenced.

I think four parts are useful to everyone:

• Daily records. Notes, journals, recordings, photos and conversations with AI, all filed by date, along with the books and articles I’ve read. This is the AI’s memory of me.
• Knowledge. I borrowed the idea of Luhmann’s slip box: ideas we’ve settled on after discussion go into the Wiki, one by one. What matters most is sources and indexes: every entry links back to where it came from, and indexes the AI generates tie them together.
• About me. What I’ve done, my personality, my experience and projects. When I write a résumé or introduce myself, this is where the material comes from.
• People around me. The AI doesn’t know what has happened between me and each person, and transcripts often get names wrong; one teacher’s name came out spelled two or three different ways. So I keep a file for each person: who they are, how their name is written, and what we’ve been through. There are over thirty so far.

Everything else is modules, added as needed. Everyone lives differently, so everyone needs different modules:

• Fitness. What I trained each time, the weights and sets, an exercise library, a training plan, and whether I’m getting enough protein each day.
• Writing. A topic list of what I want to write, am writing and have written; one folder per blog post; and a writing style guide that gains a rule each time I finish revising a piece.
• Games. If you play Genshin Impact, you could make a Genshin folder for character builds, team comps and resource planning, plus a guide noting which game version it’s for. Mine is for an auto-battler I play, with line-ups and counters.

When I need something new, I make a new folder and write its rules at the top of its main file. The fitness and spending modules came about a few days ago when I casually said “I’d like to log this every day”, and it built them on the spot.

The more I do, the more I notice tasks with fixed steps that keep coming back, so I distil them into skills: a write-up of the steps, sometimes with a script. The ones worth keeping show up while doing the work: this will come up again, and it’d be useful elsewhere too. There are seven now, such as converting recording transcripts to Markdown, importing ebooks, making Xiaohongshu cards, and pushing articles into my WeChat Official Account’s drafts. How to write Xiaohongshu copy could be distilled the same way later; writing it down is what keeps it consistent.

Starting from zero: moving the past in

For the folder to be useful, it needs enough material. But at first it was empty.

I didn’t write any scripts to move things in. I just gave the AI my Notion and flomo API keys, let it pull everything itself, and watched. Later I gave it access to the whole computer to dig through old records, and installed Claude Code on my old laptop from university too. Personal projects, little demos, my graduation project and lab reports: it turned them all up, one by one.

Once it finds things, it also files and sorts them into the right places. When it found which zip file held my graduation project’s code, it added that to my university experience while it was there.

My dated records now go back to June 2021, more than 1,200 days. That’s the foundation of how it knows me. Later I’d like to photograph my handwritten journals and bring them in too, to see how I’ve changed over the years.

What I use it for

Fitness and meals are only the most everyday uses. Over the past month I’ve also used it for plenty of other things:

• Job hunting. Revising my résumé, preparing for interviews, and reviewing afterwards which questions I didn’t answer well. It knows what I’ve done, and it can dig up experiences from old material that even I had forgotten.
• My blog. Publishing posts, switching frameworks, going live, adding analytics: I did all of it together with it.
• An information officer. It filters my information sources and picks out what’s worth reading. I wrote a whole post about that.¹
• Diagrams. I describe one and it draws it. In university, lab reports ran to dozens of pages, and whenever I needed a figure I had to switch tools and drag flowcharts together bit by bit in Visio. It tires me out just remembering it.
• Writing. It drafts and I edit. This post was written that way too.
• Understanding my feelings. Anxiety after an interview, going round in circles after a decision: I say it out loud, and it helps me take it apart bit by bit. I’ve also asked it to find things people have said in praise of me.

Reading and learning new fields deserve a few more words.

Reading

Reading has become a conversation. As I read, I tell the AI what I’m thinking: a line that struck me, a word I don’t know, a question the page raises. It annotates along with me, explains things, and fills in the background.

Sometimes it draws a map. When I was reading Paris from Bloom’s Literary Places, I didn’t know much about the city. It just drew me a map of Paris with the places from the book, so I didn’t have to go searching websites myself.

A book also leads somewhere else. Napoleon came up while I was reading about Paris, and I realised I didn’t really know what he had done. So I asked, and he went onto my list of things I want to understand.

All of these conversations are saved in the book’s own folder, next to the book. When I finish, we go through my notes a few at a time and build a mind map together, each node linking back to its place in the book. Reading used to end when I closed the book; now what I thought along the way stays.

Turning the vague into the definite

When I want to understand an unfamiliar field, I often feel I know nothing and can’t tell where I stand. Actually I know some of it; I just can’t put it into words. Once I’m afraid, I can’t break things down, and reason drops out first. AI happens to be good at structure. It splits the field into a few steps, I check myself against each one, and I can see how far I’ve come.

At my previous company I always felt I didn’t understand manufacturing. Then I had it break a factory into six parts: taking orders, design and process, planning, purchasing and warehousing, production, and quality control and shipping. Checking against them, I found I understood some parts and genuinely didn’t understand others. It wasn’t as scary as I’d thought. What drops you into worry all at once is often just the vagueness.

My thinking used to be scattered. It can find things scattered all over, piece them together, and then think with me about what new things they might add up to.

It also keeps a complete record of how something got done. I used to note down only the final conclusion; how I got there was mostly lost. Now, when I discuss things with AI, every step of how I reached a conclusion is recorded. This post went through several drafts, and every time, what I said and why I changed it is saved as a conversation.

Why a folder

Before this, I built several versions, and they left me two lessons:

• I stored memories but didn’t know what to do with them. At first I wrote my own scripts to store memories and built a little agent on top of models like Kimi and Zhipu. Later I gave my rawest journals to an AI and had it summarise what kind of person I am, hoping other agents could use the summary. I had memories and a summary, but there wasn’t much information, and I hadn’t figured out what to use them for. It felt hollow, with no loop closing back.
• Don’t build the tools yourself. After that I built my own agent with all sorts of tools; it could discuss books with me as I read, which was a step forward. But it wasn’t powerful enough, the interface wasn’t pretty, and I had to maintain the tools and the folder myself. In the end it was never as convenient as Claude Code or Codex.

After a few tries I realised that what matters most is what I actually use it for. So in this version the agent is just off-the-shelf Claude Code or Codex. I only look after the folder’s structure and put my attention on using it. The idea came from a talk by ciaoye.² Everything in the folder is plain text, so whatever model or tool I switch to, my material stays in my own hands.

I wrote just one manual for it, AGENTS.md. The three most important rules: keep original material exactly as it is, never rewritten; be clear about who is speaking, since the AI’s replies aren’t my views; and when unsure, write “to be confirmed”. Never guess dates, names or numbers.

What I’ve learned using it

My biggest takeaway after a while is to trust it, and hand over as much as I can of the things that take me ages to decide. I used to use a workout app, searching for exercises and tapping through them one by one every time. Now it’s more like having a coach: it makes the plan, I just train, and I keep an eye on details like not using momentum myself. Bookkeeping is the same. No need to build an app; I say one sentence in the chat, and it’s logged in Markdown and CSV. I learned this from Lan Xiaohuan, who gave AI his genetic and exercise data for health analysis.³

Logging my meals is something I could never manage before. In 2022 I saw a traditional Chinese medicine doctor, an elderly woman, who told me to write down my three meals every day. I found it a hassle: writing that much every day and then sorting it out. I didn’t want to do it at all. Looking back, she was right: only by writing down what you eat can you see how to adjust. My health is ultimately my own responsibility; a doctor can’t look after everything for me. Now one sentence logs it, and after a while I naturally get a sense of how to eat and live more healthily.

Handing things over doesn’t mean not caring at all.

What it pieces together is sometimes off: I did one thing and it recorded another. When I notice, I have it go back and fix it at the source. Every entry links to what I originally said, so the source can always be found. On the whole I can trust it with things, and I proofread the details. After all, only I know best what I’ve been through.

It also won’t know what I want on its own. It could only make that training plan after I told it how many times a week I train and which muscle groups to rotate. I have to think clearly first about how I want to use it, then see whether it can do it.

And plans can’t keep up with reality. That day at the start, the AI had my back workout neatly planned, and it talked me out the door. At the gym, every back machine was taken, so I trained shoulders instead. (Next time I could ask for a backup plan: which exercises to swap in when the machines are full.) However well it plans, it can’t know whether the machines are free.

That has to do with where it lives. My personal agent is a local folder on my Mac, not in the cloud. I can only use it at my computer; once I’m in the gym, I’m on my own. I think that’s fine. I plan with it at home and make my own calls on the spot. I don’t need AI with me every moment.

I’ve always had a picture in mind: Ineffa from Genshin Impact. She’s a robot Aino built, a kind of housekeeper who plans and organises Aino’s life, like splitting the budget into money for the workshop and money for snacks. In April this year I told the AI I wanted to build an assistant like her. In May it struck me that she wasn’t like that from the start either; she went through many generations of iteration and trial and error before becoming who she is. My personal agent is the same. The earlier versions were trial and error, and with this one it finally has the beginnings of a life assistant.

How would you use it?

My way isn’t necessarily right. As I see it, personal agents are still in an in-between stage. Everyone is experimenting, and that in itself is a lot of fun.

Products like Muse live in the cloud, connected to email, calendar and LinkedIn, usable anywhere, anytime. One writer took it on a trip to the US and got a meeting with someone they’d wanted to meet.⁴ But products like that mostly keep your data inside their own walls. If I want to dig out a few ideas from long ago and look at them together, or let a coding agent read them too, it isn’t easy. Mine only works at home, but its strength is that it remembers my own past, and it’s all plain text that any tool can read.

I can’t say what it will grow into. Seeing Muse, though, I’d like to learn from how it keeps tracking things, and add scheduled tasks to the folder so it can keep an eye on what I’ve been following lately. I hadn’t thought of that before; how exactly to do it, I’m still thinking about.