Part 1: It started with my son
I build AI systems for a living. One evening my son asked what that actually means, so I told him: I build machines that can think a little, use tools, have a conversation.
He went quiet for a second. Then he said, “You should build one for yourself.”
And before I could answer: “You should call it Ambi.”
“Why Ambi?”
“It’s a good name for an AI.”
That was the whole brief. And it stuck. Ambi. Ambient, always around. So I built one. Then I built it again. Then I built it a third time, and the third one is the only one I actually trust.
That gap is what this series is about. Not the assistant itself, but the three goes I had at it. Most of what gets written about building AI agents stops at the demo, the bit where it runs on your laptop and looks like magic. I want to talk about what happens after the demo. How a perfectly good agent rots. Why the obvious fix made things worse. And one feature I wanted badly, chased for months, and never got working properly.
Here’s the quick tour of the three, because the lessons all live in the differences between them.
The first ambi started as a Telegram bot wired to Claude Code on my Mac. I’d message it from my phone and it would drive Claude Code to actually get things done. Then I put a router in front of it — Gemini Flash — to triage whatever I sent: light questions went to a small Gemini agent, anything that meant real coding or deep research got handed off to Claude Code on the Mac. That split was the spine of the whole thing.
From there it just grew. Over a few months it picked up more than forty skills: calendar, email, finance, GitHub, meetings, research, home automation, even a little heads-up display. It could talk. It had a camera that noticed when I sat down at my desk. It could even write its own features. It was great to demo. The trouble was I didn’t trust a single part of it, and it never even made it into git. It was the experiment that quietly turned into the thing I relied on.
So I tried the obvious fix. I copied the whole project, called it ambi-lite, and started deleting. Cut the clever stuff, keep the basics, end up with something small and solid. It was still unstable. That threw me, and it turned into one of the more useful lessons of the whole thing: the problem was never how much it did.
The third time I stopped wiring other people’s agents together. That’s really what the first two were: a router in front of Claude Code and a Gemini agent, with ambi as the glue. ambi-core was the first time I sat down and asked what an agent actually is underneath all that. It’s a loop. Read the input, think, use a tool, answer, remember. I built that part myself and got it genuinely solid before anything else, then added the rest back slowly, one piece at a time, never letting a new piece break the part beneath it. That’s the one running on my machine now.
Too much, then too little, then about right.
My son asked for an AI called Ambi. What I built him, on the third try, turned out to be a long lesson in restraint.
Building these things is the easy part now. Trusting the ones you build is the hard part, and that’s what the rest of these posts are about.
Leave a Reply