Pip is a little octopus who lives in a tiny tide pool on my desktop. You can talk to it and it talks back. It also watches you: smile and hearts pop up; frown and it frowns too; nod and it nods; blink and it blinks back. The water in its pool rises and falls with your breath, and it glows with your heartbeat.
When you chat with it, it doesn’t only hear what you say; it knows how you looked while saying it. Once I said “no” with a small smile, and it answered: “Funny thing is, you’re smiling a little while you say that.”
How it started
On the morning of November 11 I saw this challenge on DEV: build something with Presage’s SmartSpectra SDK that responds to the person in front of the camera. My first thought was a desktop pet that reacts when you look at it.
It went through several versions. First a companion for work time that reminds you to rest before you get tired, and a tide that rises and falls with your heartbeat and breath for a calm minute; then a few-minute game of making faces. The part that was most fun was making faces and talking to it, and seeing how it responds. So it became this: a talking desktop pet with eyes for the person.
Why only this part
Pip can’t read the page you’re on or use your computer. Agents like that already exist; I didn’t need to build another whole harness that I’d never use. I built the one piece I hadn’t seen: letting desktop AI read the person in front of it.
The octopus is just one look. A fish, a cat, a little robot would all get the same new senses (your expressions, heartbeat and breathing) and respond in their own way. Codex already has desktop pets that react to what the agent is doing; with these eyes, a pet could also notice when you frown at the code.
What I learned
I never thought an ordinary webcam could see someone’s heartbeat. Each heartbeat changes the color of the skin on your face very slightly, too little for the eye but enough for a camera; breathing comes from the rise and fall of the chest. The technique is called remote photoplethysmography (rPPG) and has been studied for more than a decade. That’s why it wants you to sit still in good light, with your shoulders and chest in view for breathing. The vendor is clear that the numbers are for general reference only, not for diagnosis.
How it was made
From the first idea to the recorded demo and the published post, it took one morning. Claude Code wrote all the code; I came up with the ideas, played with it, and said what wasn’t right. As I put it at the time: everything was done by AI, I barely touched anything, I just talked, and never looked at the code. A hackathon project now takes a few hours; what matters more than writing code is what you want to make.