SavvySavvy

Overview

As a PhD student juggling classes, research, and a social life, I spent months trying to keep a strict calendar, and as a type-B person I kept failing. My plans changed constantly: work ran longer than expected, or a friend asked me to get lunch. I had the idea for Savvy six months before I built it, because I wasn’t sure I’d actually need it. Now I use it every day. If I remember something during class, I text Savvy “remind me to do this some time tomorrow.” When a new event comes up, I ask it to restructure my day. It also helps me with travel times: before it books something, it checks whether I actually have time to get there from wherever I’ll be. It even tries to find fifteen minutes a day for my long-term goals and personal projects. It isn’t perfect, and I still sometimes give it structure first, but the longer I use it, the better it gets. I named it after my childhood nickname, because I think of it as a mini-me.

Savvy is a self-hosted AI secretary. It connects Claude to Google Calendar, Gmail, Todoist, and a local diary, and works over two-way Signal messaging. All long-term memory stays on the local machine.

Keeping it private

A secretary is only useful if it knows a lot: commitments, routines, habits, and what the week actually looked like. That is exactly the kind of context that shouldn’t sit in someone else’s cloud. Savvy is designed so the full record stays at home. Conversation history, extracted facts, embeddings, diary entries, and OAuth tokens all live on local disk. Each turn, only the current message, a system prompt, and a small locally selected context window go to the API.

You ↔ Savvy ─┬─ Anthropic API (reasoning + tool use)
             ├─ Ollama (local embeddings)
             ├─ SQLite (conversation + fact storage)
             ├─ Google APIs (Calendar, Gmail, Routes)
             ├─ Todoist API (tasks, projects, labels)
             └─ Signal (signal-cli-rest-api in Podman or Docker)

The model reasons, but it never acts directly. The Anthropic API only sees tool definitions, and when it wants to act it returns a structured request. Local Python code decides whether to run it.

Capabilities

Savvy gives the model 40+ local tools.

Calendar. Create, update, move, and delete events across multiple Google accounts and sub-calendars, and check free/busy windows.

Email. Search Gmail, read threads, draft replies, manage labels, and archive. It sends mail only on explicit request; drafts are the default.

Todoist. Full read/write over tasks, projects, and labels, plus a review of what actually got done. Due dates are deliberately hidden. These projects are a backlog to pull from in spare time, not a second schedule.

Travel time. A local place book maps nicknames like “lab” or “the gym” to real addresses. Before booking, Savvy checks whether the gap between two events is long enough to get from one to the other. Routing uses the Google Routes API, because it covers public transit. Results are cached, and durations the user has confirmed always take precedence.

Diary and notes. Morning plans and evening reflections go into a local store that is pulled back into context on later turns.

Multi-account. Personal and work Google accounts can be linked at the same time. Each link is verified against the live account, so a token saved under the wrong label gets caught instead of silently writing to the wrong calendar.

Memory

Memory is the most developed part of the system. Each turn:

1. The message is embedded locally with Ollama (nomic-embed-text).
2. Savvy retrieves a hybrid set of past messages: the most recent ones plus the most semantically similar ones from history.
3. Stored facts (commitments, deadlines, preferences) are retrieved separately and added to the prompt.
4. After the reply, a second model call extracts new facts and writes them back to the local store.

Facts have a shelf life. Each one is labeled stable (goals and relationships, which never fade), situational (current projects, which fade over a few months), or ephemeral (moods and one-off days, which are gone within a month). Age acts as a bounded penalty on retrieval. An outdated situation sinks below current ones, but a clearly relevant fact still comes through. Recent short-lived facts like “trip Saturday” also appear every turn as a current situation block. Near-duplicates are merged on write, and the user can tell Savvy when something has changed.

Tool gating

Every tool call passes through a confirmation gate before the underlying function runs. Each tool is tagged with a danger level:

• Read: checking the calendar, searching email. Auto-approved.
• Internal: writes to the user’s own data, like creating an event. Auto-approved, but escalates to a human prompt when the model marks the call as high-importance, for example deleting a recurring meeting or touching a topic the user has flagged.
• External: anything that leaves the machine as communication (like sending email) or can’t be undone (like deleting a Todoist task). Always asks first.

A denied call returns "denied" to the model so it can try something else. Every call is appended to an audit log. The gate also matters for prompt injection: if an email contains adversarial instructions, the confirmation prompt is the backstop.

Everywhere it runs

Savvy runs on Linux, macOS, and Windows, and every interface shares the same memory and tools:

• Tray app. Savvy runs in the system tray with a chat window, and starts on login.
• Terminal REPL. Commands like /bod, /eod, /calendar, and /memory support morning and evening check-ins.
• Signal bot. It has its own number, so it can be texted from a phone with end-to-end encryption. Replies use a shorter style suited to texting.

An optional background scheduler sends proactive Signal messages for upcoming events and missed commitments, within quiet hours and a daily budget. Liveness alerts send a text when the bot comes up and when a service crashes. The crash hook sits outside the process, so it still fires after a hard kill. An interactive setup wizard covers the credentials, the Signal container, and the starter rules.

Personalization

Savvy is personalized in two places. The system prompt covers identity, goals, and communication style, which rarely change. A small rules file holds the operational quirks that change more often, like “never book over a standing weekly appointment” or “laundry takes a few hours and requires being home.” Keeping the two separate means day-to-day rules can change without touching the assistant’s character.

Motivation

Off-the-shelf assistants either lack the right context or send everything to a vendor’s cloud. Savvy has enough context to give specific answers while keeping the long-term record local. Building it meant answering three design questions: what context actually helps a model (recency, relevance, or structured facts that expire), where the trust boundary should sit between the model and real accounts, and how to keep behavior consistent across a tray app, a terminal, a phone, and a background process.