01The idea in one page
Three bets, taken from todo.md, decide whether this project lives:
judgment (does it ping about the right things?), memory extraction quality, and boring
reliability. Architecture is deliberately the least interesting part.
One shape for everything
A WhatsApp message, a new email, a calendar reminder and a 07:30 timer are all
Signal{ source, kind, payload }. One bus, one handler table, one audit trail
of what woke the agent up.
Two speeds, not one
Instinct (Haiku-class) always replies in under three seconds. Thinker (Opus-class) runs as a background job with tools and full context, then texts follow-ups in pieces — and may correct what Instinct said.
Postgres does the plumbing
Queue (SKIP LOCKED), wake-ups (LISTEN/NOTIFY), scheduling (a
deliver_after column), memory search (full-text). No broker, no scheduler, no
vector store.
Silence is a feature
Most email deserves no ping. An agent that pings too much gets muted, and a muted agent is dead. Ingestion never sleeps — only delivery is scheduled.
02The box it lives in
Two containers, one persistent volume each. The agent's home directory
(/home/agent) survives restarts and holds the Google OAuth credentials plus the
Gmail poll cursor. Postgres holds everything else.
03The signal bus — the nervous system
src/bus.ts is 200 lines and it is the whole queue. Publishing a signal is a
single INSERT INTO signals — from the app, or from psql, it makes
no difference. A database trigger turns that insert into a pending job; a second trigger
fires pg_notify, which wakes the worker loop instantly instead of waiting for
the next poll.
running, and boot resets those to pending before the loop starts.
Two job lanes
Jobs registered with { background: true } — the Thinker, memory extraction,
email pings — run detached so a 20-second think never blocks the next instinct reply. Everything
else runs inline, one at a time. That single flag is the whole concurrency model.
Timers instead of a scheduler
A 15-second interval checks a small array of timers and emits ordinary signals:
timer.tick hourly (a liveness probe that exercises the entire pipeline),
timer.morning-review at 07:30, and timer.google-poll every minute,
registered by the Google module itself via addTimer(). Same bus, no cron, no second
process.
ponytail: comment naming the fix
(a next_run column) if it ever bites.
04The two-speed brain
This is the differentiator. respond() always runs Instinct and
maybe enqueues a Thinker job. Instinct returns
{ reply, needsDeepThought } and its reply goes out with no artificial delay —
that is the sub-three-second acknowledgement. The Thinker then works with the full thread,
retrieved memories and tools, and texts follow-ups in pieces.
Interruption is handled twice
A newer user message makes an in-flight deep thought stale, and the code checks that on both
sides: pending thinker jobs for the thread are cancelled at enqueue time, and a
running think re-checks staleness before every single piece it sends. You never get a
follow-up to a question you already moved on from.
Instinct cannot act — by construction
Instinct is given no tools at all. Its prompt is hardened to match: any ask to do, record or change something gets a quick "on it" and a deep-thought flag, never a claim that it is done. The first eval run caught it claiming it had marked a loop done; that is why the rule is explicit now.
// src/brain.ts — the orchestrator, minus the plumbing const tail = await history(source, threadId, 12); const raw = await llm("instinct", tail, { system: INSTINCT_SYSTEM(await worldState()) }); const { reply, needsDeepThought } = parseJson(raw, { reply: raw, needsDeepThought: false }); await ch.send(threadId, reply); // no delay — this is the ack await recordMessage(source, threadId, "veda", reply); if (needsDeepThought) await enqueueJob("thinker", { source, threadId, userMessageId });
Model layer and the money cap
src/llm.ts is one function over the AI SDK. The provider is chosen by whichever
key is in .env (Anthropic wins over OpenRouter). Every call writes a row to
llm_usage with token counts and a computed cost; before every call the day's
spend is summed and compared to DAILY_BUDGET_USD (default $5). At the cap it
alerts the owner once and throws — an always-on Opus reading newsletters is a real way to
lose money quietly.
05Relevance — deciding when to stay quiet
Email arrives all day. One cheap classifier call maps each one onto a five-word enum, and the
enum decides delivery. Ties are broken toward silence: unsure between notify-now and hold,
pick hold; unsure between log and ignore, pick ignore. An unparseable answer becomes
hold — no spam ping, but nothing dropped either.
The morning brief
timer.morning-review hands the Thinker everything held overnight plus the world
state, and asks for one to three short messages: a line on the night, then what landed,
today's calendar and what it is still holding. When nothing was held it says so — "Quiet one
so far, nothing has landed." Delivered rows are stamped so they never repeat.
Calendar reminders skip the model entirely
An event starting within fifteen minutes emits event-soon, and the handler
formats a plain string: heads up — Standup at 10:00. No LLM call, because there
is no judgment to make.
06World state and open loops
Both halves of the brain get the same short text block, rebuilt on every turn by
worldState(): the current time, today's calendar, the open loops Veda is holding
(with their ids), and a digest of the last 24 hours of signals. If the calendar call fails the
brain still works — the block just says the calendar is not connected.
// what gets injected into both prompts
Now: Sat Aug 23 2026 09:14:02 GMT+0530
Today's calendar: Standup (10:00); Bill sorting (18:30)
Open loops you're holding:
#12 [task] send the zeppelin invoice to Acme — Fairooza is waiting on it
#14 [approval] gmail_send — waiting for the owner's go
Signals last 24h: gmail/email ×23, dev/message ×6, calendar/event-soon ×2
An open loop is a thing Veda is holding for its owner: a reply owed, a task, something it is
waiting on — or, since §08, a pending tool approval. The Thinker gets three tools
(open_loop_create, open_loop_complete, open_loop_list),
which is what makes "mark it as done" simply work. Only status = 'open' rows
appear in the block, so completing a loop is exactly what makes it never get mentioned again.
07Durable memory
Extraction runs as a background job after a Thinker turn — never inline in the reply path, and never as a raw-conversation-to-vector dump. One cheap call sees the last ten turns and a block of existing related memories, then returns operations: create, or supersede an existing memory by id.
Retrieval is Postgres full-text search over content || subject, plus an entity
match — if a known entity name appears in the query, memories about it come back too. Results
are ranked, recency breaks ties, and the top eight go into the Thinker's context as a compact
block with ids, so you can always see what it was recalling.
memory.ts says to add it when the eval suite catches FTS misses, not before.
08Acting on the world, behind a gate
Tools live in a Map, not a framework. Each carries a permission —
ALLOW, ASK or DENY — and gatedTools()
hands the model only the wrapped execute, so bypassing the gate is not a prompt
question. Reads are ALLOW; gmail_draft, gmail_send and
calendar_respond are ASK. Every call, whatever the outcome, writes a row to
audit — written by the runtime, not by the model.
Tool errors come back to the model as data ({ error: "…" }) rather than crashing
the turn, so a failed send becomes something Veda can tell you about instead of a silent
dropped job.
09The data model
Ten tables, defined in TypeScript with Drizzle and applied as generated SQL migrations at
boot. Raw sql is used where an ORM would fight the design: the
SKIP LOCKED claim, the full-text index, and LISTEN/NOTIFY on the raw
driver client.
| Table | Holds | Why it exists |
|---|---|---|
| signals | source, kind, payload, received_at | Append-only record of everything that ever woke the agent. Doubles as the audit trail and the "last 24h" digest. |
| jobs | kind, payload, status, run_at, attempts, last_error | The entire queue. Claimed with FOR UPDATE SKIP LOCKED; retries and backoff are two columns. |
| messages | channel, thread_id, role, content | Conversation history — what both models read, and what the staleness check queries. |
| memories | type, content, subject, confidence, status, supersedes | Long-term memory. Six types; corrections supersede instead of deleting. |
| entities | name (unique), kind | People, orgs, places, things — upserted from a memory's subject, used to widen retrieval. |
| relationships | from, to, relation | Exists per the plan; nothing writes it yet, and the schema comment says so. |
| open_loops | title, kind, context, data, status | What Veda is holding. kind='approval' reuses the same table for pending tool calls. |
| pending_notifications | summary, deliver_after, delivered_at | Held pings. One timestamp column is the whole delivery scheduler. |
| audit | tool, args, decision, result | Every tool call and its permission decision, written by the gate. |
| llm_usage | model, input/output tokens, cost_usd | One row per model call; today's sum is the hard budget stop. |
10Reliability — the unglamorous half
Expired OAuth tokens, dead containers and runaway token bills kill this kind of project long before any architecture question matters. Silent death is the failure mode, so the daemon is built to complain.
Watchdog
timer.tick fires hourly and exercises the whole pipeline. If nothing has been
processed for two hours, the owner gets told on any working channel.
Failing sources
Five consecutive failures from the same source — a revoked Gmail refresh token, say — raise an alert once, then reset the counter so it doesn't spam.
Graceful shutdown
SIGTERM stops claiming new work, finishes what's in flight, and leaves the rest
pending. tsx runs as PID 1 because pnpm swallows the signal.
Crash recovery
There is one worker, so any row still running at boot is an orphan. Boot
resets them to pending before the loop starts.
Cost ceiling
A daily USD cap, summed from llm_usage before each call. At the cap Veda
alerts once and goes quiet until tomorrow.
Cursor recovery
If the agent was down long enough for Gmail's history cursor to expire, the 404 is caught, the cursor resyncs to now, and the gap is skipped rather than crashing the poll.
Evals are the real deliverable
Five fixtures, each asserting against the database rather than against vibes. They need the daemon running on the dev compose overlay.
| Command | Asserts | Status |
|---|---|---|
| pnpm eval:two-speed | Instinct acks in under 3s, Thinker follows up under 60s | green — 2.7s / 11s |
| pnpm eval:relevance | Nine real-shaped emails land on acceptable triage decisions | grows with every misfire |
| pnpm eval:open-loops | "Mark the first as done" completes it in the DB and it is never mentioned again | green |
| pnpm eval:memory | A fact is recalled in a new session; a correction supersedes the old row | green |
| pnpm eval:tools | Send without approval is blocked and asks; "go" executes and audits | green, no Google creds needed |
11Running it
# dev: source mounted, tsx watch, ports exposed docker compose -f compose.dev.yml up # chat with it — or POST/SSE it from a script open http://localhost:8787 curl -N localhost:8787/events curl -d '{"text":"whats for today?"}' localhost:8787/message # connect Google once — writes /home/agent/google-oauth.json docker compose run --rm -p 8790:8790 agent pnpm auth:google # publish a signal from outside the app — the trigger does the rest psql $DATABASE_URL -c "INSERT INTO signals (source, kind, payload) VALUES ('gmail','email','{\"from\":\"a@b.c\",\"subject\":\"hi\",\"snippet\":\"…\"}')" # prod docker compose up -d
Where things live
| File | Responsibility |
|---|---|
| src/index.ts | Boot: connect, migrate, register handlers, start bus and channels, trap signals. |
| src/bus.ts | Emit, dispatch, worker loop, retries, timers, watchdog. |
| src/brain.ts | Instinct + Thinker prompts, the orchestrator, staleness checks. |
| src/llm.ts | Provider selection, model roles, pricing, the daily budget stop. |
| src/relevance.ts | Email triage, quiet hours, morning brief, event reminders. |
| src/world.ts | The world-state block and the open-loop tools. |
| src/memory.ts | Extraction job and retrieval. |
| src/tools.ts | Registry, permission gate, approvals, audit. |
| src/google.ts | OAuth refresh, Gmail polling, calendar reads, write actions. |
| src/channels/ | The Channel interface and the dev HTTP/SSE channel. |
12Where it stands
Milestones M0 through M7 are complete. What remains is mostly reach — a channel a phone can actually use — and the tuning period, which is the actual work rather than a chore after it.
INSERT is the publish API.