You talk to your inbox. An ElevenLabs agent picks one of seven authenticated tools, runs it, and every action streams back to the browser live. The whole thing runs as a container on Cloud Run. Here is how I built it.
Want the live developer build? It runs against a developer Gmail build (Google OAuth still in testing, so access is manually allow-listed). Email me and I'll add you.
7
voice tools
3
realtime sinks per event
13
Prisma data models
Level 0 · System Context
Browser
React 19 live feed
ElevenLabs agent
STT · reasoning · TTS
Next.js API
App Router · Cloud Run (us-east1)
Gmail API
gmail.modify
Google Pub/Sub
new-mail push
Upstash Redis
Streams + Pub/Sub
PostgreSQL
Supabase · Prisma
Zoom 1 · Voice Tool-Calling
User speech
STT
Agent reasoning
Authenticated HTTP
Next.js route
Gmail
TTS reply
x-user-idx-email-account-idx-api-key→ missing or wrong key returns 401The 7 tools the agent can call
search_emails
GET /api/google/threads/voice-summary
read_email
GET /api/voice/read
draft_email
POST /api/voice/draft
send_email
POST /api/voice/send
archive_emails
POST /api/voice/archive
unsubscribe
POST /api/voice/unsubscribe
get_inbox_metrics
GET /api/voice/metrics
Zoom 2 · Real-Time Activity Fan-out
emitVoiceActivity() call. It writes to an in-memory emitter, a Redis Stream, and Redis Pub/Sub, persists to Postgres, and the result reaches the browser through the Upstash Realtime client over SSE, with no database polling.emitVoiceActivity()
utils/voice-activity-emitter.ts
In-memory emitter
same-instance SSE
Redis Stream
XADD · event history
Redis Pub/Sub
PUBLISH · cross-instance
Postgres persist
VoiceActivity history
Realtime client
useRealtime · SSE
Live feed
React activity stream
Why Redis primitives directly
@upstash/realtime v1.0.2 required Zod 4, but the app is on Zod 3. So I call XADD and PUBLISH directly, mirroring the SDK's internals without the version conflict.
Works across instances
Cloud Run instances come and go. Because delivery rides Redis Pub/Sub instead of in-process state, an event emitted on one instance reaches a browser connected to another, with no polling loop.
Zoom 3 · Gmail Sync & Ingress
lastSyncedHistoryId. A Cloud Scheduler job renews the Gmail watch before its ~7-day expiry.New email
arrives in inbox
Gmail
watch active
Google Pub/Sub
push notification
Webhook
process history
historyId
advanced
Cloud Scheduler
watch renewal job
OAuth2 restricted scopes
gmail.modify is a Google-restricted scope (read, label, archive, draft, send), so it requires Google's verification and CASA review.
utils/gmail/scopes.tsAES-256-GCM token storage
Access/refresh tokens are encrypted at rest with a key derived from server secrets, never stored in plaintext.
utils/encryption.tsDual auth modes
NextAuth sessions for the web app; signed header + INTERNAL_API_KEY for ElevenLabs tool calls.
utils/middleware.tsZoom 4 · Deployment Pipeline
GitHub
push to main
Docker
build baked in
Cloud Build
builds the image
Artifact Registry
stores the image
Cloud Run
us-east1
Cloud Run runtime
Observability
PostHog captures product analytics, session replay, and exception tracking, wired in at build time so the client bundle reports from the first paint.
Data Model
VoiceActivity history.Relationships shown: User → Account → EmailAccount, and EmailAccount fans out to Groups, Concepts (many-to-many via ConceptGroup), and VoiceActivity.
Engineering Decisions & Tradeoffs
Nothing touches your inbox unless you ask for it out loud. There are no rules quietly auto-sorting or auto-replying behind your back. gmail.modify is a broad scope, so I wanted every change to trace back to a sentence you actually said, not a rule that fired while you were asleep.
@upstash/realtime 1.0.2 needs Zod 4, and the app runs on Zod 3. Bumping Zod across the whole codebase just to get one realtime helper was not worth the blast radius, so I called the same XADD and PUBLISH commands the SDK uses internally. A bit more code, but nothing else had to move and validation stayed on one Zod version.
The whole app ships as a Docker image to Cloud Run. The Next.js production build runs when the image is built, so a container that starts up serves traffic right away instead of compiling first. Migrations run once on startup, before the server takes any requests.
gmail.modify is a restricted scope. Shipping it to the public means clearing Google's OAuth verification and a CASA security assessment first. Until that is done, the app runs as a developer build with a hand-kept list of allowed test users. That is why this is a case study with a request-access flow instead of an open sign-up.
The provider layer is abstracted, with two implementations behind it. Gmail is the primary path and the most exercised. Outlook also works, built on Microsoft Graph: the same voice tools (read, draft, send, archive) switch to Graph's Mail.ReadWrite and Mail.Send when the signed-in account is Microsoft, and consumer Outlook accounts can self-consent to those scopes without the restricted-scope verification Gmail's gmail.modify forces. The real next step is not another provider, it is clearing Google's verification so this can open past the allow-list.
Want the live developer build?
Inboxa AI runs against a developer Gmail build. Google OAuth is still in testing, so access is manually allow-listed. Request a spot, or reach out directly and I'll add you.