Conversational Agent
voksa
A WhatsApp assistant that turns a text or voice note in English, Hindi or Hinglish into a confirmed cash-on-delivery food or grocery order.
- Agent tools
- 19
- Languages
- English · Hindi · Hinglish
- Cart cap
- ₹1000
- Payment
- Cash on delivery
Problem
Ordering food or groceries means opening an app, searching, comparing menus and building a cart by hand. Handing that to an agent is only useful if it can never spend money by mistake: a garbled voice note, a retried request or a stale cart must not turn into an unwanted order.
Approach
voksa runs locally and connects WhatsApp to Swiggy Food and Instamart through Swiggy's MCP servers, with Gemini planning each step in a tool-calling loop over 19 tools (10 for Food, 7 for Instamart, plus confirmation and delivery-address tools). Voice notes are transcribed in a separate call with no tool access, and a scope guard classifies every turn before any tool-capable call runs. The live cart is re-fetched from Swiggy on every turn instead of being cached, orders need an explicit numbered confirmation in the immediately preceding turn, food carts are capped at ₹1000, and because order placement is not idempotent a failed call is reconciled against the order history rather than retried. The code follows a ports-and-adapters layout, and a local MCP stub with failure injection keeps the test suite honest.
Highlights
- Money-safety by design: explicit confirmation step, ₹1000 cart cap, and reconciliation instead of retrying non-idempotent order calls
- Scope guard that classifies every turn before any tool-capable model call runs
- Stateless cart re-fetched from the source every turn, removing stale-state failures
- Ports-and-adapters architecture with a local MCP stub and failure injection for testing
Architecture
Select a layer to read why it is there.
Incoming messages are verified and checked against message IDs already seen before any processing, so a redelivered webhook cannot start a second turn.
Voice notes are transcribed first, in a call with no tools; the scope guard then decides whether the turn is in scope.
The model sits behind a port, so the planning loop can be tested against a fake.
Order placement is not idempotent, so a failed call is checked against order history instead of being retried.
Fastify serves the webhook; Postgres, Redis and BullMQ hold state and queued jobs.
Decisions & trade-offs
Reconcile, don't retry
Placing an order and checking out are not idempotent, so after a server error the agent reads back the order history before doing anything else. A blind retry could place the same order twice; reconciling costs an extra call and a little latency.
A scope guard before any tool call
Every turn is classified by a separate step before a tool-capable model call runs. Keeping the guard outside the system prompt makes it testable on its own, and refusals are logged only as a coarse category.
Transcription with no tool access
Voice notes are transcribed in a call that cannot use tools, and the transcript passes through the scope guard like typed text. Transcripts are held only in memory.
Never cache the cart
The cart is fetched fresh on every turn and again immediately before an order, where the ₹1000 cap is checked. That costs extra calls, but the agent never acts on a stale cart.
Impact
A working agent that places real cash-on-delivery orders from a single message, designed so that every rupee spent has passed an explicit confirmation and a server-side check.
Outcome
- About 370 tests across 37 test files, run against a local MCP stub that includes a mode where the server errors after the order was actually created.
- 18 bugs found in real use written up with cause and fix in a code log.
- A recorded lesson: the suite once passed while cash-on-delivery ordering was broken, because the stub accepted more than the real server. The stub is now held to never be more permissive than production.
- Built in twelve phases, each tied to a commit.