NowBuilding Voice AI agents at Swivl

Daniel Nguyen — Senior Full-Stack & AI Engineer

I build AI agents that pick up the phone — and the systems that keep them honest.

11+ years architecting high-scale SaaS and autonomous agent platforms — real-time voice pipelines answering under 500ms, agent runtimes for brands like SiriusXM and Sonos, and payment pipelines for 3M and Unilever.

live monitor/facility #0412
call 01/04
caller speaking
Caller

Agent

end-to-end
···msbudget 500
sentiment
0.50

Simulated calls · modelled on the Voice AI agent I built at Swivl

0+

Years shipping production software

0+

Facilities served by the Voice AI platform

<0ms

End-to-end p95 voice latency

0%

Fewer live-agent handoffs, QoQ

0+

Storage units kept in real-time sync

0+

Enterprise clients on payment pipelines

01Selected systems

Interactive tours of things I've shipped

Not screenshots — small, working models of production systems I designed and built. Data is simulated; the architecture is real.

A phone call, answered in under 500ms

The end-to-end path I own at Swivl — from Twilio audio frames to an LLM that can take a payment mid-call. Toggle the semantic cache to see why 35% of calls never touch inference.

Latency budget · illustrative turn485 / 500ms
0msfull inference path500ms
Redis semantic cache

Hashes the caller-intent embedding and returns stored answers for high-frequency questions — bypassing inference on ~35% of inbound calls and shaving 600ms off p95.

02Experience

A decade of systems that run in production

From ITSM platforms and enterprise receivables to conversational agents and real-time voice — the throughline is reliability at scale.

Jan 2025 — Present

Sr. Full Stack & AI Engineer @ Swivl

Led development of Swivl's real-time Voice AI Agent and its supporting infrastructure for a self-storage conversational AI platform. Owned the voice pipeline end-to-end — from Twilio telephony ingestion to LLM response — built bi-directional FMS integrations, and stood up the AWS infrastructure and CI/CD serving 4,500+ facilities.

<500ms
p95 end-to-end latency
−40%
live-agent routing QoQ
35%
calls served from cache
  • Architected the Voice AI Agent in Python/FastAPI: ingests Twilio Media Streams as base64 μ-law audio, runs Whisper STT and feeds transcripts into an LLM loop with barge-in handling — cutting live-agent routing 40% QoQ at sub-500ms end-to-end latency.
  • Designed a Redis semantic caching layer that hashes caller-intent embeddings and serves stored answers for high-frequency questions (gate codes, balances, office hours), bypassing inference on 35% of inbound calls and shaving 600ms off p95.
  • Built the intent classification engine with LLM function calling, routing utterances to autonomous actions (payments, access codes, autopay), knowledge-base retrieval, or live-agent escalation with full context summaries.
  • Assembled the pgvector RAG system that chunks and indexes facility FAQs, storage policies and unit pricing — grounding every answer in each property's actual data.
  • Shipped the Next.js Live Monitor: React Query + SSE stream real-time transcripts, LLM sentiment scores flag negative tonal shifts, and managers toggle which intents the AI handles vs. escalates.
PythonFastAPIWebSocketsNext.jsReactTypeScriptTailwind CSSReact QueryRedisPostgreSQLpgvectorCeleryOpenAI APITwilio Media StreamsWhisperStripe ConnectSSEOAuth2/JWTDockerAWSTerraformGitHub ActionsPytest
03Capabilities

Full-stack depth, AI-native instincts

Equally at home tuning HNSW indexes, designing Kafka topologies, or making a 100k-row grid scroll like butter.

Voice & Conversational AI

Telephony-to-LLM pipelines with barge-in, streaming STT, semantic caching and live escalation.

Agent Infrastructure

ReAct runtimes, tool registries, multi-model routing, RAG and eval gates that block regressions.

Distributed Systems

Event-driven microservices on Kafka, distributed locking, idempotent payment flows at scale.

Frontend at Scale

Next.js & React dashboards streaming over SSE, virtualized grids rendering 100k+ rows smoothly.

Hover a skill to see where I used itSwivlSierraHighRadiusBMC Software
04Contact

Let's build something that answers on the first ring.

Working on a hard problem in AI platforms, voice agents or high-scale product engineering? I'd like to hear about it.

Connect on LinkedIn
Houston, TX
(325) 222-9022
B.S. Computer Science
University of Houston · 2012 — 2015
Full résumé
Download PDF