06 / 09Voice agents
Call AI Automation
Pipeline for AI phone agents
Places ElevenLabs outbound calls through Twilio, then extracts, scores and routes every conversation into Salesforce, Google Sheets and analytics.
Overview
The backend behind an AI calling operation. It places outbound calls with ElevenLabs voice agents over Twilio, receives post-call webhooks, uses Claude or OpenAI to extract variables and score each call against a rubric, and routes the results to Salesforce, Google Sheets and a call analytics dashboard.
01 The problem
AI calls at volume produce transcripts nobody has time to read. The useful parts, who answered, what they said and what happens next, needed to land in the CRM and the team's sheets on their own.
02 The approach
FastAPI webhooks handle new leads and call-end events, Celery and Redis run the background work, and an LLM layer with a switchable provider turns each transcript into structured data that is written to Salesforce and Google Sheets.
Highlights
- 01Outbound ElevenLabs voice-agent calls placed through Twilio
- 02Post-call webhooks that log every call and its outcome
- 03LLM variable extraction and rubric scoring, with Claude or OpenAI selectable
- 04Results written to Salesforce and Google Sheets, with GoHighLevel webhooks feeding new leads in
- 05Call analytics, background jobs on Celery and Redis, JWT-protected API
My part
This was a team project. These are the pieces I built, taken from my commits in the repository.
- Added JWT authentication to the API
- Integrated OpenAI alongside Claude as a selectable LLM provider
- Built the GoHighLevel webhook flow and a lead migration script
- Made output Google Sheets dynamic per workflow and removed hard-coded prompts
- Added rubric-based scoring prompts, area-code geo mapping and WhatsApp notifications
Stack
Building something similar? Get in touch or see all work.