What is a voice agent? How it works and how to deploy one
A voice agent is an AI system that talks over the phone and can query data or take actions. Learn how it works, use cases, costs, permissions, privacy and how to pilot one.
A voice agent is an artificial-intelligence system that holds a spoken conversation and can use tools to complete a task: check an order, propose an appointment, open a ticket or transfer the call to a person with context.
Voice is only the interface. Value appears when the agent understands the call, consults an authorised source, applies business rules and leaves a traceable result.
How a voice agent works
Two architectures are common:
- A pipeline combines speech recognition, a language model and text-to-speech.
- A speech-to-speech model processes and generates audio directly while keeping conversation context and calling tools when needed.
Both normally add:
- Telephony or SIP to receive and transfer calls.
- A prompt defining identity, tone, limits and mandatory scripts.
- Tools for CRM, calendars, orders or internal knowledge.
- Permission rules for every read or action.
- Human handoff, logs and evaluation.
OpenAI's Realtime API, for example, supports SIP calls, tools, remote MCP servers and speech-to-speech models. Other providers let teams combine telephony, transcription, models and voices. The right architecture depends on country, language, latency, integrations, data location and total cost per resolved call.
What it can do
Answer repeated questions
Opening times, location, required documents, policies or request status are candidates when there is a current source and a clear answer.
The agent should not memorise changing information. It should query the owning system or a dated, permissioned knowledge base.
Manage appointments and bookings
It can check availability, propose times, create an appointment draft and send confirmation. Exceptional changes or sensitive appointment types may require human approval.
Check orders and incidents
After verifying the caller, it can read order status, create a ticket or summarise an incident. Refunds, irreversible cancellations and financially significant decisions need stricter limits.
Qualify sales calls
It can capture the need, location, timing and contact details, register the lead and route it to the right team. It must not invent price, availability or commitments that the source system has not confirmed.
Support employees
An agent can answer staff questions about procedures or documents. It must preserve permissions: a caller should not obtain information by voice that they could not access in the original application.
What to design before connecting the phone
Identity and verification
Define what can be disclosed without verification and which check each data type requires. Asking for opening times is different from reading an invoice or changing a medical appointment.
Avoid questions with guessable answers. For higher-risk actions, combine the call with a one-time code or verification through an authenticated channel.
Tool permissions
Do not provide one credential with full access. Separate read and write tools and limit accepted parameters.
A first pilot can run read-only. Add reversible actions later, such as creating a ticket or draft. Sensitive actions should remain blocked or require confirmation.
Human handoff
Transfer is more than forwarding the call. The team needs:
- Contact reason.
- Verified data.
- Conversation summary.
- Queries and actions performed.
- Exact escalation reason.
There must also be a fallback when no person is available: callback, ticket or minimum information capture.
Logs, transcripts and recordings
Record enough events to investigate errors without retaining unnecessary data. Do not treat recording, transcription and analytics as one automatic bundle: each needs its own purpose, access and retention.
An identifiable voice or recording may be personal data. It can fall into special biometric categories when technically processed to uniquely identify a person. Review the actual purpose and lawful basis rather than relying on a generic “GDPR compliant” claim.
Transparency and the AI Act
Article 50 transparency obligations have applied since 2 August 2026. In practical terms, a system designed to interact directly with people must inform them they are interacting with AI unless that is obvious from context.
The opening should therefore identify the assistant. If the call is recorded or transcribed, provide the relevant privacy information and channels. Exact wording and further duties depend on purpose, sector and configuration; validate the notice with legal counsel.
A phased deployment
1. Analyse real calls
Classify a sample by reason, duration, data consulted, final action, exceptions and escalation point. Do not start with the voice vendor.
2. Choose a low-risk workflow
A good pilot has volume, a verifiable answer and a human exit. Opening times, requirements or read-only status checks are safer than complaints or complex sales.
3. Define a conversation policy
Document what it can say, what must be read verbatim, what it cannot promise, which data it asks for and when it ends or transfers.
4. Connect one source and one action
Keep the first scope narrow—for example, check a calendar and create an appointment draft. Adding CRM, tickets, WhatsApp and payments at once makes failures harder to isolate.
5. Simulate before customer calls
Test interruptions, noise, language switches, names and numbers, angry users, silence, tool failure, ambiguous data and attempts to bypass permissions.
6. Launch with limits
Enable one time window, enquiry type or call percentage. Review daily at first and retain a fast path back to human service.
What to measure
Do not track only “calls answered.” Use:
- Correct resolution without intervention.
- Correct and late transfers.
- Actions confirmed by the owning system.
- Data, permission and commitment errors.
- Time to resolution.
- Repetition after handoff.
- Total cost per correctly resolved call.
Review a sample that includes failures. A high automation rate can hide poor answers when nobody checks quality.
What it costs
There is no universal per-minute figure. Total cost combines:
- Telephony and numbers.
- Audio, transcription or realtime models.
- Reasoning and tool calls.
- Integrations, storage and observability.
- Development, evaluation, support and human review.
Model at least three volume scenarios and include peaks, transfers and retries. Compare cost per correct resolution, not only cost per minute.
When not to use one
It is not a good first project when:
- Most calls require negotiation or expert judgement.
- There is no reliable source of truth.
- Caller verification is not possible when needed.
- There is no escalation team or fallback.
- Volume does not justify telephony integration and maintenance.
A form, WhatsApp chatbot or assisted internal tool may be a better fit.
Final recommendation
A useful voice agent does not try to sound human at all costs. It identifies itself as AI, resolves a narrow workflow, uses minimum permissions and knows when to stop.
Navel Digital can review a sample of calls, map tools and propose a pilot through our AI automation and agent service.