What happens when a human takes over a live AI call

When a voice agent hands a call to a person, the interesting question is not "how is the transfer done". It is what happens to everything else: the recording, the transcript, the context, and whether the agent is still listening. CallAgent answers it differently on two paths, and one of them is off by default. I am writing both, because writing only the flattering one is exactly the thing this site does not do.

It is not a SIP transfer

First correction of vocabulary. On every path, human takeover is a bridge, not a REFER. There is no REFER anywhere in the transfer service.

On the default path, with Twilio, the caller is redirected into a conference and the operator is dialled in as a second participant. The obvious shape — hang up and dial the operator — was rejected for two reasons written into the code: the caller hears carrier ringback for as long as the operator's phone rings, and it leaves you nowhere to whisper anything to the operator before they join.

What the conference path costs you

Here is the part that does not appear in brochures. A Twilio redirect tears down the media stream connected to the gateway. In practice: the media socket closes about half a second after the transfer starts, and everything hanging off the gateway session goes with it.

The consequences, as recorded in the code: the recorder keeps only the assistant's half, live transcription stops, the live-calls screen shows nothing, and the supervisor console has no session to monitor. The human half of the call arrives separately, as a carrier recording, mixed to mono.

That is a real limitation, it is documented, and it is why a second path exists.

The second path: the caller does not move

There is a mechanism where the operator attaches as a second participant to the existing gateway session, and the caller is not moved anywhere. The session stays the same, so the recording, the transcription and the metrics carry on unbroken.

It is off by default. The gateway side is implemented and tested, but the operator and supervisor consoles do not speak that protocol yet, so it is enabled from an environment variable only once a console ships that opens the socket. We are not claiming it is available; the code says it is not.

The rest of this piece describes what that path does, because it is the correct design and because it is what is coming.

The agent is stood down, not stopped

The distinction matters. When the operator attaches, the agent is stood down: no turn runs, nothing is spoken to fill the silence, and the silence timeout does not fire.

What does not stop: the caller's audio is still recorded, still transcribed and still written to the transcript. That is the whole point of holding the caller here rather than redirecting them away.

Silencing the agent is not a new implementation. It uses exactly the same primitive as ordinary barge-in, with the reason operator_takeover. The comment in the code says why: a second implementation would be a second set of bugs.

Two state transitions, not one

The call does not jump straight to "connected to a human". It goes through human_queue, then into human_connected. The reason is that the permitted transition table — the same one in TypeScript and in the PHP enum — has no direct edge from the previous state.

That same table explains handback: there is no edge from human_connected straight back to listening. Returning to the agent goes through human_queue too, explicitly.

A supervisor is a different thing

A supervisor attaching to listen triggers no state transition and does not stand the agent down. A supervisor listening is not an operator connecting, and the events are kept separate precisely so they do not look the same in a report.

Also in the "validated on the server, not in the browser" category: the supervisor's mode — listen, whisper, or join the conversation — is checked server-side. "The browser said barge" must not be able to put a listener on the caller's line.

Recording: channel 1 is "whoever is answering"

On the path where the session survives, channel 1 is "whoever is answering" for the whole call — the assistant, then the operator, with no seam between them. That is why stitching two files together is a legacy path rather than the normal one: there is no second half to stitch on.

There are three recording modes: mono, stereo and dual channel. On stereo and dual channel the caller is on the left and whoever answers is on the right; dual channel additionally guarantees per-channel files, which is what diarization and QA scoring consume.

The transcript: the seam row has no text

When the handover happens, a row with the role handoff is written into the transcript. It carries the operator's id, their name, whether it is final, the offset in milliseconds from the start of the call, and a metadata object with the reason.

What it does not carry: content. The field is deliberately left empty, and the reason is written there: a Romanian sentence baked in at write time would be wrong for half its readers for ever. The metadata carries the facts, and the view writes the sentence in the reader's own language.

The row is idempotent. If a handover to the same operator happens twice, the existing row is returned.

The briefing is frozen at enqueue

The operator receives a context: the handoff reason, the caller, the assistant, the call, a summary, the intent, the topics, the sentiment, the last 40 turns of transcript, the results of any tools called, and the customer record.

It is assembled once, when the call enters the queue, and frozen. Not at answer time. The reason: assembling it at answer time would put a multi-query transcript read on the critical path between the operator clicking accept and audio flowing, and would produce a different briefing each time the call bounced between operators.

If the operator's connection drops

The call is not hung up. There is a reconnect window of 12 seconds. The gateway emits an event saying it lost the operator's leg and pushes an artefact to the control plane — the gateway holds no carrier credentials, so falling back to a conference is the control plane's job.

Past the window, the leg is detached and a handback event from operator to assistant is emitted.

What to take away

If you are designing your escalation flow now: the context is built at enqueue, not at answer, so put everything you want the operator to see in it. Handback goes through the queue, so do not assume the state jumps straight back. And if continuous transcription across the moment of takeover matters to you, check which path you are on — on the conference path, as it stands today, you do not have it.