Introducing telli Duo Agents: Why we are betting on speech-to-speech AI

A good conversation has a rhythm. We pause, interrupt, and respond before the other person has finished their sentence. It happens so naturally that we rarely think about it.

On a phone call, people don’t always behave as expected. They start answering before a question is finished, correct themselves half-way through, ask for something to be looked up while they keep talking, or become frustrated and need to hear a calm, reassuring voice.

Building that into a voice agent has been impossible, until now.

Introducing telli Duo Agents

Today we are releasing telli Duo agents to the public - our new voice agents that can listen and speak at the same time.

Over the past two years, the industry has adopted what is known as the "cascading architecture". This means when a human speaks to an agent, the speech gets transcribed to text, fed into an AI model and converted back to voice audio.

This architecture has a fundamental flaw: it is impossible for an agent to listen and speak at the same time. Instead, we stitched even more models together, like voice activity detection models and turn-taking models, to try to guess when a user has stopped speaking and when the agent should respond. But no matter how hard we tried, we could never get the agent to feel truly human.

The Future of Voice AI

At telli, we have always believed that speech-to-speech models will be the future of Voice AI, but every model we evaluated would fail when it comes to following instructions and not breaking in odd ways.

This changed when we received early access to OpenAI’s new GPT-Live model a couple months ago. It’s a full-duplex speech-to-speech model that can keep listening while it speaks. It handles the flow of speech and interruptions directly, and can be prompted to delegate tasks that require deeper reasoning to another model.

After our initial benchmarking, we quickly decided to rebuild our voice infrastructure around this approach. It took a lot of work and some sleepless nights, but we could draw on two years of experience and millions of production calls.

The result is the new telli Duo agent. Two models working alongside each other to create the best customer experience on voice AI calls. One model listens and speaks to the user at the same time. Another model can take care of complex tasks that require deeper reasoning.

From Model to Product

To make this work in every environment and use case, we added a routing mechanism that knows when to delegate to more intelligent models. We also built in our voicemail detection, warm transfers, consent management and all other features that telli users already love in our platform.

And as prompting Duo agents is more complex and nuanced, we trained Charlie - our AI co-worker - to help our customers easily build them without having to worry about breaking one.

The early feedback from customers has been off the charts. Many have seen an instant uptick in their KPIs and we are excited about what new use-cases this unlocks by finally being able to deploy voice agents that feel truly human.

At telli, we believe talking to a company should be as easy as talking to a friend. Duo brings us a step closer to that vision.

Try it yourself on telli.com or book a demo and we will show you what telli Duo agents can do.