Introducing Prism: the first composable speech-to-speech model

Today Rime is launching Prism, the first speech-to-speech model that passes a stricter version of the audio turing test: it not only sounds like a human, it listens and acts like a human, too. 

Prism responds to user audio in about 500ms, faster than the most comparably intelligent model, with tool calls happening in parallel to speech, and an inspectable text layer to run guardrails over before the model speaks.

How did we get here

Eighteen months ago I said, very publicly and very confidently, that speech-to-speech models were a decade out from real use. That’s because I lacked imagination about what speech-to-speech models could be. I took, at face value, that the way that the biggest labs build speech-to-speech models must be the only way, with text and audio understanding fused at runtime. This fused runtime means the voice agent sounds responsive, but you can’t inspect what it’s about to say, you can’t use it in a composable system, and you can’t own it.

At the same time, we all recognize that the cascade hasn’t been long for this world. While I’m proud that Rime is the voice you hear when you call some of the world’s leading brands, I haven’t been proud about the overall quality of the experience. The only saving grace of the cascade is that you can build with it. That saving grace is also a hard ceiling. If text is the only thing that mediates a voice interaction, you are destroying the most important information in interaction.

Choosing the big lab paradigm over the cascade is a classic Faustian bargain - a trade where you get something powerful now in exchange for something you’ll regret giving up later. You get a model that hears, but you lose the ability to read what it’s about to say, to rewrite it. Or, with the cascade, you get control, but the price is that you lose all the intelligence accessible in audio - hesitation, anger, laughter, or the dog barking in the background – all of it is buried before the model even decides what to say.

This is why we spent the last six months building Prism.

How Prism works

Prism is trained end-to-end as one system that does three jobs: it listens, it writes what it’s going to say, and when it speaks, it is still hearing the user. In Prism the text is a checkpoint. The decision was made over the audio, your rules can read every word before it becomes speech, and the speech still carries the caller’s frustration, laughter, or the dog barking in the background. That checkpoint is also where you get rewritable text for action and content tags, phonemic conditioning for names and account terms, and tool management for composable agent flows.

‍

How Prism performs among other speech-to-speech models

We put Prism on the same harness Artificial Analysis uses for its speech-to-speech leaderboard, and we ran it the way you would run it on a live call: no reasoning step before it speaks. τ-Voice is Artificial Analysis's benchmark for voice agents, tool calling and instruction following on simulated customer-service calls across airline, retail and telecom. Prism scores 56.6, ahead of every realtime model from OpenAI and Google that isn't reasoning first, by 17 to 30 points, and ahead of GPT-Realtime-2.1 even with its reasoning turned up. In telecom, the domain with the most account numbers and phone numbers to get right, Prism scores 82.5 against 45.6 for GPT-Live-1.

It does that in 0.57 seconds to first audio, about half the wait of GPT-Live-1, GPT-Realtime-2.1 and Gemini 3.8 Live. The two models that score above Prism on τ-Voice get there by reasoning before they speak, and they take 1.24 to 1.35 seconds to start talking.

Own your intelligence

Today's voice agents can't think before they speak. Chain-of-thought reasoning, the thing that makes text models better at hard problems, blows the latency budget of a live call. So the intelligence in the weights is the intelligence you get, and the only way to get better in your particular domain is to own those weights.

Prism is the first speech-to-speech model anyone can fine-tune. With our beta customers we're building self-serve reinforcement learning (RL) and supervised fine-tuning (SFT) APIs for training and deploying your own Prism.

‍

‍

Revmo runs its ordering agent on OpenAI today. We took 10 hours of Revmo's production calls, tool traces, and transcripts, built an RL environment around them, and trained a Prism on them. On Revmo calls, Prism asks before it acts: 94% of orders went in only after the caller said yes to "shall I place it?" OpenAI got that yes 57% of the time. 

Sweet lessons

Eighteen months ago I confidently stated that speech-to-speech was a decade out. Then I staked Rime's entire roadmap on Prism. So here's my new prediction, and you can hold me to this one too: eighteen months from now, voice agents built on the cascade will look like pale, ghastly imitations of agents built on Prism.

The bitter lesson, Richard Sutton's argument that general methods that learn from data eventually beat systems built on hand-engineered knowledge, isn't all that bitter if you know how to design something that's built for a purpose. It's actually pretty sweet.

Try Prism for free today.

‍

‍