Recent AIProxySwift releases expand both sides of an audio experience: turning text into synchronized speech with ElevenLabs and transcribing audio with Mistral's Voxtral models or ElevenLabs Scribe.

ElevenLabs text-to-speech with timestamps

ElevenLabs text-to-speech requests can now return character alignment alongside the generated audio. Both buffered and streaming interfaces are available. In a streaming UI, each chunk includes audio data plus alignment and normalized-alignment values, which can be used to highlight words, drive captions, or synchronize animation as speech plays.

If your app needs provider response details for accounting or debugging, the ElevenLabs interfaces can also return response headers alongside buffered and streaming results.

Mistral Voxtral transcription

AIProxySwift now supports Mistral's audio transcription endpoint for both complete and streaming responses. Audio can be supplied as file data, a URL, or an uploaded file ID. Optional controls include language hints, speaker diarization, context bias, temperature, and word- or segment-level timestamps.

Streaming transcriptions arrive as typed events for text deltas, detected language, timed segments, and the completed result. Usage information includes token totals and audio duration when Mistral returns them.

ElevenLabs Scribe v2

The ElevenLabs speech-to-text request now includes scribe_v2 in addition to scribe_v1. The existing Swift interface supports language hints, speaker diarization, audio-event tagging, and timestamp controls.

Complete examples for ElevenLabs and Mistral are available in the AIProxySwift README. As with the rest of the library, you can use a direct provider key while developing and switch to a protected AIProxy service before shipping.