Input

Configure audio formats and language hints for a Speechmatics Realtime transcription session.

Supported input audio formats

Sessions can be configured to use two types of audio input: file and raw.
We recommend using the raw option, unless you have a specific reason to use the file option.

For capturing raw audio in the browser, try our browser-audio-input package.

audio_format

The format must be supplied in the audio_format field of the StartRecognition message. See the API reference.

AudioFormatIn the API reference: StartRecognition

AudioFormatRaw | AudioFormatFile

one of

AudioFormatRaw

Raw

optional

Raw audio samples, described by the following additional mandatory fields:

type

"raw"

encoding

RawAudioEncodingEnum"pcm_f32le""pcm_s16le""mulaw"

sample_rate

integer

The sample rate of the audio in Hz.

AudioFormatFile

File

optional

Choose this option to send audio encoded in a recognized format. The AddAudio messages have to provide all the file contents, including any headers. The file is usually not accepted all at once, but segmented into reasonably sized messages.

Note: Only the following formats are supported: wav, mp3, aac, ogg, mpeg, amr, m4a, mp4, flac

type

"file"

Language hints

Language hints are optional and apply only to the Melia 1 and Oak 1 models, which detect supported languages automatically. For the Enhanced and Standard models, set language to a supported language instead.

Set language_hints in the StartRecognition message to the supported languages you expect in your audio. Hints have two effects:

• Language labels are restricted to the hinted languages. Every word in the transcript is labelled with one of the languages you list.
• Transcription is biased toward the hinted languages. This can improve accuracy, especially for short utterances and similar-sounding languages.

Speech in other languages may still be transcribed in some cases, but it is labelled as one of the hinted languages. Only use hints when you know which languages your audio contains.

With one hint, the whole session is labelled as that language. With two or more, labels can switch between the hinted languages. This config hints that the audio contains English and Arabic:

{
  "message": "StartRecognition",
  "audio_format": {
    "type": "raw",
    "encoding": "pcm_s16le",
    "sample_rate": 16000
  },
  "transcription_config": {
    "model": "melia-1",
    // Melia 1 requires "multi" rather than a single language code
    "language": "multi",
    "language_hints": ["en", "ar"]
  }
}

Speechmatics language hints restrict output rather than only biasing it. Unlike hints that make listed languages more likely, they exclude unlisted languages from labelling.

To render each detected language correctly, refer to the LanguageInfo message.

Sending audio

After receiving a RecognitionStarted message, you can start sending audio over the Websocket connection. Audio is sent as binary data, encoded in the format specified in the StartRecognition message. See Protocol overview for complete details of the API protocol.

Next steps

View our guides:

• using a microphone to learn how to capture audio from a microphone.
• using FFMPEG to find out how to pipe microphone audio to the API.