Skip to main content

Assist Pipelines

The Assist pipeline integration runs the common steps of a voice assistant:

  1. Speech to text
  2. Intent recognition
  3. Text to speech

Pipelines are run via a WebSocket API:

{
"type": "assist_pipeline/run",
"start_stage": "stt",
"end_stage": "tts",
"input": {
"sample_rate": 16000,
}
}

The following input fields are available:

NameTypeDescription
start_stageenumRequired. The first stage to run. One of stt, intent, tts.
end_stageenumRequired. The last stage to run. One of stt, intent, tts.
inputdictDepends on start_stage. For STT, the dictionary should contain a key sample_rate with an integer value. For intent and TTS, the key text should contain the input text.
pipelinestringOptional. ID of the pipeline (use assist_pipeline/pipeline/list to get names).
conversation_idstringOptional. Unique id for conversation.
timeoutnumberOptional. Number of seconds before pipeline times out (default: 30).

Events

As the pipeline runs, it emits events back over the WebSocket connection. The following events can be emitted:

NameDescriptionEmittedAttributes
run-startStart of pipeline runalwayspipeline - ID of the pipeline
language - Language used for pipeline
runner_data - Extra WebSocket data:
  • stt_binary_handler_id is the prefix to send speech data over.
  • timeout is the max run time for the whole pipeline.
run-endEnd of pipeline runalways
stt-startStart of speech to textaudio onlyengine: STT engine used
metadata: incoming audio metadata
stt-endEnd of speech to textaudio onlystt_output - Object with text, the detected text.
intent-startStart of intent recognitionalwaysengine - Agent engine used
language: Processing language.
intent_input - Input text to agent
intent-endEnd of intent recognitionalwaysintent_output - conversation response
tts-startStart of text to speechaudio onlyengine - TTS engine used
language: Output language.
voice: Output voice.
tts_input: Text to speak.
tts-endEnd of text to speechaudio onlymedia_id - Media Source ID of the generated audio
url - URL to the generated audio
mime_type - MIME type of the generated audio
errorError in pipelineOn errorcode - Error code
message - Error message