WHIP client

An output that lets you connect to a WHIP server endpoint to stream video and audio to it.

Usage

To use WHIP Output you must register it first. You can do it by sending a request like this:

Example request

POST: /api/output/:output_id/register
Content-Type: application/json

{
  "type": "whip_client",
  "endpoint_url": "https://example.com/whip",
  "bearer_token": "<TOKEN>",
  "video": {
    "resolution": { "width": 1280, "height": 720 },
    "encoder_preferences": [
      { "type": "ffmpeg_h264", "preset": "ultrafast" },
      { "type": "ffmpeg_vp8" },
      { "type": "any" }
    ],
    "initial": {
      "root": {
        "type": "view"
      }
    }
  },
  "audio": {
    "channels": "stereo",
    "initial": {
      "inputs": [{ "input_id": "input_1", "volume": 1 }]
    }
  }
}

Reference

Type definitions

type WhipClient = {
  type: "whip_client";
  endpoint_url: string;
  bearer_token?: string;
  video?: VideoOptions;
  audio?: AudioOptions;
};

Parameters for an WHIP output stream from Smelter.

Properties

endpoint_url

The destination URL for sending media streams using the WHIP.

Type: string

bearer_token

Token used for authentication when connecting to the WHIP endpoint.

Type: string

video

Parameters of a video included in the WHIP stream.

Type: VideoOptions

audio

Parameters of an audio included in the WHIP stream.

Type: AudioOptions

VideoOptions

Parameters of a video source included in the WHIP stream.

Type definitions

type VideoOptions = {
  resolution: {
    width: u32;
    height: u32;
  };
  send_eos_when?: OutputEndCondition;
  encoder_preferences?: VideoEncoderOptions[];
  initial: { root: Component; };
};

Properties

resolution

Output resolution in pixels.

Type: { width: number; height: number; }

send_eos_when

Condition for termination of the output stream based on the input streams states. If output includes both audio and video streams, then EOS needs to be sent for every type.

Type: OutputEndCondition

encoder_preferences

An ordered list of preferred video encoders. The first element in the list has the highest priority during WHIP negotiation.

Type: VideoEncoderOptions[]
Default value: [{ type: "any" }]

Behavior:

If the list ends with “any”:
- Smelter will first try the encoders explicitly listed (in order) and use the first one that is supported and negotiated in WHIP signaling.
- If none of the listed encoders are supported, Smelter will fall back to any supported codec from the negotiated list that wasn’t already in the preferences.
If “any” is not included:
- Only the encoders listed will be considered.
- If none are supported, no fallback will occur.

initial

Root of a component tree/scene that should be rendered for the output.

Type: { root: Component; }

VideoEncoderOptions

Type definitions

type VideoEncoderOptions =
  | ({ type: "ffmpeg_h264"; } & FfmpegH264EncoderOptions)
  | ({ type: "ffmpeg_vp8"; } & FfmpegVp8EncoderOptions)
  | ({ type: "ffmpeg_vp9"; } & FfmpegVp9EncoderOptions)
  | ({ type: "vulkan_h264"; } & VulkanH264EncoderOptions)
  | { type: "any" };

Configuration for the video encoder, based on the selected codec. Visit encoder documentation to learn more.

AudioOptions

Parameters of an audio source included in the WHIP stream.

Type definitions

type AudioOptions = {
  mixing_strategy?: "sum_clip" | "sum_scale";
  send_eos_when?: OutputEndCondition;
  encoder_preferences?: AudioEncoderOptions[];
  channels?: "mono" | "stereo";
  initial: { inputs: InputAudio[]; };
}

Properties

mixing_strategy

Specifies how audio should be mixed.

Type: "sum_clip" | "sum_scale"
Default value: "sum_clip"
Supported values:
- sum_clip - First, the input samples are summed. If the result exceeds the i16 PCM range, it is clipped.
- sum_scale - First, the input samples are summed. If the result exceeds the i16 PCM range, the summed samples are scaled down by a factor to fit within the range.

send_eos_when

Condition for termination of the output stream based on the input streams states. If output includes both audio and video streams, then EOS needs to be sent for every type.

Type: OutputEndCondition

encoder_preferences

An ordered list of preferred audio encoders. The first element in the list has the highest priority during WHIP negotiation.

Type: AudioEncoderOptions[]
Default value: [{ type: "any" }]

Behavior:

If the list ends with “any”:
- Smelter will first try the encoders explicitly listed (in order) and use the first one that is supported and negotiated in WHIP signaling.
- If none of the listed encoders are supported, Smelter will fall back to any supported codec from the negotiated list that wasn’t already in the preferences.
If “any” is not included:
- Only the encoders listed will be considered.
- If none are supported, no fallback will occur.

channels

Channels configuration

Type: "mono" | "stereo"
Default value: "stereo"
Supported values:
- mono - Mono audio (single channel).
- stereo - Stereo audio (two channels).

initial

Initial audio mixer configuration for output.

Type: { inputs: InputAudio[]; }

AudioEncoderOptions

Configuration for the audio encoder.

Type definitions

type AudioEncoderOptions =
  | ({ type: "opus"; } & OpusEncoderOptions)
  | ({ type: "any"; });

Configuration for the audio encoder. Visit encoder documentation to learn more.

Opus

The type: "any" specifies that any audio encoder supported by Smelter can be used. If "any" is not included in the encoder_preferences list, Smelter will only use the encoders explicitly listed in the preferences.

InputAudio

Type definitions

type InputAudio = {
  input_id: string;
  volume?: f32;
}

Properties

input_id

ID of an input. It identifies a stream registered using a register input method.

Type: string

volume

Input volume in range [0, 2]

Type: f32
Default value: 1.0
Supported values: [0, 2]

OutputEndCondition

Defines when the output stream should end based on the state of the input streams. Only one of the nested fields can be set at a time.

By default, the input stream is considered finished/ended when:

TCP connection was dropped/closed.
RTCP Goodbye packet (BYE) was received.
MP4 track has ended.
Input was unregistered already (or never registered).

Type definitions

type OutputEndCondition =
  | { any_of: string[]; }
  | { all_of: string[]; }
  | { any_input: bool; }
  | { all_inputs: bool; };

Properties

any_of

List of the input streams. The output stream will terminate if any stream in the list finishes.

Type: string[]

all_of

List of the input streams. The output stream will terminate when all streams in the list finish.

Type: string[]

any_input

Terminate the output stream if any of the input streams end, including streams added after the output was registered. Notably, the output stream will not terminate if no inputs were ever connected.

Type: bool

all_inputs

Terminate the output stream only when all input streams have finished. Notably, the output stream will terminate if no inputs were ever connected.

Type: bool