[Q] How to support adaptive bitrate for audio?

[falk-stefan]

It's possible that I have misunderstood something, but I don't quite understand what is required in order to support an adaptive bitrate for, for example, audio.

The following seems to create different versions of the input audio:

audio_channel_layouts:

  mono:
    max_channels: 1
    bitrates:
      aac: '96k'

  stereo:
    max_channels: 2
    bitrates:
      aac: '192k'

  surround:
    max_channels: 6
    bitrates:
      aac: '256k'

audio_und_1c_96k_aac.mp4
audio_und_2c_192k_aac.mp4

Note: For some reason the 6-channel surround version is not being created, that's probably a separate issue.

However, I would have assumed that I'd need to provide multiple sample-rates for a codec. For example:

audio_channel_layouts:
  stereo:
    max_channels: 2
    bitrates:
      aac: 
        - '96k'
        - '128k'
        - '192k'
        - '256k'

in order to provide different bitrates to DASH or HLS. Since this does not seem to be the case, I seek to ask if this needs to be done in a different way or do I not have to care about this at all?

pipeline.yaml

# Streaming mode.  Can be live or vod.
streaming_mode: vod

# A list of channel layouts to encode.
channel_layouts:
  - mono
  - stereo
  - surround

# The codecs to encode with.
audio_codecs:
  - aac

# Manifest format (dash, hls or both)
manifest_format:
  - dash
  - hls

# Length of each segment in seconds.
segment_size: 10

# Forces the use of SegmentTemplate in DASH.
segment_per_file: False

[joeyparrish]

Note: For some reason the 6-channel surround version is not being created, that's probably a separate issue.

If the input is < 6 channels, the 6-channel output will not be created. Similarly, if input video is < 4K, 4K output will not be created. We don't upscale video, and we don't up-mix audio channels, because those would be wasteful and not increase quality.

I would have assumed that I'd need to provide multiple sample-rates for a codec.

That is not how the configs are currently structured. I'm open to changes, but the simplistic way it works now is as described in the configs docs:

https://shaka-project.github.io/shaka-streamer/configuration_fields.html#streamer.bitrate_configuration.AudioChannelLayout.bitrates

I know these generated docs are not super friendly to read. A friendlier explanation is this:

Each resolution and channel layout is encoded in just one bitrate. It doesn't (currently) have options for multiple bitrates at the same video resolution or at the same audio channel layout (stereo, mono, surround, etc), except in the case of different codecs.

Different codecs don't need the same number of bits to achieve a target quality. So the default bitrate for 2-channel Opus is lower than for 2-channel AAC. So the config involves a mapping of codec to bitrate for each channel layout. Ex (taken from the defaults):

  mono:
    max_channels: 1
    bitrates:
      aac: '64k'
      opus: '32k'

  stereo:
    max_channels: 2
    bitrates:
      aac: '128k'
      opus: '64k'

  surround:
    max_channels: 6
    bitrates:
      aac: '256k'
      opus: '128k'

That said, I think your idea is a good one. I think the default should still be one bitrate per layout, but I see no reason to deny someone the ability to ask for more. The custom bitrate config is the right place to do it. So I would support a PR to make this the format instead:

audio_channel_layouts:
  stereo:
    max_channels: 2
    bitrates:
      aac: 
        - '96k'
        - '128k'
        - '192k'
        - '256k'

This would be a breaking change, so the PR title should begin with 'feat!: ...' to ensure a bump to a new major version. Anyone with an existing custom bitrate config would have to update it to the new format.

@falk-stefan, are you interested in implementing this?

[falk-stefan]

I really wish I could help out here but I'm already completely under water with my current project (and full-time job). :sweat:

[joeyparrish]

No worries. We'll just leave this open in case someone is interested in implementing the recommendations above.