Support of multiple audio and video tracks in WebM

[iSchluff]

FFmpeg is able to encode multiple videotracks into a WebM-Container, but shaka-packager only ever recognizes one video- and one audiotrack. With this feature it would be possible to mux multi-quality webm-dash over a single pipe.

Commands

mkfifo pipe

ffmpeg -y -hide_banner -analyzeduration 10000 -i rtmp://foo/rtmp/bar \
  -aspect 16:9 -flags +global_header -c:v libvpx -pix_fmt yuv420p -deadline realtime -threads 4 \
  -map v:0 -s:0 1280x720 -bufsize:0 6.0M -b:v:0 4.5M -maxrate:0 5.0M -crf:0 10 -keyint_min:0 50 -g:0 50 -r:0 25 \
  -map v:0 -s:1 1024x576 -bufsize:1 3.5M -b:v:1 2.0M -maxrate:1 2.5M -crf:1 10 -keyint_min:1 50 -g:1 50 -r:1 25 \
  -c:a libvorbis -ar 44100 -ac 2\
  -map a:0 -b:a:0 192k \
  -map a:0 -b:a:1 128k \
  -f webm pipe

shaka-packager \
  "input=pipe,stream_selector=0,init_segment=v720.webm,segment_template=v720_\$Number\$.webm,bandwidth=4500000" \
  "input=pipe,stream_selector=1,init_segment=v576.webm,segment_template=v576_\$Number\$.webm,bandwidth=2000000" \
  "input=pipe,stream_selector=2,init_segment=a192.webm,segment_template=a192_\$Number\$.webm,bandwidth=192000" \
  "input=pipe,stream_selector=3,init_segment=a128.webm,segment_template=a128_\$Number\$.webm,bandwidth=128000" \
  --profile live --single_segment=0 --dump_stream_info\
  --time_shift_buffer_depth=300 --segment_duration=3 --io_block_size 65536 \
  --mpd_output manifest.mpd

Output

Input #0, flv, from 'rtmp://foo/rtmp/bar':
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0: Audio: aac (LC), 44100 Hz, stereo, fltp
    Stream #0:1: Video: h264 (Constrained Baseline), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 50 fps, 50 tbr, 1k tbn, 100 tbc
[libvpx @ 0x15b9d20] v1.5.0
[libvpx @ 0x1776480] v1.5.0
Output #0, webm, to 'pipe':
  Metadata:
    encoder         : Lavf56.40.101
    Stream #0:0: Video: vp8 (libvpx), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=-1--1, 4500 kb/s, 25 fps, 1k tbn, 25 tbc
    Metadata:
      encoder         : Lavc56.60.100 libvpx
    Stream #0:1: Video: vp8 (libvpx), yuv420p, 1024x576 [SAR 1:1 DAR 16:9], q=-1--1, 2000 kb/s, 25 fps, 1k tbn, 25 tbc
    Metadata:
      encoder         : Lavc56.60.100 libvpx
    Stream #0:2: Audio: vorbis (libvorbis), 44100 Hz, stereo, fltp, 192 kb/s
    Metadata:
      encoder         : Lavc56.60.100 libvorbis
    Stream #0:3: Audio: vorbis (libvorbis), 44100 Hz, stereo, fltp, 128 kb/s
    Metadata:
      encoder         : Lavc56.60.100 libvorbis
Stream mapping:
  Stream #0:1 -> #0:0 (h264 (native) -> vp8 (libvpx))
  Stream #0:1 -> #0:1 (h264 (native) -> vp8 (libvpx))
  Stream #0:0 -> #0:2 (aac (native) -> vorbis (libvorbis))
  Stream #0:0 -> #0:3 (aac (native) -> vorbis (libvorbis))

File "pipe":
Found 2 stream(s).
Stream [0] type: Audio
 codec_string: vorbis
 time_scale: 1000000
 duration: 18446744073709550616 (18446744073709.6 seconds)
 is_encrypted: false
 codec: Vorbis
 sample_bits: 16
 num_channels: 2
 sampling_frequency: 44100
 language: und

Stream [1] type: Video
 codec_string: vp08.00.00.08.01.01.00.00
 time_scale: 1000000
 duration: 18446744073709550616 (18446744073709.6 seconds)
 is_encrypted: false
 codec: VP8
 width: 1280
 height: 720
 pixel_aspect_ratio: 1:1
 trick_play_rate: 0
 nalu_length_size: 0

[1118/101421:ERROR:packager_util.cc(215)] Invalid argument --stream=2; should be 'audio', 'video', or a number within [0, 1].

[kqyang]

Thanks for reporting the issue. We'll look into it!

[colleenkhenry]

Is there any progress on this? Any workaround for ABR vp9 webm dash with Shaka packager?

[kqyang]

Unfortunately, no. We are super busy with something else right now. Haven't had a chance to look into it yet.

[colleenkhenry]

Thanks for the update! I look forward to when you can look.

[iSchluff]

This can be worked around now that we added WebM support to the ffmpeg dash-muxer. However it would still be useful to have this capability in shaka-packager for encrypted livestreams.

[kqyang]

We do not have time to work on this feature request in Shaka Packager right now. If anyone wants to contribute, we welcome PR!