Support raw AAC through mux.js

[ejadib]

Have you read the FAQ and checked for duplicate open issues? Yes

What version of Shaka Player are you using? v2.5.12 (the demo app)

Please ask your question I'm testing a HLS MPEG-2 TS VOD content, which has .aac audio in Windows 10 + Firefox 76 using the demo app.

The video is being reproduced ok, however there is no audio. The demo app has mux.js so I would assume content is being transmuxed as expected.

I debugged a bit and I ended up seeing that that the stream info for the audio is not being created, as it fails when trying to get the start time from the segment (https://github.com/google/shaka-player/blob/9584804b6cfad07aa4a44e4c7537ef6413c43194/lib/hls/hls_parser.js#L1783) because the mimeType is audio/aac and RAW AAC its not supported.

Manifest looks like

#EXTM3U
#EXT-X-VERSION:4
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-STREAM-INF:BANDWIDTH=471788,AVERAGE-BANDWIDTH=471788,RESOLUTION=640x360,FRAME-RATE=29.998,CODECS="avc1.64001E,mp4a.40.2",AUDIO="audio_0"
c7b82f632b604441a1b1504492806203/9884942bfb9d413da5d267f1dadd91d9/index_1.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=2114619,AVERAGE-BANDWIDTH=2114619,RESOLUTION=1920x1080,FRAME-RATE=29.998,CODECS="avc1.640032,mp4a.40.2",AUDIO="audio_0"
c7b82f632b604441a1b1504492806203/9884942bfb9d413da5d267f1dadd91d9/index_2.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1410466,AVERAGE-BANDWIDTH=1410466,RESOLUTION=1600x900,FRAME-RATE=29.998,CODECS="avc1.640028,mp4a.40.2",AUDIO="audio_0"
c7b82f632b604441a1b1504492806203/9884942bfb9d413da5d267f1dadd91d9/index_3.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=3235745,AVERAGE-BANDWIDTH=3235745,RESOLUTION=1920x1080,FRAME-RATE=29.998,CODECS="avc1.640032,mp4a.40.2",AUDIO="audio_0"
c7b82f632b604441a1b1504492806203/9884942bfb9d413da5d267f1dadd91d9/index_4.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=334640,AVERAGE-BANDWIDTH=334640,RESOLUTION=512x288,FRAME-RATE=29.998,CODECS="avc1.640015,mp4a.40.2",AUDIO="audio_0"
c7b82f632b604441a1b1504492806203/9884942bfb9d413da5d267f1dadd91d9/index_5.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=789715,AVERAGE-BANDWIDTH=789715,RESOLUTION=1024x576,FRAME-RATE=29.998,CODECS="avc1.64001F,mp4a.40.2",AUDIO="audio_0"
c7b82f632b604441a1b1504492806203/9884942bfb9d413da5d267f1dadd91d9/index_6.m3u8
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=149768,CODECS="avc1.64001E",RESOLUTION=640x360,URI="c7b82f632b604441a1b1504492806203/9884942bfb9d413da5d267f1dadd91d9/index_7.m3u8"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio_0",CHANNELS="2",NAME="Default",LANGUAGE="en",DEFAULT=YES,AUTOSELECT=YES,URI="c7b82f632b604441a1b1504492806203/9884942bfb9d413da5d267f1dadd91d9/index_8_0.m3u8"

I would like to understand if this is the expected behavior for this type of content or I'm missing something here.

[joeyparrish]

We can't support raw AAC directly at this time. If mux.js can transmux raw AAC, it's possible we could leverage that to support raw AAC in the future.

I'm going to mark this as an enhancement and rename it accordingly. Thanks!

[tareq-rezk]

@joeyparrish
you can't support on tizen only ? or on all platforms includes web ?

[jamescahall]

Checking in to see if this is being considered in the near future to be supported. When importing RTP streams in to AWS MediaLive, we can output video in to various flavors in TS output container. For supporting multiple audio tracks in different languages, however, the only option for output containers with AAC codecs is raw ".aac" segments. It does not appear too support aac audio transmuxed in to TS segments. All native players for various platforms (iOS/tvOS, Android/Android TV, Roku and several other HTML5 players fully support this). We have use cases where ShakaPlayer must be used (such as VIZIO TVs to support DRM) so this limits our ability to have multiple languages on live streams.

[joeyparrish]

I'm going to close this in favor of #2337 (direct support of raw AAC and other containerless formats), which is on our roadmap for the near term.