MPL audio parser error with audio tracks encapsulated by .ts container

GoogleCodeExporter commented 9 years ago

What steps will reproduce the problem?
1. Prepare HLS multiple audio tracks stream.
2. Launch custom receiver to play. --- mediaPlayer.load(protocol)
3. Switch to alternative audio track. --- protocol.enableStream(audioIndex, 
true); mediaPlayer.reload()
4. Javascript console error - HLS Host error 1.

What is the expected output? What do you see instead?
MPL should switch to the specified audio track. Now error popped up.

What version of the product are you using? On what operating system?
MPL 1.0.0.

Please provide any additional information below.
I suspect that it is because MPL doesn't support TS container audio track. MPL 
currently only support AAC container?
The stream is good on Mac/QuickTime 10.4.

Original issue reported on code.google.com by nicevinc...@gmail.com on 29 Mar 2015 at 3:30

Attachments:

[Screen Shot 2015-03-28 at 10.59.45 PM.png](https://storage.googleapis.com/google-code-attachments/google-cast-sdk/issue-549/comment-0/Screen Shot 2015-03-28 at 10.59.45 PM.png)

GoogleCodeExporter commented 9 years ago

Update: I even tested with aac file as alternative audio m3u8 stream, still 
getting the aac parser error.

Original comment by nicevinc...@gmail.com on 31 Mar 2015 at 1:41

GoogleCodeExporter commented 9 years ago

Update 2: I download the Apple bip-bop advanced stream and host it locally. 
Then use the custom receiver to switch to another audio track. Now getting 
another error.

==========
Uncaught QuotaExceededError: Failed to execute 'addSourceBuffer' on 
'MediaSource': This MediaSource has reached the limit of SourceBuffer objects 
it can handle. No additional SourceBuffer objects may be added.
==========

Original comment by nicevinc...@gmail.com on 31 Mar 2015 at 3:13

Attachments:

[Screen Shot 2015-03-31 at 11.11.42 AM.png](https://storage.googleapis.com/google-code-attachments/google-cast-sdk/issue-549/comment-2/Screen Shot 2015-03-31 at 11.11.42 AM.png)

GoogleCodeExporter commented 9 years ago

What is the format of your AAC file and does it start with the ADTS header or 
ID3 tag? Also, since your segment ends with ts, have you tried playing your 
manifest using cast.player.api.HlsSegmentFormat.MPEG2_TS? 
(https://developers.google.com/cast/docs/reference/player/cast.player.api.HlsSeg
mentFormat#.MPEG2_TS)

Original comment by jonathan...@google.com on 1 Apr 2015 at 5:33

Changed state: NeedInfo

GoogleCodeExporter commented 9 years ago

#1 For TS container:
I take your suggestion to explicitly to set the MPEG2_TS format, no luck... So 
I attached the console log. Also note that this is LIVE stream.

protocol = cast.player.api.CreateHlsStreamingProtocol(host, 
cast.player.api.HlsSegmentFormat.MPEG2_TS);

#2 For aac container:
I reviewed our stream, and found out that our aac wrapper is not correct. So I 
need to get this fix then test again. However, I don't want to deliver the AAC 
stream since it has A/V out of sync for other platforms.

Original comment by nicevinc...@gmail.com on 1 Apr 2015 at 7:03

Attachments:

multiple_audio_ts_container.txt

GoogleCodeExporter commented 9 years ago

Since your ts segment is in TS format (not MPEG2_ES), you have to use MPL like 
this:

    this.host_ = new cast.player.api.Host({
      'mediaElement': mediaElement,
      'url': URL_OF_MANIFEST
    });

    this.player_ = new cast.player.api.Player(this.host_);

    // You're calling the API like this: cast.player.api.CreateHlsStreamingProtocol(this.host_, cast.player.api.HlsSegmentFormat.MPEG_AUDIO_ES);
    // you shouldn't pass cast.player.api.HlsSegmentFormat.MPEG_AUDIO_ES
    this.protocol_ = cast.player.api.CreateHlsStreamingProtocol(this.host_);
    this.player_.load(this.protocol_, initialTime);

Original comment by jonathan...@google.com on 2 Apr 2015 at 6:58

GoogleCodeExporter commented 9 years ago

I tried both ways, actually 3, none of these is working.
1. this.protocol_ = cast.player.api.CreateHlsStreamingProtocol(this.host_);
2. this.protocol_ = cast.player.api.CreateHlsStreamingProtocol(this.host_, 
cast.player.api.HlsSegmentFormat.MPEG2_TS);
3. this.protocol_ = cast.player.api.CreateHlsStreamingProtocol(this.host_, 
cast.player.api.HlsSegmentFormat.MPEG_AUDIO_ES);

Original comment by nicevinc...@gmail.com on 2 Apr 2015 at 7:10

GoogleCodeExporter commented 9 years ago

Can you provide your manifest file for us to test?

Original comment by jonathan...@google.com on 2 Apr 2015 at 9:00

GoogleCodeExporter commented 9 years ago

here you go

http://nlds159.cdnak.neulion.com/nldsu/univision/as/live/uni_hd_ipad.m3u8

Original comment by nicevinc...@gmail.com on 2 Apr 2015 at 9:08

GoogleCodeExporter commented 9 years ago

Thanks, I'll let you know if I need any additional information.

Original comment by jonathan...@google.com on 2 Apr 2015 at 10:20

Changed state: Acknowledged

GoogleCodeExporter commented 9 years ago

Hi,
Any updates? Just want to know it is the video stream issue or MPL issue? 
Thanks.

Original comment by nicevinc...@gmail.com on 9 Apr 2015 at 4:13

GoogleCodeExporter commented 9 years ago

We are still investigating the issue.

Original comment by jonathan...@google.com on 9 Apr 2015 at 6:37

GoogleCodeExporter commented 9 years ago

When using an alternative audio track on Chromecast, the video and audio tracks 
should be provided as separate files. Your ts segment had both audio and video. 
Please make sure to include audio and video as separate files.

Original comment by jonathan...@google.com on 11 Apr 2015 at 12:40

Changed state: Invalid

GoogleCodeExporter commented 9 years ago

If you look at the spec, it is not necessary to separate the audio and video 
files. You can combine audio and video into one ts file as default 'Rendition' 
if URI attribute is missing.

======
https://tools.ietf.org/html/draft-pantos-http-live-streaming-14#section-4.3.4.2.
1

   If the media type is VIDEO or AUDIO, a missing URI attribute
   indicates that the media data for this Rendition is included in the
   Media playlist of any EXT-X-STREAM-INF tag referencing this MEDIA
   tag.  If the media type is AUDIO and the URI attribute is missing,
   clients MAY assume that the audio data for this Rendition is present
   in every video Rendition specified by the EXT-X-STREAM-INF tag.
======

Original comment by nicevinc...@gmail.com on 11 Apr 2015 at 2:45

GoogleCodeExporter commented 9 years ago

[deleted comment]

GoogleCodeExporter commented 9 years ago

MPL supports muxed video+audio in the default rendition specified via 
EXT-X-STREAM-INF.  But for EXT-X-MEDIA:TYPE=AUDIO, the ts stream should only 
have audio.  Is this the case with your streams?

Original comment by vadi...@google.com on 13 Apr 2015 at 10:22

GoogleCodeExporter commented 9 years ago

The alternative audio m3u8 (EXT-X-MEDIA:TYPE=AUDIO) contains audio samples only.

If you grab one ts segment from the audio m3u8:
http://nlds159.cdnak.neulion.com/nldsu/univision/as/live/uni_audio_eng_ipad.m3u8

Use a tool to parse ts segment (VLC with command + i, or ffmpeg), you will see 
the segment contains audio samples only.

==================================
vincent@Yuanshengs-MacBook-Pro:~/temp $ ffprobe uni_audio_eng_20150413234610.ts 
ffprobe version 2.4.4 Copyright (c) 2007-2014 the FFmpeg developers
  built on Feb  1 2015 00:45:59 with Apple LLVM version 6.0 (clang-600.0.56) (based on LLVM 3.5svn)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/2.4.4 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-nonfree --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-libx264 --enable-libfaac --enable-libmp3lame --enable-libxvid --enable-vda
  libavutil      54.  7.100 / 54.  7.100
  libavcodec     56.  1.100 / 56.  1.100
  libavformat    56.  4.101 / 56.  4.101
  libavdevice    56.  0.100 / 56.  0.100
  libavfilter     5.  1.100 /  5.  1.100
  libavresample   2.  1.  0 /  2.  1.  0
  libswscale      3.  0.100 /  3.  0.100
  libswresample   1.  1.100 /  1.  1.100
  libpostproc    53.  0.100 / 53.  0.100
Input #0, mpegts, from 'uni_audio_eng_20150413234610.ts':
  Duration: 00:00:09.85, start: 80872.494444, bitrate: 137 kb/s
  Program 1 
    Stream #0:0[0xc0]: Audio: aac ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 128 kb/s

Original comment by nicevinc...@gmail.com on 13 Apr 2015 at 11:53

GoogleCodeExporter commented 9 years ago

Even though you can combine audio and video in the default Rendition, MPL does 
not support swapping a track that has both audio and video with an audio-only 
track. The main track needs to have separate audio and video files.

Original comment by jonathan...@google.com on 16 Apr 2015 at 1:31

GoogleCodeExporter commented 9 years ago

For alternative audio (muxed video and audio in main track),
1. iPad Safari HTML5 supports it.
2. Mac Desktop QuickTime supports it.
3. MPL is inconsistent with the spec, so we need to maintain additional m3u8(s) 
just for alternative audio feature, and need to double the storage for video 
content for all the bitrates. And this stream is only dedicated to Chromecast, 
because for those not alternative audio capable devices, we don't want the user 
to experience silent video.

Original comment by nicevinc...@gmail.com on 16 Apr 2015 at 3:44

GoogleCodeExporter commented 9 years ago

Sorry about all the confusion on this issue.  MPL supports a main rendition 
where video and audio are muxed and supplementary audio is in an audio only ts. 
 When the audio only stream is enabled in addition to the main rendition, MPL 
should use the video from the main rendition and audio from the other enabled 
stream.  It is not the expected behavior that this doesn't work for you.   We 
will investigate using your repro stream.

Original comment by vadi...@google.com on 16 Apr 2015 at 4:07

GoogleCodeExporter commented 9 years ago

You can make this work with MPL as is today by changing your supplementary 
audio streams to be a raw AAC audio stream (not in a TS).  So your main 
rendition will stay TS with muxed video+audio, but supplementary should become 
MPEG4 audio ES format, i.e. raw AAC file.  This should work in other HLS 
players as well as with MPL on Chromecast.

Original comment by vadi...@google.com on 16 Apr 2015 at 7:17

GoogleCodeExporter commented 9 years ago

Thanks for the following up. We will try to test with AAC stream once have the 
media server up. And will let you know if we see any problems.

At the meantime, in the next release, will MPL change to support TS container 
for supplementary audio?

Original comment by nicevinc...@gmail.com on 17 Apr 2015 at 3:20

GoogleCodeExporter commented 9 years ago

We set up another AAC stream here:
http://nlds159.cdnak.neulion.com/nldsu/univision/as/live/uni_hd_ced.m3u8

I couldn't make it work. There's an exception in the console. I attached the 
log and screenshot.

Here's what my code did:
1. Load the stream from sender (supplementary audio is AAC): 
http://nlds159.cdnak.neulion.com/nldsu/univision/as/live/uni_hd_ced.m3u8

2. Create the StreamingProtocol object:
protocol = cast.player.api.CreateHlsStreamingProtocol(host);

3. After the video content showed up:
protocol.enableStream(1, true);
mediaPlayer.reload();

4. Exception:
2015-04-17 12:59:14.397 media_player.js:83 Uncaught InvalidStateError: Failed 
to read the 'buffered' property from 'SourceBuffer': This SourceBuffer has been 
removed from the parent media source.media_player.js:83 Jd.Yfmedia_player.js:28 
ecmedia_player.js:30 acmedia_player.js:26 (anonymous function)

P.S.
Here's the information printed out by calling 
JSON.stringify(protocol.getStreamInfo(stream_index))
=====
    2015-04-17 12:57:55.968 VM297:6 stream 0 {
      "codecs": "avc1.4d401e,mp4a.40.2",
      "mimeType": "video/mp2t",
      "bitrates": [
        800000,
        1600000,
        3000000,
        4500000
      ],
      "language": null,
      "name": null
    }
    2015-04-17 12:57:55.974 VM297:6 stream 1 {
      "codecs": "mp4a.40.2",
      "mimeType": "audio/mp4",
      "bitrates": [
        0
      ],
      "language": "eng",
      "name": "English"
    }
    2015-04-17 12:57:55.978 VM297:6 stream 2 {
      "codecs": "mp4a.40.2",
      "mimeType": "audio/mp4",
      "bitrates": [
        0
      ],
      "language": "por",
      "name": "Portuguese"
    }
===

Original comment by nicevinc...@gmail.com on 17 Apr 2015 at 6:34

Attachments:

[Screen Shot 2015-04-17 at 1.04.10 PM.png](https://storage.googleapis.com/google-code-attachments/google-cast-sdk/issue-549/comment-22/Screen Shot 2015-04-17 at 1.04.10 PM.png)
cast_1.log

GoogleCodeExporter commented 9 years ago

I also tried another constructor to create protocol object:
cast.player.api.CreateHlsStreamingProtocol(host, 
cast.player.api.HlsSegmentFormat.MPEG_AUDIO_ES);

Now it is worse, because the DEFAULT rendition cannot be played successfully. 
In short, based on my test, the alternative audio feature is completely broken.

2015-04-17 13:07:29.849 media_player.js:81 Uncaught NotSupportedError: Failed 
to execute 'addSourceBuffer' on 'MediaSource': The type provided ('audio/mp4; 
codecs="avc1.4d401e,mp4a.40.2"') is unsupported.media_player.js:81 
h.createBuffermedia_player.js:80 Jdmedia_player.js:94 iemedia_player.js:157 
h.updatemedia_player.js:155 h.onManifestReadymedia_player.js:178 
Y.onManifestReadymedia_player.js:120 Le.pbmedia_player.js:67 
h.ucmedia_player.js:68 zd.ucmedia_player.js:67 h.Ycmedia_player.js:32 
jcmedia_player.js:31 h.dispatchEventmedia_player.js:40 Ecmedia_player.js:38 
E.Zgmedia_player.js:38 E.Uf.

Original comment by nicevinc...@gmail.com on 17 Apr 2015 at 6:37

Attachments:

[Screen Shot 2015-04-17 at 1.08.23 PM.png](https://storage.googleapis.com/google-code-attachments/google-cast-sdk/issue-549/comment-23/Screen Shot 2015-04-17 at 1.08.23 PM.png)
cast_2.log

GoogleCodeExporter commented 9 years ago

Thank you for trying this out.  Unfortunately, despite the fact that MPL tries 
to create a source buffer with a video only codec for the main rendition, 
Chrome audio pipeline doesn't currently support this when the main rendition is 
muxed video+audio.

I believe the best you can do is something like this:

http://hls.ted.com/talks/1954.m3u8

i.e. the main rendition has a video only TS and there are supplementary audio 
streams.  Again, sorry about all the confusion on this.

Original comment by vadi...@google.com on 17 Apr 2015 at 10:23

GoogleCodeExporter commented 9 years ago

The provided stream doesn't work for me because CORS problem for the aac file.
    http://tedcdnpb-a.akamaihd.net/bumpers/hls/audio/in/Ripple-600k.aac

Anyway... And you are saying this MediaSource problem, is it possible to get it 
fixed in the next Chromecast kernel update?

Original comment by nicevinc...@gmail.com on 17 Apr 2015 at 11:21

GoogleCodeExporter commented 9 years ago

Hi there,

I did take off the audio track from main stream. Now this stream feed only 
contains video only for main track.

http://nlds159.cdnak.neulion.com/nldsu/univision/as/live/uni_hd_ced.m3u8

Now, I got 'metadata' error problem. Log attached.

Original comment by nicevinc...@gmail.com on 20 Apr 2015 at 6:47

Attachments:

audio_only.log

GoogleCodeExporter commented 9 years ago

Sorry, that was because our main m3u8 output is incorrect for the additional 
audios. Now it is working if we separate the video and audio.

Original comment by nicevinc...@gmail.com on 20 Apr 2015 at 9:35

gingerbeur / google-cast-sdk

MPL audio parser error with audio tracks encapsulated by .ts container #549