search

shaka-project / shaka-player-embedded

Shaka Player in a C++ Framework
Apache License 2.0
239 stars 62 forks source link

Support MPEG2-TS media #33

Open nedmundwong opened 5 years ago

nedmundwong commented 5 years ago

I'm trying to use the sample_xcode_project.xcodeproj to play some HLS videos with TS fragment. But after trying with many HLS testing videos (e.g. https://bitdash-a.akamaihd.net/content/MI201109210084_1/m3u8s/f08e80da-bf1d-4e3d-8899-f0f6155f6efa.m3u8 ), the player keep showing : Shaka Error MANIFEST.CONTENT_UNSUPPORTED_BY_BROWSER ()

However, when using the browser version of Shaka Player

(https://shaka-player-demo.appspot.com/demo/#asset=https://bitdash-a.akamaihd.net/content/MI201109210084_1/m3u8s/f08e80da-bf1d-4e3d-8899-f0f6155f6efa.m3u8;lang=en-US;videoRobustness=http://192.168.18.193:8111/playlist.m3u8;audioRobustness=http://192.168.18.193:8111/playlist.m3u8;build=uncompiled)

The HLS video can be played. Hence I'm not sure if the problem is related to the player or that particular m3u8 file format is not supported. If the problem is related to the latter one, could you please provide some m3u8 files that can be played for this player. Thanks in advance!

TheModMaker commented 5 years ago

We currently only support MP4 and WebM (WebM isn't added by default on iOS). We usually just forward things to FFmpeg, so there is a slim chance of it working somewhat by just adding a flag to configure:

./configure --containers mov,mpegts

You'll need to edit the build.sh file in the Xcode project and delete the ShakaPlayerEmbedded.framework folder to get it work in the sample project.

The HLS+MP4 Angel One clips should work fine: https://storage.googleapis.com/shaka-demo-assets/angel-one-hls/hls.m3u8

nedmundwong commented 5 years ago

Many thanks for your prompt reply! The clip that you provide works fine.

I've tried your method by adding flag to configure , and changed the framework in the .xcodeproj Although the error Shaka Error MANIFEST.CONTENT_UNSUPPORTED_BY_BROWSER () is still shown, but by checking the log, it seems something different:

When loading (http://184.72.239.149/vod/smil:BigBuckBunny.smil/playlist.m3u8)

[Log]: "Unload latency:" 0.004 [Log]: "Guessing stream type for" "#EXT-X-STREAM-INF:BANDWIDTH=\"560000\",RESOLUTION=\"560x320\",URI=\"chunklist_w847761226_b560000.m3u8\"" [Log]: "Guessing multiplexed audio+video." [Log]: "Guessing stream type for" "#EXT-X-STREAM-INF:BANDWIDTH=\"280000\",RESOLUTION=\"280x160\",URI=\"chunklist_w847761226_b280000.m3u8\"" [Log]: "Guessing multiplexed audio+video." [Log]: "First segment" "media_w847761226_b560000_0.ts" "starts at" 0 [Log]: "First segment" "media_w847761226_b280000_0.ts" "starts at" 0 [Log]: "Dropping variant - video not compatible with platform" "type=video codecs=avc1.42E01E,mp4a.40.2 bandwidth=undefined frameRate=undefined width=560 height=320" [Log]: "Dropping variant - video not compatible with platform" "type=video codecs=avc1.42E01E,mp4a.40.2 bandwidth=undefined frameRate=undefined width=280 height=160" [Log]: "load() failed:" {category:4, code:4032, data:[...], handled:false, message:"Shaka Error MANIFEST.CONTENT_UNSUPPORTED_BY_BROWSER ()", severity:2, stack:"Error@ ...

When using previous framework, in the Log file, the player cannot even find the first ts segment. After changing the framework, it can detect the ts and try to load it. However, when the player loading the ts, it always prompt the error "Dropping variant - video not compatible with platform" "type=video codecs=avc1.42E01E,mp4a.40.2 bandwidth=undefined frameRate=undefined width=560 height=320" where only the width and height is loaded correctly.

Does it mean there is no chance to play TS on this embedded player?

TheModMaker commented 5 years ago

That content is multiplexed, meaning the media files contain both audio and video streams. Shaka Player doesn't officially support that since browser support is inconsistent and not really aligned with streaming content where audio and video are usually separate. But Shaka Player will just add the segments and leave it to the browser.

Shaka Player Embedded doesn't support multiplexed content at all. You'll need to convert your content to have audio and video in separate streams. We could add support for multiplexed content at some point, but it breaks a lot of assumptions in the media pipeline now.

nedmundwong commented 5 years ago

Thank you @TheModMaker for the explanation. And I've tried to separate the content to have audio and video in separate streams. But seems the player still got into some problems:

I've followed the tutorial, and tried to use the shaka packager to encode the video content. Here are the basic information:

2 single track TS streams (1 video, 1 audio)

Shaka Packager Command:

packager \
  'in=small.mp4,stream=audio,segment_template=$Number$a.ts,playlist_name=maina.m3u8,hls_group_id=audio,hls_name=ENGLISH' \
  'in=small.mp4,stream=video,segment_template=$Number$v.ts,playlist_name=mainv.m3u8' \
  --hls_master_playlist_output h264_master.m3u8

Result: Error

[Log]: "Unload latency:" 0.004 [Log]: "First segment" "1a.ts" "starts at" 0.1 [Log]: "Guessing stream type for" "#EXT-X-STREAM-INF:BANDWIDTH=\"646562\",AVERAGE-BANDWIDTH=\"646562\",CODECS=\"avc1.42c01e,mp4a.40.2\",RESOLUTION=\"560x320\",AUDIO=\"audio\",URI=\"mainv.m3u8\"" [Log]: "Guessing video." [Log]: "First segment" "1v.ts" "starts at" 0.1 [Log]: "Dropping variant - audio not compatible with platform" "type=audio codecs=mp4a.40.2 bandwidth=undefined channelsCount=1" [Log]: "load() failed:" {category:4, code:4032, data:[...], handled:false, message:"Shaka Error MANIFEST.CONTENT_UNSUPPORTED_BY_BROWSER ()", severity:2, stack:"Error@...

1 single track TS stream (1 video)

packager \
  'in=small.mp4,stream=video,segment_template=$Number$v.ts,playlist_name=mainv.m3u8' \
  --hls_master_playlist_output h264_master.m3u8

Result: Error

[Log]: "Unload latency:" 0.005 [Log]: "Guessing stream type for" "#EXT-X-STREAM-INF:BANDWIDTH=\"505562\",AVERAGE-BANDWIDTH=\"505562\",CODECS=\"avc1.42c01e\",RESOLUTION=\"560x320\",URI=\"mainv.m3u8\"" [Log]: "Guessing video-only." [Log]: "First segment" "1v.ts" "starts at" 0.1 [Log]: "Dropping variant - video not compatible with platform" "type=video codecs=avc1.42c01e bandwidth=undefined frameRate=undefined width=560 height=320" [Log]: "load() failed:" {category:4, code:4032, data:[...], handled:false, message:"Shaka Error MANIFEST.CONTENT_UNSUPPORTED_BY_BROWSER ()", severity:2, stack:"Error@...

2 groups of streams (each with an init segment and a series of media segments) (MP4 Output)

Shaka Packager Command:

packager \
  'in=small.mp4,stream=audio,init_segment=audio/init.mp4,segment_template=audio/$Number$.m4s,playlist_name=audio/main.m3u8,hls_group_id=audio,hls_name=ENGLISH' \
  'in=small.mp4,stream=video,init_segment=h264_480p/init.mp4,segment_template=h264_480p/$Number$.m4s,playlist_name=h264_480p/main.m3u8,iframe_playlist_name=h264_480p/iframe.m3u8' \
  --hls_master_playlist_output h264_master.m3u8

Result: Played Successfully

2 single file MP4 streams (1 video, 1 audio)

packager \
  in=small.mp4,stream=audio,output=audio.mp4,playlist_name=audio.m3u8,hls_group_id=audio,hls_name=ENGLISH \
  in=small.mp4,stream=video,output=h264_480p.mp4,playlist_name=h264_480p.m3u8 \
  --hls_master_playlist_output h264_master.m3u8

Result: Played Successfully

From the above results, we can see that TS streams still could not be played after separating audio and video or only playing the video stream. The contents that could be played are m4s in m3u8 and the single mp4 streams. However, when I tried to play the above streams from the shaka player demo website, all streams could be played successfully. I'm not sure if I used the shaka packager with the wrong command or if the player could not support separated audio and video streams with TS as segments.

TheModMaker commented 5 years ago

One thing I can think of is the video is too small to get hardware decoding. By default on iOS, we'll only play videos we can get hardware decoding on. (I also just realized there isn't a flag to disable force hardware decoding, but there is a flag to enable it) So try adding the --gn-args force_hardware_decode=false flag. I think you can also set the environment variable GLOG_v=1 when running to get verbose logging.

nedmundwong commented 5 years ago

I've used the flag to configure and build the framework again. After added the environment variable, more information was provided for the above issue:

When playing this HLS (https://bitdash-a.akamaihd.net/content/MI201109210084_1/m3u8s/f08e80da-bf1d-4e3d-8899-f0f6155f6efa.m3u8), still prompted the same error Shaka Error MANIFEST.CONTENT_UNSUPPORTED_BY_BROWSER () The Log is as follow:

[Log]: "Unload latency:" 0.004 [Log]: "First segment" "segment_0.ts" "starts at" 0 [Log]: "Guessing stream type for" "#EXT-X-STREAM-INF:BANDWIDTH=\"628000\",CODECS=\"avc1.42c00d,mp4a.40.2\",RESOLUTION=\"320x180\",AUDIO=\"audio\",URI=\"f08e80da-bf1d-4e3d-8899-f0f6155f6efa_video_180_250000.m3u8\"" [Log]: "Guessing video." [Log]: "First segment" "segment_0.ts" "starts at" 0 [Log]: "Guessing stream type for" "#EXT-X-STREAM-INF:BANDWIDTH=\"1728000\",CODECS=\"avc1.42c00d,mp4a.40.2\",RESOLUTION=\"640x360\",AUDIO=\"audio\",URI=\"f08e80da-bf1d-4e3d-8899-f0f6155f6efa_video_360_800000.m3u8\"" [Log]: "Guessing video." [Log]: "First segment" "segment_0.ts" "starts at" 0 [Log]: "Guessing stream type for" "#EXT-X-STREAM-INF:BANDWIDTH=\"928000\",CODECS=\"avc1.42c00d,mp4a.40.2\",RESOLUTION=\"480x270\",AUDIO=\"audio\",URI=\"f08e80da-bf1d-4e3d-8899-f0f6155f6efa_video_270_400000.m3u8\"" [Log]: "Guessing video." [Log]: "First segment" "segment_0.ts" "starts at" 0 [Log]: "Guessing stream type for" "#EXT-X-STREAM-INF:BANDWIDTH=\"2528000\",CODECS=\"avc1.42c00d,mp4a.40.2\",RESOLUTION=\"960x540\",AUDIO=\"audio\",URI=\"f08e80da-bf1d-4e3d-8899-f0f6155f6efa_video_540_1200000.m3u8\"" [Log]: "Guessing video." [Log]: "First segment" "segment_0.ts" "starts at" 0 [Log]: "Guessing stream type for" "#EXT-X-STREAM-INF:BANDWIDTH=\"4928000\",CODECS=\"avc1.42c00d,mp4a.40.2\",RESOLUTION=\"1280x720\",AUDIO=\"audio\",URI=\"f08e80da-bf1d-4e3d-8899-f0f6155f6efa_video_720_2400000.m3u8\"" [Log]: "Guessing video." [Log]: "First segment" "segment_0.ts" "starts at" 0 [Log]: "Guessing stream type for" "#EXT-X-STREAM-INF:BANDWIDTH=\"9728000\",CODECS=\"avc1.42c00d,mp4a.40.2\",RESOLUTION=\"1920x1080\",AUDIO=\"audio\",URI=\"f08e80da-bf1d-4e3d-8899-f0f6155f6efa_video_1080_4800000.m3u8\"" [Log]: "Guessing video." [Log]: "First segment" "segment_0.ts" "starts at" 0.08 [Log]: "First segment" "segment_0.ts" "starts at" 0.08 [Log]: "First segment" "segment_0.ts" "starts at" 0.08 [Log]: "First segment" "segment_0.ts" "starts at" 0.08 [Log]: "First segment" "segment_0.ts" "starts at" 0.08 [Log]: "First segment" "segment_0.ts" "starts at" 0.08 WARNING: Logging before InitGoogleLogging() is written to STDERR I0208 11:15:07.781924 129216512 media_utils.cc:138] Container 'mp2t' (normalized from 'mp2t') is not supported [Log]: "Dropping variant - audio not compatible with platform" "type=audio codecs=mp4a.40.2 bandwidth=undefined channelsCount=null" I0208 11:15:07.782481 129216512 media_utils.cc:138] Container 'mp2t' (normalized from 'mp2t') is not supported [Log]: "Dropping variant - audio not compatible with platform" "type=audio codecs=mp4a.40.2 bandwidth=undefined channelsCount=null" I0208 11:15:07.782693 129216512 media_utils.cc:138] Container 'mp2t' (normalized from 'mp2t') is not supported [Log]: "Dropping variant - audio not compatible with platform" "type=audio codecs=mp4a.40.2 bandwidth=undefined channelsCount=null" I0208 11:15:07.782980 129216512 media_utils.cc:138] Container 'mp2t' (normalized from 'mp2t') is not supported [Log]: "Dropping variant - audio not compatible with platform" "type=audio codecs=mp4a.40.2 bandwidth=undefined channelsCount=null" I0208 11:15:07.783174 129216512 media_utils.cc:138] Container 'mp2t' (normalized from 'mp2t') is not supported [Log]: "Dropping variant - audio not compatible with platform" "type=audio codecs=mp4a.40.2 bandwidth=undefined channelsCount=null" I0208 11:15:07.783499 129216512 media_utils.cc:138] Container 'mp2t' (normalized from 'mp2t') is not supported [Log]: "Dropping variant - audio not compatible with platform" "type=audio codecs=mp4a.40.2 bandwidth=undefined channelsCount=null" [Log]: "load() failed:" {category:4, code:4032, data:[...], handled:false, message:"Shaka Error MANIFEST.CONTENT_UNSUPPORTED_BY_BROWSER ()", severity:2, stack:"Error@/Users/...

After several trials with different HLS sources, it keeps showing: media_utils.cc:138] Container 'mp2t' (normalized from 'mp2t') is not supported, after some Googling in the Internet, it seems related to media_utils.cc. Not sure if the shaka player requires to check browser compatibility with the ts files before playing.

TheModMaker commented 5 years ago

Note that there is no browser here (or more accurately Shaka Player Embedded is the browser). You are correct about a "media_utils" file, but it is actually here. I looked a bit at this and we definately will need to make some changes for this to work. There is some weirdness about segment appending that we'll need to fix.

Romantic-LiXuefeng commented 4 years ago

@TheModMaker Is there any plan to support this issue?

TheModMaker commented 4 years ago

We do plan on supporting it eventually, but we're not working on it yet. This is in the v1.1 milestone, so we'll start working on it after the v1.0 release is done.

YutaMoriJP commented 6 months ago

@TheModMaker I was having issues with MPEG2-TS and wasn't sure whether Shaka Player now supports it or not. Is mux.js needed or can it be used without it?