search

shaka-project / shaka-player-embedded

Shaka Player in a C++ Framework
Apache License 2.0
239 stars 63 forks source link

High-level design document #32

Open chrisfillmore opened 5 years ago

chrisfillmore commented 5 years ago

My team is going to look at shaka-player-embedded to try to get a sense of its capabilities. Currently we deliver video playback on iOS through Apple's native AVPlayer.

Something I haven't really seen in the docs (sorry if I've missed something) is a high-level overview of how shaka-player-embedded works. Are you bundling Chromium? Where does the video playback surface come from (i.e. the "video element")? Are there any known limitations compared to using Apple's native tools?

Any information is great. Thanks so much.

theodab commented 5 years ago

Well, compared to an AVPlayer, you won't have the same native AirPlay support. Because most of that stuff was hidden behind private APIs last I looked into it.

TheModMaker commented 5 years ago

We don't bundle Chromium, especially since I think it doesn't do video rendering on iOS. I think Chrome just uses the native web views since iOS has restrictions on HTML rendering. We use JavaScriptCore directly as the JavaScript engine and implement any of the required web APIs.

For video/audio playback, we use FFmpeg to demux/decode the content (using native iOS hardware acceleration) and draw the texture onto an AVImage. The C++ API returns the frame to the app to draw it as they see fit (see the demo). If you use the Objective-C/Swift API, it draws the image on the ShakaPlayerView. So the ShakaPlayerView (which is a UIView) acts as the video element. Audio is played using SDL on the default audio device (see #31).

We don't have AirPlay or FairPlay support and by default we only support H.264+AAC in MP4 (although you can change this with command-line flags). We also don't support direct playback of media files, we only support HLS or DASH.

But unlike AVPlayer, we also support playback of DASH content and clear-key encrypted content.

We should add a high-level design document.

chrisfillmore commented 5 years ago

Thanks for that info! We are definitely interested in DASH-WV support on iOS. I may come back with more questions if my team has some.