search

google / ExoPlayer

This project is deprecated and stale. The latest ExoPlayer code is available in https://github.com/androidx/media
https://developer.android.com/media/media3/exoplayer
Apache License 2.0
21.7k stars 6.02k forks source link

Read data from CodecPrivate when playing AVC or HEVC #7267

Closed steffenmanden closed 4 years ago

steffenmanden commented 4 years ago

[REQUIRED] Use case description

It has recently been decided in the mkv standard to add support for extra data in the codecprivate area when playing AVC or HEVC content.

This allows Dolby Vision data, as well as MVC data, to be properly read during playback and thus enable Dolby Vision support and more.

Proposed solution It would be great it Exoplayer could be made possible to playback using this data allowing monitors to utilize Dolby Vision data.

The create of MakeMKV has allowed contact to him for easy explaination of how the data is stored and implemented

Alternatives considered Workarounds are currently used through mp4 and ts files - however they lack a lot of features regarding subtitles, chapters and more! Directly play from the most common format (mkv) would make it super easy to use for the average user.

kim-vde commented 4 years ago

Is it standard to have Dolby Vision streams inside MKV? It is not listed in the Matroska codec IDs.

steffenmanden commented 4 years ago

It's not in the standard definition. Dolby Vision has not previously been in the standard.

This implementation extends the format to allow for Dolby Vision within this format. Currently solutions (mp4 and ts) however have the standard, however are limited in a lot of features.

So although its not a part of the standard of mkv, it is an extention of the standard which will improve the player capabilities in a massive way without going out of bounds in a massive way.

MKV is after all the most used format :)

kim-vde commented 4 years ago

Ok. I have flagged the issue as a low-priority enhancement for now. Thanks for reporting it.

steffenmanden commented 4 years ago

Thanks :) - referenced the MakeMKV creator to comment in here, as he can give more details as to how the format was extended! Hopefully he will comment

MikeChenMM commented 4 years ago

I'm author of MakeMKV. For the record, I 'm willing to provide any help to make this happen.

Now, few points: "Matroska standard" is de-facto non-existent thing and is very outdated. :( Dolby Vision is not a separate codec, but rather a (HEVC) stream extension - DV data (EL+RPU) are encoded in separate NAL units, according to Dolby spec. The presence of DV data can be checked by several different methods - most obvious are: (1) DV descriptor in codecprivate data (there is an "standard" of storing additional elements in codec private for AVC and HEVC streams, that was used for ages since 3D MVC video) or (2)presence of DV RPU unit in first frame.

And, given that DV data is just a raw data that has to be handled to decoder and is already stored in decoder-ready format, it would take your developer literally ONE HOUR OF TIME to enable playback of such MKV files (assuming you can play back DV from other containers).

cogira commented 4 years ago

Really very interested in this. Hope that will soon be implemented in the exoplayer. Thanks.

Grencola commented 4 years ago

1000% YES!! If you need any assistance at all I will gladly help make this happen!

pervin11111 commented 4 years ago

Honestly, I would love to see this implementation. I am so used to packing chapters, subtitles in MKV.

ddogishere commented 4 years ago

I'm also registering my interest as this will be epic Thx

kcxserver commented 4 years ago

yes it would be wonderful if the exoplayer could play dv in mkv! 😍

doctorsirius commented 4 years ago

It's also important to note that current support of DV on .ts files doesn't work combined with the use of PGS subtitles, this should be also tested while it is implemented. It works for HDR but not for DV.

Wampaa commented 4 years ago

Hi @MikeChenMM thanks for your work on MakeMKV and on DolbyVision support!

"Matroska standard" is de-facto non-existent thing and is very outdated. :(

The Matroska specifications are an ongoing effort by the cellar working group at the IETF, in its current state it is composed of:

The working group is on GitHub and accepts contributions https://github.com/cellar-wg/matroska-specification

If changes (or clarifications) in the specifications are needed for this particular problem it would be good to propose them!

steffenmanden commented 4 years ago

Hi @MikeChenMM thanks for your work on MakeMKV and on DolbyVision support!

"Matroska standard" is de-facto non-existent thing and is very outdated. :(

The Matroska specifications are an ongoing effort by the cellar working group at the IETF, in its current state it is composed of:

The working group is on GitHub and accepts contributions https://github.com/cellar-wg/matroska-specification

If changes (or clarifications) in the specifications are needed for this particular problem it would be good to propose them!

And they now agreed on a solution for saving data in codecprivate :-) https://github.com/cellar-wg/matroska-specification/pull/377

@kim-vde the changes now allows for more usage than just Dolby Vision. This includes MVC data.

As you can see it is in final approval to be added to the standard.

Since this adds more functionality than just Dolby Vision, shouldnt this justify an increase in the priority ?

kim-vde commented 4 years ago

Feel free to make a pull request to add this feature and we will be happy to review it. If you do so, please include tests with a minimal test asset. Thanks!

ElegyD commented 4 years ago

https://github.com/cellar-wg/matroska-specification/pull/390 has been merged and work towards this issue can be started.

willtrking commented 4 years ago

I'm interested in working on this, and made changes to MatroskaExtractor locally to support Dolby Vision data in BlockAdditionMapping per the final spec here (https://github.com/cellar-wg/matroska-specification/pull/390). It seems like since the dvvc and dvcc is already supported for mp4's, all that's needed is to modify MatroskaExtractor to properly extract a DolbyVisionConfig and pass the relevant information to com.google.android.exoplayer2.Format.

Working on getting a small sample MKV to test this with before submitting a PR. On that note, @MikeChenMM you wouldn't happen to have a small test MKV in the finished spec, would you?

kim-vde commented 4 years ago

I will close this issue as the corresponding pull request has been merged.