androidx / media

Jetpack Media3 support libraries for media use cases, including ExoPlayer, an extensible media player for Android
https://developer.android.com/media/media3
Apache License 2.0
1.73k stars 413 forks source link

OverlayShaderProgram does not support HDR colors yet #723

Open steelbytes opened 1 year ago

steelbytes commented 1 year ago

Compiled and installed demo-transformer onto my Pixel 7 Pro.
Selected a video recorded on my pixel. Choose video effect overlay logo & timer. Clicked export. Failed.

VERY surprised you don't support videos created on your own phone

Caused by: java.lang.IllegalArgumentException: OverlayShaderProgram does not support HDR colors yet.
                                                               at androidx.media3.common.util.Assertions.checkArgument(Assertions.java:55)
                                                               at androidx.media3.effect.OverlayShaderProgram.<init>(OverlayShaderProgram.java:58)
                                                               at androidx.media3.effect.OverlayEffect.toGlShaderProgram(OverlayEffect.java:44)
                                                               at androidx.media3.effect.OverlayEffect.toGlShaderProgram(OverlayEffect.java:27)
                                                               at androidx.media3.effect.DefaultVideoFrameProcessor.getGlShaderProgramsForGlEffects(DefaultVideoFrameProcessor.java:707)
                                                               at androidx.media3.effect.DefaultVideoFrameProcessor.createOpenGlObjectsAndFrameProcessor(DefaultVideoFrameProcessor.java:606)
                                                               at androidx.media3.effect.DefaultVideoFrameProcessor.access$100(DefaultVideoFrameProcessor.java:70)
                                                               at androidx.media3.effect.DefaultVideoFrameProcessor$Factory.lambda$create$0$androidx-media3-effect-DefaultVideoFrameProcessor$Factory(DefaultVideoFrameProcessor.java:258)
                                                               at androidx.media3.effect.DefaultVideoFrameProcessor$Factory$$ExternalSyntheticLambda0.call(Unknown Source:20)
                                                               at java.util.concurrent.FutureTask.run(FutureTask.java:264)
                                                               at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
                                                               at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644)
                                                               at java.lang.Thread.run(Thread.java:1012)
droid-girl commented 1 year ago

OverlayShaderProgram does not currently support HDR colors. You can use setHdrMode(HDR_MODE_TONE_MAP_HDR_TO_SDR_USING_OPEN_GL) to tone-map your HDR video to SDR

steelbytes commented 1 year ago

thanks. probably should update the sample app

steelbytes commented 1 year ago

tried that setting (found it was unhelpfully hidden in demo app) and tried both the opengl and the mediacodec choices and both failed to make a video resembles the original.

so still a no go for me and my needs :-(

dway123 commented 1 year ago

Unfortunately, tone-mapping does involve converting HDR to SDR, which also removes the extra brightness information. Until OverlayShaderProgram supports HDR colors, I think tone-mapping and applying an overlay are the best Transformer can do out of the box. You could also implement your own custom Overlay implementation that supports HDR overlays, to avoid needing to tone map.

Adding HDR support for overlays isn't currently planned, but when we do add that support, I suspect your needs could be met through that too.

steelbytes commented 1 year ago

it's not the accuracy of the overlay colours that are bugging me. its the colours of the base video as it appears in the output are so dead looking vs original.

dway123 commented 1 year ago

Yes, what I meant is that because OverlayShaderProgram only supports SDR and not HDR, you must input SDR colors to use it. From there, tone-mapping in general converts HDR colors to SDR, which clips out the brighter and darker parts of the base video (which might be contributing to the "dead-looking" look, relative to the original HDR input video)

droid-girl commented 1 month ago

Transformer now supports: