media-kit / media-kit

A cross-platform video player & audio player for Flutter & Dart.
https://github.com/media-kit/media-kit
MIT License
916 stars 132 forks source link

chore: migration from SurfaceTextureEntry to SurfaceProducer #824

Open abdelaziz-mahdy opened 2 weeks ago

abdelaziz-mahdy commented 2 weeks ago

this issue is a fix for https://github.com/media-kit/media-kit/issues/707 , since flutter migrated to SurfaceProducer for impeller https://docs.flutter.dev/release/breaking-changes/android-surface-plugins

right now this is blocked by setOnFrameAvailableListener , since i cant find it in the new class and cant find any mention of it anywhere

abdelaziz-mahdy commented 2 weeks ago

i did try https://developer.android.com/reference/android/view/SurfaceHolder either that i am doing it wrong due to not understanding the code fully, or it doesnt work for our case anyway, leaving this here for tracking https://github.com/abdelaziz-mahdy/media_kit/tree/migration-to-impller-test

abdelaziz-mahdy commented 1 week ago

also i tested with Choreographer in https://github.com/abdelaziz-mahdy/media_kit/tree/migration-to-impller-Choreographer

both branches have the same problem, the first video rendering will be incorrect, and then the next video will be correct,note the video 1 and 2 works maybe the switch gives it time, i dont know , and there is a black screen that happens when switching alot of times and its error is

I/media_kit(12018): com.alexmercerind.media_kit_video.VideoOutputManager.createSurface: -5476376647959460208
W/System.err(12018): java.lang.IllegalStateException: Surface has already been released.
W/System.err(12018):    at android.view.Surface.checkNotReleasedLocked(Surface.java:757)
W/System.err(12018):    at android.view.Surface.lockCanvas(Surface.java:435)
W/System.err(12018):    at com.alexmercerind.media_kit_video.VideoOutput.clearSurface(VideoOutput.java:195)
W/System.err(12018):    at com.alexmercerind.media_kit_video.VideoOutput.createSurface(VideoOutput.java:163)
W/System.err(12018):    at com.alexmercerind.media_kit_video.VideoOutputManager.createSurface(VideoOutputManager.java:55)
W/System.err(12018):    at com.alexmercerind.media_kit_video.MediaKitVideoPlugin.onMethodCall(MediaKitVideoPlugin.java:115)
W/System.err(12018):    at io.flutter.plugin.common.MethodChannel$IncomingMethodCallHandler.onMessage(MethodChannel.java:267)
W/System.err(12018):    at io.flutter.embedding.engine.dart.DartMessenger.invokeHandler(DartMessenger.java:292)
W/System.err(12018):    at io.flutter.embedding.engine.dart.DartMessenger.lambda$dispatchMessageToQueue$0$io-flutter-embedding-engine-dart-DartMessenger(DartMessenger.java:319)
W/System.err(12018):    at io.flutter.embedding.engine.dart.DartMessenger$$ExternalSyntheticLambda0.run(Unknown Source:12)
W/System.err(12018):    at android.os.Handler.handleCallback(Handler.java:938)
W/System.err(12018):    at android.os.Handler.dispatchMessage(Handler.java:99)
W/System.err(12018):    at android.os.Looper.loopOnce(Looper.java:201)
W/System.err(12018):    at android.os.Looper.loop(Looper.java:288)
W/System.err(12018):    at android.app.ActivityThread.main(ActivityThread.java:7839)
W/System.err(12018):    at java.lang.reflect.Method.invoke(Native Method)
W/System.err(12018):    at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:548)
W/System.err(12018):    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1003)

example of rendering problems the video will flicker only colors

image

the video scale is incorrect

image

yes i am testing on emulator but on main none of these problems happen so there is a problem in the migration i did, @alexmercerind looks like i failed to do it and will need your help in this one

also let me know which branch is more clear to be merged with this one.

abdelaziz-mahdy commented 2 days ago

cant figure out how to fix those

the first video render as a flicker of colors

black screen instead of rendering, it doesnt happen all the time, so i cant figure when and why it happens

video scale is wrong(this is gone, or much less) so lets ignore it for now (only for the first video from the sample it happens)