Open isnzenauz opened 3 months ago
please show me the log following issue template
Experiencing the same issue on Windows. I am using the Player low level object and showing the texture:
Center(
child: ValueListenableBuilder<int?>(
valueListenable: player.textureId,
builder: (context, id, _) => id == null
? const SizedBox.shrink()
: Texture(textureId: id),
),
),
And even with a bare bone UI (nothing else but the player's texture) I get severe frame dropping compared to the vanilla Flutter player.
Here's the logs although, ignore the buffering my internet was a bit unstable at that moment: https://pastebin.com/VczcUvV7
I also have a sample to showcase the issue:
https://github.com/user-attachments/assets/0e79f876-7879-4789-a67d-631d494712bf
In case it's not clear enough in the sample, here you have another player at the top and FVP at the bottom. You can clearly see the dropped frame after slowing down the footage
https://github.com/user-attachments/assets/658f66c0-4cea-44dc-af1c-5507cf653792
@azukaar
flutter's rendering performance is very poor, comparing to my Qt examples, higher cpu and gpu load, lower fps. It's texture api for native plugins is not good enough, we can't know the rendering context used by flutter gui, so in fvp I create another context to render video. but it's impossible to sync 2 contexts in fvp without knowing flutter's context, so if flutter does not handle resource synchronization correctly(I guess yes), you will see tearing. You can download libmdk and run ./glfwplay.exe -c:v MFT -d3d11 -log all $video_url
to see drop count, if you can reproduce the same issue in glfwplay, then it's a bug in libmdk. glfwplay is a simple example without ui.
What is vanilla Flutter player?
THE PROBLEM IN ANDROID TV,
THE PROBLEM IN ANDROID TV,
I need the log, otherwise I can't help
flutter's rendering performance is very poor, comparing to my Qt examples, higher cpu and gpu load, lower fps. It's texture api for native plugins is not good enough
is there a way to fix this? for example adding a buffer for the texture synchronisation?
What is vanilla Flutter player?
I mean the basic VideoPlayer component in Flutter. It does not have the chopiness issue
You can download libmdk and run ./glfwplay.exe
OK so I tried it. It does work much better but I think. See the footage. First half is MDK, second half is MPV. BTW there is an issue in glfwplay.exe I think because cli options (like adding a video URL or adding -log all) don't do anything, so unfortunately I wasn't able to get any log.
https://github.com/user-attachments/assets/d580236c-33ba-47d2-bfc2-a2580609d62f
@azukaar
is there a way to fix this? for example adding a buffer for the texture synchronisation?
Impossible to fix tearing issue. Must fix in flutter side.
I mean the basic VideoPlayer component in Flutter. It does not have the chopiness issue
which platform?
BTW there is an issue in glfwplay.exe I think because cli options (like adding a video URL or adding -log all) don't do anything, so unfortunately I wasn't able to get any log.
I run commands in msys2, then log will print to msys2 terminal. If no msys2, you can add -logfile glfwplay.log
in the command line, then log will save to a file, but drop info is only visible in terminal. To play a local file, you can also run install.bat
as admin, then right click a video file you can choose glfwplay to open.
Impossible to fix tearing issue. Must fix in flutter side.
It's not a tearing issue, the issue is frame dropping / stuttering. And when using the Flutter Video_player or Media_kit packages instead, there's no issues at all, that's why I think it comes from the FVP package (not MDK lib itself, just the way the binding to Flutter is). The video_player and media_kit also use a binding on Texture Component so it is capable or rendering the video without stutter for sure. I am having trouble finding the correct root cause thought, still investigating
Also see for the terminal, if I run the command, the terminal detaches:
It's not a tearing issue, the issue is frame dropping / stuttering
I see tearing in https://github.com/wang-bin/fvp/issues/134#issuecomment-2335153764
./glfwplay.exe -log all -logfile glfwplay.log -c:v MFT -d3d11 $video_url
will save log to a file. Or install msys2 and run command in msys2 terminal to see log in terminal.
I see tearing in #134 (comment)
No it's not tearing, I superposed two video. The top half is MPV, the bottom half is MDK
@azukaar
I see tearing in #134 (comment)
No it's not tearing, I superposed two video. The top half is MPV, the bottom half is MDK
I edit the screenshot. Do you mean there are 2 players in your recorded video, top left triangle is mpv, bottom right triangle is mdk? What I see is tearing at diagonal.
yes correct that is it i edited the video to show the difference better
is there a way to fix this? for example adding a buffer for the texture synchronisation?
Be sure to specify the texture size in the updateTexture(width, heigth) method. I've encountered disgusting lags when playing 4K videos. If you do not specify the size, the plugin gives the flutter a full texture with a size of 3840×2160 and the flutter scales it to the size of the window. If you specify the texture size, the plugin will adjust the size itself and the flutter needs to perform much fewer actions.
I see thanks, I am assuming that only is relevant when the viewport is not 4k
is there a way to fix this? for example adding a buffer for the texture synchronisation?
Be sure to specify the texture size in the updateTexture(width, heigth) method. I've encountered disgusting lags when playing 4K videos. If you do not specify the size, the plugin gives the flutter a full texture with a size of 3840×2160 and the flutter scales it to the size of the window. If you specify the texture size, the plugin will adjust the size itself and the flutter needs to perform much fewer actions.
try master branch, and set max texture size as screen size or smaller
registerWith(options: {
'maxWidth': screenWidth,
'maxHeight': screenHeight,
});
https://github.com/user-attachments/assets/d76de813-a393-4c22-bee6-5d0fcb298270
Look drops in the apk test
4k live Android mobile
https://github.com/user-attachments/assets/5d75c709-01d4-452d-b94c-bf076b103f36
Same Channel 4k working good in vlc player
https://github.com/user-attachments/assets/8e85e87d-7755-4422-9fbc-1d642d67e0d2
Same Channel 4k working good in vlc player
Yes. Flutter is not the best choice for a video application. It copies each frame to his own scene. This is a very large volume of more than 9Mb for a 2000x1200 screen. I spent a lot of time and now regret this choice. I found a temporary solution for android. I took the native implementation of the mdk-sdk and brought out its SurfaceView as the root of FragmentActivity and a transparent FlutterFragment is located above it. Fragment, respectively, is my entire former application, only without the video texture. The performance is staggering.
try master branch, and set max texture size as screen size or smaller
It had no effect. I did an experiment. I took the mdk-sdk and brought it to Flutter as native Android views. the example is taken from here: https://github.com/wang-bin/mdk-android In my app:
class NativeView implements PlatformView {
private SurfaceView view;
private MDKPlayer mPlayer;
private static final String videoURL = Environment.getExternalStorageDirectory().getPath()
+ "/Download/test4k.webm";
NativeView(@NonNull Context context, int id, @Nullable Map<String, Object> creationParams) {
view = new SurfaceView(context);
mPlayer = new MDKPlayer();
mPlayer.setSurfaceView(view);
mPlayer.setMedia(videoURL);
mPlayer.setState(PlaybackState.PLAYING);
}
@Override
public View getView() {
return view;
}
@Override
public void dispose() {
mPlayer.surfaceDestroyed(view.getHolder());
view.getHolder().getSurface().release();
mPlayer.setState(PlaybackState.STOPPED);
}
}
In Flutter:
Widget build(BuildContext context) {
return Scaffold(
body: AndroidView(
viewType: "videoPlayer",
layoutDirection: TextDirection.ltr,
creationParams: creationParams,
creationParamsCodec: const StandardMessageCodec(),
),
);
In this case, there are no lags. Although Native View uses the same texture copying technology.
Unfortunately, I can't build such an option in release mode. mdk-sdk in build.gradle contains: minifyEnabled false But it is redefined by flutter and flutter itself is not going with this option. As a result, the debug version works and the release crashes with an error.
Although Native View uses the same texture copying technology.
I think this is the important point. Every players uses the same technique I think but only the MDK-flutter one seems to have the lags. I think there might just be a bug rather than a hard Flutter limitation. Is there some additional computing that block the thread in the MDK flutter lib may be? Something to do with VSYNC? I am not really sure
Maybe it has some meaning. There is a constant message in the logs, similar to missed frames. But I do not know exactly what it means.
W/vulkan (10596): vkAcquireNextImageKHR: non-infinite timeouts not yet implemented D/ImageReaderSurfaceProducer(10596): ImageTextureEntry can't wait on the fence on Android < 33 W/vulkan (10596): vkAcquireNextImageKHR: non-infinite timeouts not yet implemented W/vulkan (10596): vkAcquireNextImageKHR: non-infinite timeouts not yet implemented W/vulkan (10596): vkAcquireNextImageKHR: non-infinite timeouts not yet implemented W/vulkan (10596): vkAcquireNextImageKHR: non-infinite timeouts not yet implemented W/vulkan (10596): vkAcquireNextImageKHR: non-infinite timeouts not yet implemented I/chatty (10596): uid=10167(ru.sserjirk.flvideo) 1.raster identical 4 lines W/vulkan (10596): vkAcquireNextImageKHR: non-infinite timeouts not yet implemented W/vulkan (10596): vkAcquireNextImageKHR: non-infinite timeouts not yet implemented I/chatty (10596): uid=10167(ru.sserjirk.flvideo) 1.raster identical 8 lines W/vulkan (10596): vkAcquireNextImageKHR: non-infinite timeouts not yet implemented W/vulkan (10596): vkAcquireNextImageKHR: non-infinite timeouts not yet implemented W/vulkan (10596): vkAcquireNextImageKHR: non-infinite timeouts not yet implemented W/vulkan (10596): vkAcquireNextImageKHR: non-infinite timeouts not yet implemented W/vulkan (10596): vkAcquireNextImageKHR: non-infinite timeouts not yet implemented
any news ?
any news ?
Well, this is not a package issue. It's a terrible texture job. Flutter copies the buffer twice. This article describes how a similar problem was solved by reassembling the flutter: https://www.mo4tech.com/real-time-rendering-is-not-a-dream-optimizing-the-rendering-performance-of-flutter-surrounding-textures-through-shared-memory.html Regarding the media_kit package, I dug into the source code. The author uses some pretty dirty hacks. On the one hand, this solves the problem of lags, on the other hand, it causes a huge overhead. The phone is literally heating up in my hand. And on Windows, it loads two cores from the i7-12700H processor by almost 80% and overclocks the processor to 2.9 Ghz, and when paused, the frequency drops to 1.3 Ghz. We will be waiting for the release of the flutter_gpu project, where direct access to gpu memory is expected. I also saw an onFrame callback in the mdk-sdk of the player, which is not implemented in fvp. I would like to try to subscribe to this callback and draw the current frame directly on Canvas. But my C knowledge is not enough for such an experiment.
There was something strange. Something is happening when the video and audio are synchronized. If you disable audio decoding: player.setDecoders(MediaType.audio, [""]); That video in the Texture becomes smooth as when using the native Surfaceview in the mdk-sdk examples. I have tested this on both small 720p and 4K files, the video is equally smooth everywhere without an audio track.
There was something strange. Something is happening when the video and audio are synchronized. If you disable audio decoding: player.setDecoders(MediaType.audio, [""]); That video in the Texture becomes smooth as when using the native Surfaceview in the mdk-sdk examples. I have tested this on both small 720p and 4K files, the video is equally smooth everywhere without an audio track.
thanks for your information. I guess the reason is audio time is not accurate. Can you test other platforms?
Can you test other platforms?
On Windows, deleting the audio decoder has no result. But here is another obvious bug. I figured out how to get around it a long time ago. Instead of Texture, I use the Stateful class. The point is to mark the widget for updating on each frame of the Flutter rendering. Then the video is not much different from the native FFplay player. Which is interesting. If there is one such widget, then you can add as many Textures as you want with the same textureID and everything will be smooth. This is how it looks in a simple version, but of course I have subscriptions to the player status so that the update only works during playback.
class VideoTexture extends StatefulWidget {
final int textureId;
const VideoTexture({required this.textureId, super.key});
@override
State<VideoTexture> createState() => _VideoTextureState();
}
class _VideoTextureState extends State<VideoTexture> {
@override
Widget build(BuildContext context) {
Future.delayed(
const Duration(microseconds: 10),
() => setState(() {}),
);
return Texture(
textureId: widget.textureId,
);
}
}
for android, use the latest libmdk via environment var FVP_DEPS_LATEST=1
. The result should be better.
Mr Wang-Bin this build: https://github.com/wang-bin/fvp/actions/runs/11199447631/job/31131798284
Working good in android can you do this build in pub.dev to use it ?!
Mr Wang-Bin this build: https://github.com/wang-bin/fvp/actions/runs/11199447631/job/31131798284
Working good in android can you do this build in pub.dev to use it ?!
in a few days
Mr Wang-Bin this build: https://github.com/wang-bin/fvp/actions/runs/11199447631/job/31131798284 Working good in android can you do this build in pub.dev to use it ?!
in a few days
I hope to not forget it 🥇 thank you.
Mr Wang-Bin this build: https://github.com/wang-bin/fvp/actions/runs/11199447631/job/31131798284 Working good in android can you do this build in pub.dev to use it ?!
in a few days
I hope to not forget it 🥇 thank you.
try 0.27.0
thank you so mush i will try it and back to you soooooon
Today's nightly build of libmdk works fine on android. But sourceforge did not allow it to be downloaded for several days. I gave the assembly for apple to all requests. And the auto-upload during the build loads the version from 30.08 on which the frames are lost. I think the complete closure of the problem will be when the mdk-sdk build number is updated.
Perfect Job Mr Wang Bin, it working good in HD, FHD, not like before, no more drops. But in 4k ( lives, movies ) still have mush drops, in vlc and media kit in the same device 4k working good,
thank you so mush bro i respect you job its very good <3
Thanks! On Android, FHD does work well indeed but 4K is impossible (despite being able to play 4k with other players). I noted two things:
On my phone (where playback is choppy) I have these logs in loop:
mdk.FINE: 18:08:51.777: texture created: 4
2024-10-12 18:08:51.777 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:51.777: texture created: 4
2024-10-12 18:08:51.777 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:51.777: texture created: 4
2024-10-12 18:08:51.777 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:51.777: texture created: 4
2024-10-12 18:08:51.777 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:51.777: texture created: 4
2024-10-12 18:08:51.777 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:51.777: plane linesize 0: padded = 2908, effective = 2908. theoretical plane size: 727x60. coordinate crop: 1.000000
2024-10-12 18:08:51.777 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:51.777: plane linesize 0: padded = 2908, effective = 2908. theoretical plane size: 727x60. coordinate crop: 1.000000
2024-10-12 18:08:51.777 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:51.777: plane linesize 0: padded = 2908, effective = 2908. theoretical plane size: 727x60. coordinate crop: 1.000000
2024-10-12 18:08:51.777 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:51.777: plane linesize 0: padded = 2908, effective = 2908. theoretical plane size: 727x60. coordinate crop: 1.000000
2024-10-12 18:08:51.777 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:51.777: plane linesize 0: padded = 2908, effective = 2908. theoretical plane size: 727x60. coordinate crop: 1.000000
2024-10-12 18:08:51.778 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:51.778: crop_w=1.000000=>1.000000
2024-10-12 18:08:51.778 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:51.778: crop_w=1.000000=>1.000000
2024-10-12 18:08:51.778 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:51.778: crop_w=1.000000=>1.000000
2024-10-12 18:08:51.778 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:51.778: crop_w=1.000000=>1.000000
2024-10-12 18:08:51.778 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:51.778: crop_w=1.000000=>1.000000
2024-10-12 18:08:51.778 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:51.778: texture 0: 727x60
2024-10-12 18:08:51.778 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:51.778: texture 0: 727x60
2024-10-12 18:08:51.778 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:51.778: texture 0: 727x60
2024-10-12 18:08:51.778 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:51.778: texture 0: 727x60
2024-10-12 18:08:51.778 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:51.778: texture 0: 727x60
2024-10-12 18:08:52.042 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:52.041: [FFmpeg:hevc] Unknown profile bitstream
2024-10-12 18:08:52.042 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:52.041: [FFmpeg:hevc] Unknown profile bitstream
2024-10-12 18:08:52.042 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:52.041: [FFmpeg:hevc] Unknown profile bitstream
2024-10-12 18:08:52.042 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:52.041: [FFmpeg:hevc] Unknown profile bitstream
2024-10-12 18:08:52.042 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:52.041: [FFmpeg:hevc] Unknown profile bitstream
2024-10-12 18:08:53.511 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:53.511: [FFmpeg:hevc] Unknown profile bitstream
2024-10-12 18:08:53.511 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:53.511: [FFmpeg:hevc] Unknown profile bitstream
2024-10-12 18:08:53.511 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:53.511: [FFmpeg:hevc] Unknown profile bitstream
2024-10-12 18:08:53.511 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:53.511: [FFmpeg:hevc] Unknown profile bitstream
2024-10-12 18:08:53.511 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:53.511: [FFmpeg:hevc] Unknown profile bitstream
2024-10-12 18:08:55.018 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:55.018: [FFmpeg:hevc] Unknown profile bitstream
2024-10-12 18:08:55.018 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:55.018: [FFmpeg:hevc] Unknown profile bitstream
2024-10-12 18:08:55.018 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:55.018: [FFmpeg:hevc] Unknown profile bitstream
2024-10-12 18:08:55.018 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:55.018: [FFmpeg:hevc] Unknown profile bitstream
2024-10-12 18:08:55.018 25890-26004 flutter com.azukaar.aurora I mdk.FINE: 18:08:55.018: [FFmpeg:hevc] Unknown profile bitstream
is it normal?
On my Nvidia Shield, the playback does not start in 4k (but works well in FHD). The app just crash. Here are the logs. The timeout and "stop called" are repeated until it crashes completely (I hear 1sec of sound, and no image).
2024-10-12 18:14:38.724 21733-21761 flutter com.azukaar.aurora I mdk.FINE: 18:14:38.713: texture 1: 1920x1080
2024-10-12 18:14:38.913 21733-22474 .azukaar.auror com.azukaar.aurora D PlayerBase::stop() from IPlayer
2024-10-12 18:14:38.913 21733-22474 AudioTrack com.azukaar.aurora D stop(1688): called with 6656 frames delivered
2024-10-12 18:14:38.942 21733-22474 .azukaar.auror com.azukaar.aurora D PlayerBase::stop() from IPlayer
2024-10-12 18:14:38.942 21733-22474 AudioTrack com.azukaar.aurora D stop(1688): called with 256 frames delivered
2024-10-12 18:14:38.958 21733-22474 .azukaar.auror com.azukaar.aurora D PlayerBase::stop() from IPlayer
2024-10-12 18:14:38.958 21733-22474 AudioTrack com.azukaar.aurora D stop(1688): called with 256 frames delivered
2024-10-12 18:14:38.965 21733-22474 .azukaar.auror com.azukaar.aurora D PlayerBase::stop() from IPlayer
2024-10-12 18:14:38.965 21733-22474 AudioTrack com.azukaar.aurora D stop(1688): called with 256 frames delivered
2024-10-12 18:14:38.966 21733-22474 .azukaar.auror com.azukaar.aurora D PlayerBase::stop() from IPlayer
2024-10-12 18:14:38.966 21733-22474 AudioTrack com.azukaar.aurora D stop(1688): called with 256 frames delivered
2024-10-12 18:14:38.967 21733-22474 .azukaar.auror com.azukaar.aurora D PlayerBase::stop() from IPlayer
2024-10-12 18:14:38.967 21733-22474 AudioTrack com.azukaar.aurora D stop(1688): called with 256 frames delivered
2024-10-12 18:14:38.968 21733-22474 .azukaar.auror com.azukaar.aurora D PlayerBase::stop() from IPlayer
2024-10-12 18:14:38.968 21733-22474 AudioTrack com.azukaar.aurora D stop(1688): called with 256 frames delivered
2024-10-12 18:14:38.969 21733-22474 .azukaar.auror com.azukaar.aurora D PlayerBase::stop() from IPlayer
2024-10-12 18:14:38.969 21733-22474 AudioTrack com.azukaar.aurora D stop(1688): called with 256 frames delivered
2024-10-12 18:14:38.970 21733-22474 .azukaar.auror com.azukaar.aurora D PlayerBase::stop() from IPlayer
2024-10-12 18:14:38.970 21733-22474 AudioTrack com.azukaar.aurora D stop(1688): called with 256 frames delivered
2024-10-12 18:14:38.983 21733-21761 flutter com.azukaar.aurora I mdk.FINE: 18:14:38.980: texture 2: 1920x1080
2024-10-12 18:14:38.986 21733-22474 .azukaar.auror com.azukaar.aurora D PlayerBase::stop() from IPlayer
2024-10-12 18:14:38.986 21733-22474 AudioTrack com.azukaar.aurora D stop(1688): called with 256 frames delivered
2024-10-12 18:14:39.497 21733-21761 flutter com.azukaar.aurora I mdk.FINE: 18:14:39.496: material changed: 0x0 => 0x4b85
2024-10-12 18:14:39.497 21733-21761 flutter com.azukaar.aurora I mdk.FINE: 18:14:39.497: planar: 1, alpha: 0, texture 2d: 1, external: 0, rg semi-planar: 0, 16=>8: 0, xyz: 0, yuv sampler: 0, generic packed yuv: 0, cocgsy: 0, primaries diff: 1, trc: 3=>1, tone map: 1
2024-10-12 18:14:39.497 21733-21761 flutter com.azukaar.aurora I mdk.FINE: 18:14:39.497: bind attribute: a_Position => 0
2024-10-12 18:14:39.498 21733-21761 flutter com.azukaar.aurora I mdk.FINE: 18:14:39.498: bind attribute: a_TexCoords0 => 1
2024-10-12 18:14:39.552 21733-22474 .azukaar.auror com.azukaar.aurora D PlayerBase::stop() from IPlayer
2024-10-12 18:14:39.552 21733-22474 AudioTrack com.azukaar.aurora D stop(1688): called with 768 frames delivered
2024-10-12 18:14:39.557 21733-21761 flutter com.azukaar.aurora I mdk.FINE: 18:14:39.557: UBO ConstBuffer: 0/4294967295
2024-10-12 18:14:39.557 21733-21761 flutter com.azukaar.aurora I mdk.FINE: 18:14:39.557: uniform locations:
2024-10-12 18:14:39.557 21733-21761 flutter com.azukaar.aurora I mdk.FINE: 18:14:39.557: u_Texture0: 2
2024-10-12 18:14:39.558 21733-21761 flutter com.azukaar.aurora I mdk.FINE: 18:14:39.557: u_Texture1: 3
2024-10-12 18:14:39.558 21733-21761 flutter com.azukaar.aurora I mdk.FINE: 18:14:39.558: u_Texture2: 4
2024-10-12 18:14:39.564 21733-21761 flutter com.azukaar.aurora I mdk.FINE: 18:14:39.558: u_Matrix: 0, u_TexMatrix: 1
2024-10-12 18:14:39.565 21733-21761 flutter com.azukaar.aurora I mdk.FINE: 18:14:39.564: build glsl elapsed: 250881us
2024-10-12 18:14:39.565 21733-21761 flutter com.azukaar.aurora I mdk.FINE: 18:14:39.565: creating vbo...
2024-10-12 18:14:39.565 21733-21761 flutter com.azukaar.aurora I mdk.FINE: 18:14:39.565: creating vao...
2024-10-12 18:14:39.566 21733-21761 flutter com.azukaar.aurora I fvp.FINE: 18:14:39.566: 1058875561 player1125388768 onEvent: render.video - 1st_frame - 626
2024-10-12 18:14:39.574 21733-22472 libOpenSLES com.azukaar.aurora E frameworks/wilhelm/src/itf/IBufferQueue.cpp:56: pthread_mutex_lock_timeout_np returned 110
2024-10-12 18:14:39.576 21733-22474 .azukaar.auror com.azukaar.aurora D PlayerBase::stop() from IPlayer
2024-10-12 18:14:39.576 21733-22474 AudioTrack com.azukaar.aurora D stop(1688): called with 256 frames delivered
2024-10-12 18:14:39.577 21733-22474 .azukaar.auror com.azukaar.aurora D PlayerBase::stop() from IPlayer
2024-10-12 18:14:39.577 21733-22474 AudioTrack com.azukaar.aurora D stop(1688): called with 256 frames delivered
2024-10-12 18:14:39.578 21733-22474 .azukaar.auror com.azukaar.aurora D PlayerBase::stop() from IPlayer
2024-10-12 18:14:39.578 21733-22474 AudioTrack com.azukaar.aurora D stop(1688): called with 256 frames delivered
2024-10-12 18:14:39.578 21733-22474 .azukaar.auror com.azukaar.aurora D PlayerBase::stop() from IPlayer
2024-10-12 18:14:39.578 21733-22474 AudioTrack com.azukaar.aurora D stop(1688): called with 256 frames delivered
2024-10-12 18:14:39.580 21733-22474 .azukaar.auror com.azukaar.aurora D PlayerBase::stop() from IPlayer
2024-10-12 18:14:39.580 21733-22474 AudioTrack com.azukaar.aurora D stop(1688): called with 256 frames delivered
2024-10-12 18:14:39.580 21733-22474 .azukaar.auror com.azukaar.aurora D PlayerBase::stop() from IPlayer
@azukaar it's using software decoder instead of hardware decoder, performance will be poor. can you show me the complete log so i can know what happens? your phone log repeats several times, it's weird.
https://github.com/wang-bin/mdk-examples/blob/master/flutter/simple/lib/main.dart you can use this example to test, logging is enabled there.
Perfect Job Mr Wang Bin, it working good in HD, FHD, not like before, no more drops. But in 4k ( lives, movies ) still have mush drops, in vlc and media kit in the same device 4k working good,
thank you so mush bro i respect you job its very good <3
can you show me 4k playback log? you can use this example to test: https://github.com/wang-bin/mdk-examples/blob/master/flutter/simple/lib/main.dart
@wang-bin thanks for your answer, it is weird that it's using the software decoder, I am doing
void _initializePlayer() {
print('VP: Initializing player');
print('_videoState.position: ${_videoState.position}');
Logger.root.level = Level.ALL;
final df = DateFormat("HH:mm:ss.SSS");
Logger.root.onRecord.listen((record) {
print(
'${record.loggerName}.${record.level.name}: ${df.format(record.time)}: ${record.message}');
});
fvp.registerWith();
player.media = "${widget.item.originURL}${widget.item.stream}";
player.state = _videoState.state;
player.updateTexture();
player.setBufferRange(min: 3000, max: 240000);
timer = Timer.periodic(Duration(seconds: 5), (Timer t) => onTick());
}
i was assuming that registerWith without decoder would select the default decoders for each platform (at least that's what I see in the code). Also in the logs I can see that it selected FFmpeg as decoder no?
Here are the logs on the phone when playing the 4k video: https://pastebin.com/u589xTYh
@wang-bin thanks for your answer, it is weird that it's using the software decoder, I am doing
void _initializePlayer() { print('VP: Initializing player'); print('_videoState.position: ${_videoState.position}'); Logger.root.level = Level.ALL; final df = DateFormat("HH:mm:ss.SSS"); Logger.root.onRecord.listen((record) { print( '${record.loggerName}.${record.level.name}: ${df.format(record.time)}: ${record.message}'); }); fvp.registerWith(); player.media = "${widget.item.originURL}${widget.item.stream}"; player.state = _videoState.state; player.updateTexture(); player.setBufferRange(min: 3000, max: 240000); timer = Timer.periodic(Duration(seconds: 5), (Timer t) => onTick()); }
i was assuming that registerWith without decoder would select the default decoders for each platform (at least that's what I see in the code). Also in the logs I can see that it selected FFmpeg as decoder no?
Here are the logs on the phone when playing the 4k video: https://pastebin.com/u589xTYh
If you are using backend api, you have to set hardware decoders manually like this https://github.com/wang-bin/fvp/blob/v0.27.0/lib/src/video_player_mdk.dart#L149-L158
and https://github.com/wang-bin/fvp/blob/v0.27.0/lib/src/video_player_mdk.dart#L220
btw, the latest release add more features in VideoPlayerController via extension, so we can reuse video_player code with most of backend api features, and no need to add configurations like video_player_mdk.dart(e.g. hardware decoder). see https://pub.dev/documentation/fvp/latest/fvp/FVPControllerExtensions.html
Today's nightly build of libmdk works fine on android. But sourceforge did not allow it to be downloaded for several days.
I have the same problem, github actions can download them, I don't know why.
I gave the assembly for apple to all requests. And the auto-upload during the build loads the version from 30.08 on which the frames are lost.
?
I think the complete closure of the problem will be when the mdk-sdk build number is updated.
for macos and ios, increasing mdk version in fvp.podspec will download libmdk again, but there's no new libmdk release, so you have to clean up cocoapods cache
OK that makes sense, it works now, the only last issue I am having is the crash on the NVidia Shield with the 4k video and the log I posted before. I'll create a different ticket for it Thanks a lot for your help, and great job honestly
?
I do not know exactly what kind of problem soruceforge has. I go to the nightly builds page, click download mdk-sdk-android.7z and... It is being downloaded mdk-sdk-apple.zip . Similarly, in android studio, I completely clean the flutter clean cache. I am running the build and it is impossible to download the mdk-sdk-android.7z. Just go to github yourself and put this archive in \fvp-xxx\android yourself. This has been going on for the last week.
OK that makes sense, it works now, the only last issue I am having is the crash on the NVidia Shield with the 4k video and the log I posted before. I'll create a different ticket for it Thanks a lot for your help, and great job honestly
I your libmdk.so, libmdk.so.dsym, and crash info from logcat, run
adb logcat -c
adb logcat 2>&1 |tee test.log
before playing a video, when crash occurs, wait a few seconds then ctrl+c
to stop logging , and send me test.log, or show me backtrace starting with *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
in the log. You can also copy the log from android studio, log filter must be empty, otherwise backtrace may be incomplete.
Here's the latest log from my NVidia shield with 0.27, it seems to mostly work but I still have choppy playback (subtle but present lags every few seconds) The thing is the lags are present regardless of the movie played, even a "small" movie of 2gb will have the same lags, which leads me to believe that it's not necessarily a performance issue. Would it be possible for the hardware decoder to not properly recognize the hardward (because the Shield is older) and run in CPU? How would I debug that?
Here are the logs for info: https://pastebin.com/69ia7J81
Here's the latest log from my NVidia shield with 0.27, it seems to mostly work but I still have choppy playback (subtle but present lags every few seconds) The thing is the lags are present regardless of the movie played, even a "small" movie of 2gb will have the same lags, which leads me to believe that it's not necessarily a performance issue. Would it be possible for the hardware decoder to not properly recognize the hardward (because the Shield is older) and run in CPU? How would I debug that?
Here are the logs for info: https://pastebin.com/69ia7J81
it's using hardware decoder, but many frames are dropped in renderer. what about disabling audio tracks: player.activeAudioTracks=[] ?
Same results with no audio: https://pastebin.com/MAkhz9X5
@azukaar try master branch code and send me log again, audio can be enabled.
This is the log after a few secs of playback : https://pastebin.com/WYeucfsJ
I/flutter (29194): mdk.FINE: 11:40:20.174: 0x475bf570>25232 00:00:25.2/02:22:55 cache 0v 3.6s/49591KB 6945KB/s |+4|<4ms update 41.5ms 24.0fps draw 1/2 +0ms gpu 1903/2012us drop 18
I/flutter (29194): mdk.FINE: 11:40:20.300: 0x475bf570>25358 00:00:25.3/02:22:55 cache 0v 3.6s/49509KB 6670KB/s |+3|<4ms update 41.9ms 24.0fps draw 1/2 +0ms gpu 1899/2012us drop 18
I/flutter (29194): mdk.FINE: 11:40:20.426: 0x475bf570>25483 00:00:25.4/02:22:55 cache 0v 3.6s/49467KB 6535KB/s |+2|<4ms update 41.9ms 24.0fps draw 1/2 +0ms gpu 1896/2012us drop 18
I/flutter (29194): mdk.FINE: 11:40:20.547: 0x475bf570>25608 00:00:25.6/02:22:55 cache 0v 3.6s/49487KB 6406KB/s |+3|<4ms update 41.7ms 24.0fps draw 1/2 +0ms gpu 1882/2003us drop 18
I/flutter (29194): mdk.FINE: 11:40:20.674: 0x475bf570>25733 00:00:25.7/02:22:55 cache 0v 3.6s/49574KB 6162KB/s |+3|<4ms update 41.7ms 24.0fps draw 1/2 +0ms gpu 1889/2045us drop 18
I/flutter (29194): mdk.FINE: 11:40:20.801: 0x475bf570>25858 00:00:25.8/02:22:55 cache 0v 3.6s/49468KB 6107KB/s |+2|<4ms update 41.9ms 23.7fps draw 1/2 +0ms gpu 1903/2045us drop 18
I/flutter (29194): mdk.FINE: 11:40:20.926: 0x475bf570>25983 00:00:25.9/02:22:55 cache 0v 3.6s/49404KB 5893KB/s |+2|<4ms update 41.7ms 23.7fps draw 1/2 +0ms gpu 1906/2045us drop 18
I/flutter (29194): mdk.FINE: 11:40:21.044: 0x475bf570>26108 00:00:26.1/02:22:55 cache 0v 3.6s/49263KB 6575KB/s |+2|<4ms update 41.2ms 23.7fps draw 1/2 +0ms gpu 1898/2045us drop 18
I/flutter (29194): mdk.FINE: 11:40:21.169: 0x475bf570>26233 00:00:26.2/02:22:55 cache 0v 3.6s/49131KB 6346KB/s |+2|<4ms update 41.5ms 23.7fps draw 1/2 +0ms gpu 1886/2045us drop 18
I/flutter (29194): mdk.FINE: 11:40:21.293: 0x475bf570>26359 00:00:26.3/02:22:55 cache 0v 3.6s/48937KB 6414KB/s |+3|<4ms update 41.3ms 23.8fps draw 1/2 +0ms gpu 1891/2045us drop 18
I/flutter (29194): mdk.FINE: 11:40:21.419: 0x475bf570>26484 00:00:26.4/02:22:55 cache 0v 3.6s/48729KB 6280KB/s |+3|<4ms update 41.3ms 23.8fps draw 1/2 +0ms gpu 1870/1942us drop 18
I/flutter (29194): mdk.FINE: 11:40:21.550: 0x475bf570>26609 00:00:26.6/02:22:55 cache 0v 3.6s/48486KB 5772KB/s |+2|<4ms update 41.3ms 23.9fps draw 1/2 +0ms gpu 1871/1942us drop 18
I/flutter (29194): mdk.FINE: 11:40:21.671: 0x475bf570>26734 00:00:26.7/02:22:55 cache 0v 3.6s/48323KB 5633KB/s |+1|<4ms update 41.8ms 23.9fps draw 1/2 +0ms gpu 1868/1949us drop 18
I/flutter (29194): mdk.FINE: 11:40:21.797: 0x475bf570>26859 00:00:26.8/02:22:55 cache 0v 3.6s/47870KB 5174KB/s |+0|<4ms update 41.8ms 24.0fps draw 1/2 +0ms gpu 1876/1949us drop 18
I/flutter (29194): mdk.FINE: 11:40:21.924: 0x475bf570>26984 00:00:26.9/02:22:55 cache 0v 3.6s/47600KB 4645KB/s |+3|<4ms update 42.0ms 24.4fps draw 1/3 +0ms gpu 1870/1949us drop 18
Does this mean that 18 frames have been dropped in total? If yes it does not match what I see on screen, I can clearly see many more frame drops that do not increment that value. Sometime even the sound stutters
what about when FPS goes < 24, could it be what I see? 23.7 is very close to 24 FPS, but if the frames are dropped instead of drawn, it could be what I see no? The thing is my Windows PC also goes to 23.9 FPS but has no visible stutter so probably not the issue
hi dear, i have problem with fvp drop frames on android (mobile, tv) im trying to use AMediaCodec and FFmpeg but his give me same results frame drops i test it in: XIAOMI MI BOX S2 (Android), And mobile RED MAGIC 6, Poco x3 Pro, Tecno Spark 8 tecno spark 10 pro, give me same results, drop frames in LIVE M3u8, and Movies MP4, MKV
I/AudioTrack(21381): AudioTrack::setVolume L:1.000000 R:1.000000 and the pakage is com.poppycorn.rax.poppycorn I/AudioTrack(21381): AudioTrack::setVolume L:1.000000 R:1.000000 I/AudioTrack(21381): AudioTrack::setVolume L:1.000000 R:1.000000 and the pakage is com.poppycorn.rax.poppycorn I/AudioTrack(21381): AudioTrack::setVolume L:1.000000 R:1.000000 I/AudioTrack(21381): AudioTrack::setVolume L:1.000000 R:1.000000 and the pakage is com.poppycorn.rax.poppycorn I/AudioTrack(21381): AudioTrack::setVolume L:1.000000 R:1.000000 I/AudioTrack(21381): AudioTrack::setVolume L:1.000000 R:1.000000 and the pakage is com.poppycorn.rax.poppycorn I/AudioTrack(21381): AudioTrack::setVolume L:1.000000 R:1.000000 I/AudioTrack(21381): AudioTrack::setVolume L:1.000000 R:1.000000 and the pakage is com.poppycorn.rax.poppycorn I/AudioTrack(21381): AudioTrack::setVolume L:1.000000 R:1.000000 I/AudioTrack(21381): AudioTrack::setVolume L:1.000000 R:1.000000 and the pakage is com.poppycorn.rax.poppycorn I/AudioTrack(21381): AudioTrack::setVolume L:1.000000 R:1.000000 I/AudioTrack(21381): AudioTrack::setVolume L:1.000000 R:1.000000 and the pakage is com.poppycorn.rax.poppycorn I/AudioTrack(21381): AudioTrack::setVolume L:1.000000 R:1.000000 I/AudioTrack(21381): AudioTrack::setVolume L:1.000000 R:1.000000 and the pakage is com.poppycorn.rax.poppycorn I/AudioTrack(21381): AudioTrack::setVolume L:1.000000 R:1.000000 I/AudioTrack(21381): AudioTrack::setVolume L:1.0
The log note give me anything about dropframes