Memory leak when using the video player on a imx.6 board

[sehrlieb]

Hello,

We use the gstreamer video-player on an imx6 board using the Coda960 driver (and v4l2 codec for H264) to allow taping into the VPU hardware. This requires allocating CMA memory, but unfortunately we are facing some memory leak and after playing a few videos, CMA allocation fails. We do not know if the problem lies in gstreamer or flutter-pi (we are new to both).

What we noticed is that the reference count toward the coda driver is incremented each time we start playing a video but never decremented (via lsmod), even after the dispose of the videoplayer. If it was going back to zero we could force unloading and reloading the driver, as in this case, memory is recovered.

In https://gstreamer.freedesktop.org/documentation/plugin-development/basics/states.html?gi-language=c , it is described that the transition to READY_TO_NULL, then an element should unload these libraries and free all allocated resources (like HW devices).

We can see that there is maybedeinit that seems to set the pipeline to NULL but we lack knowledge if this will set the READY_TO_NULL transition. Is it the case?

It would be great if you could see if we misused flutter-pi or provide guidance:

At the application level, we instantiate a VideoPlayerController (package video_player 2.7.2) with the constructor asset:

_controller = GVideoPlayerController.asset(_currentVideo)
    ..addListener(_listen)
    ..initialize().then((_) {
        setState(() {});
});

The controller is disposed in the parent widget dispose method:

void dispose() {
    _controller.removeListener(_listen);
    _controller.dispose();
    super.dispose();
 }

The video is played with (and here the ref count is increased by 1): _controller.play(); And in the main, we call :

FlutterpiVideoPlayer.registerWith();

From flutterpi_gstreamer_video_player ^0.1.1+1

Other than this issue, flutter-pi works like a charm for all other aspect for us. Great stuff 🙂

Thank you.

[sehrlieb]

Hello,

So I was able to resolve the memory leak by comparing flutter-pi video player implementation with the one from https://github.com/sony/flutter-elinux-plugins/tree/main/packages/video_player. Both were almost identical except when transiting to the STATE_NULL.

It seems that there is a need to transit to the STATE_READY before to go to the STATE_NULL to avoid CMA memory leakage. Here is the patch:

`--- a/src/plugins/gstreamer_video_player/player.c 2023-10-30 06:16:06.433565192 +0100 +++ b/src/plugins/gstreamer_video_player/player.c 2023-12-14 07:14:43.498738016 +0100 @@ -1000,6 +1000,7 @@ player->bus = NULL; } if (player->pipeline != NULL) {

• gst_element_set_state(GST_ELEMENT(player->pipeline), GST_STATE_READY); gst_element_set_state(GST_ELEMENT(player->pipeline), GST_STATE_NULL); gst_object_unref(GST_OBJECT(player->pipeline)); player->pipeline = NULL;`

Thanks

[ardera]

That's interesting. The docs say transitioning to GST_STATE_NULL directly works (will transition through all the intermediary states): https://gstreamer.freedesktop.org/documentation/gstreamer/gstelement.html#gst_element_set_state

Did you test if your patch fixes the issues? If yes, might be a gstreamer bug.

[sehrlieb]

Hi, the patch work on my environment. I was able to run a demo that play 4 videos in a loop for more than 24H without any issue and previously I would run out of memory after a few iterations. I have also removed the patch and the memory issue was back. So it definitively helps.

I am using gstreamer 1.20.3 and Kernel 5.15.142 (for the CODA driver). Indeed, it is possible that this is a bug in gstreamer or the v4l2 plugin as the imx6's VPU requires CMA memory. If time allowed, I might try to upgrade to check if this helps.

Otherwise, everything in flutter-pi work like a charm. Very good stuff.

[ardera]

Okay, I'll apply it then. Maybe it also helps with https://github.com/ardera/flutter_packages/issues/25

Otherwise, everything in flutter-pi work like a charm. Very good stuff.

Thanks!

[sehrlieb]

Cool. Thanks!