[Android] Memory leak with video playback for Amlogic/Rockchip CPUs devices

itlancer commented 1 year ago

Problem Description

Memory leak with video playback for Amlogic/Rockchip CPUs Android devices.

It has been tested with multiple AIR versions, even latest AIR 50.0.1.1 with multiple Android devices with different AIR applications with different OS versions with different architectures (armv7/armv8) with different videos. Same issue in all cases for Amlogic/Rockchip CPUs devices. There is no such issue with non-Amlogic/Rockchip CPUs devices (Samsung Exynos, Qualcomm Snapdragon, ...). There is no such issues with Windows/macOS/iOS. There is no such memory leaks with non-AIR video players. For example, if you use ExoPlayer even as a AIR native extension.

Problem devices: x96mini, x96 Max Plus, Xiaomi MiTV-MSSP1 and other Android TV Boxes with Amlogic S905X3, Amlogic S905Y2, Amlogic A311D, Rockchip RK3399 CPUs. Most of Amlogic CPUs devices support only armv7.

Steps to Reproduce

Launch AIR application with code below for Amlogic/Rockchip CPU Android device. It just play video in loop. Check application memory consuption every few minutes via ADB command: adb shell dumpsys meminfo android.amlogic.video.memory.leak There is no memory leak in Scout. So you should use some "native" memory measurements.

Application example with sources, video sample and ADB Logcat logs attached. android_amlogic_video_memory_leak.zip

package {
    import flash.display.Sprite;
    import flash.events.Event;
    import flash.net.NetConnection;
    import flash.net.NetStream;
    import flash.media.Video;
    import flash.events.NetStatusEvent;

    public class AndroidAmlogicVideoMemoryLeak extends Sprite {
        private var nc:NetConnection;
        private var ns:NetStream;
        private var video:Video = new Video(640, 480);

        public function AndroidAmlogicVideoMemoryLeak() {
            addEventListener(Event.ADDED_TO_STAGE, init);
        }

        private function init(e:Event):void {
            removeEventListener(Event.ADDED_TO_STAGE, init);

            startVideo();
        }

        private function startVideo():void {
            removeChildren();

            if (ns != null){
                video.attachNetStream(null);
                ns.removeEventListener(NetStatusEvent.NET_STATUS, nsHandler);
                ns.dispose();
            }

            if (video != null){
                video.clear();
            }

            video = new Video(640, 480);
            addChild(video);

            if (nc != null){
                nc.removeEventListener(NetStatusEvent.NET_STATUS, ncHandler);
                nc.close();
            }

            nc = new NetConnection();
            nc.addEventListener(NetStatusEvent.NET_STATUS, ncHandler);
            nc.connect(null);
        }

        private function ncHandler(e:NetStatusEvent):void {
            if (e.info.code == "NetConnection.Connect.Success"){
                ns = new NetStream(nc);
                ns.checkPolicyFile = false;
                ns.client = {onMetaData:getMeta};
                ns.addEventListener(NetStatusEvent.NET_STATUS, nsHandler);

                video.attachNetStream(ns);
                ns.play("video.mp4");
            }
        }

        private function nsHandler(e:NetStatusEvent):void {
            trace(e.info.code);
            if (e.info.code == "NetStream.Play.Stop"){
                startVideo();
            }
        }

        private function getMeta(mdata:Object):void { }
    }
}

Actual Result: Depends on video and device memory leak speed ~0.5-2.5 MB/minute. After ~4 hours application sample will consume ~500 MB RAM and it leads to app crash by "out of memory".

Expected Result: No memory leak.

Known Workarounds

none Write own native extensions to video playback but it cannot be used with DisplayList or as a Texture.

ajwfrost commented 1 year ago

Hi

We're just looking into this, I think we'll need to send you some test builds but to help narrow down the options here:

either, we have a small frame-by-frame leak
or, we have a leak that happens during tear-down/set-up

So would it be possible to run two different tests using the same code, but in one case have a video MP4 file that's a lot longer (ideally 4x) than the other one? Then if you could run them for the same durations, we can see whether the one that only restarts the stream periodically has less of a memory leak -> in which case, it's related to the overall set-up/tear-down of the video stream.

thanks

itlancer commented 1 year ago

@ajwfrost Made such two tests with code sample in original post. First with "short" (~2 seconds) video (attached in original post). And second with "long" (20 seconds) video. Here "long" video: video_long.zip Both MP4 videos with the same codec and resolution (1920*1080).

Each 3 seconds made application RAM consumprions measurements (in KBytes, vertical axis) for a 10 minutes. Total 200 measurements (horizontal axis).

So "long" video playback cause "more intensive" memory leak than with "short" video. Is it enough to get test builds?

ajwfrost commented 1 year ago

That's quite interesting, and pretty much the opposite of what I'd have expected. If it's a frame-by-frame issue then you'd expect the rate of increase to be the same; if it's a per-video increase you'd expect the short one to go up higher. If the longer video is going up quicker then it may be related to the buffer usage/recycling in the OpenMax layers, which is also something we'd wondered based on the logcat output..

Will see if we can add some logs, or tweak the mechanism to see if this is something we can work around...

thanks

ajwfrost commented 1 year ago

Hi

Whilst looking at the code paths, we've found a way to switch to the older rendering mechanism and I'm wondering whether this would help here .. are you able to try this please (from https://community.adobe.com/t5/flash-player-discussions/3-14-2017-release-air-25-runtime-and-sdk/td-p/8936007)

Option to fallback to older video pipeline for Android

AIR 22 had significant changes to the Android Video pipeline. For more information, see the Release Notes for that release. If your application is facing issues because of these changes, starting in AIR 25, you can fallback to the older video pipeline using a new tag named in the application descriptor to enable/disable MediaCodec for Android. When setting this flag as true, MediaCodec is disabled and video is decoded using OpenMax AL just like in AIR 21. The default value for this tag is false.

Sample snippet: Here is an example code snippet for disabling MediaCodec.

<android>

            <manifestAdditions><![CDATA[

            …

            …

            ]]></manifestAdditions>

            <disableMediaCodec>true</disableMediaCodec>

</android>

If that doesn't help then we can add further debugging output into the media codec code and display rendering..

thanks

itlancer commented 1 year ago

@ajwfrost With <disableMediaCodec>true</disableMediaCodec> seems no memory leak.

But I don't known how it could affect real applications. Earlier, before AIR 22-25, there are some visual issues or crashes related to video playback for Android. Right now I can see that with <disableMediaCodec>true</disableMediaCodec> video playback start takes more time. Visually it not good.

ajwfrost commented 1 year ago

Thanks -> this helps us though to pinpoint that the leak is happening in the codec side, rather than anything that's common between the two options. Will send over a runtime apk that has some additional debugging in it - it was a 64-bit ARM build I believe?

itlancer commented 1 year ago

@ajwfrost It was 32-bit (armv7). Most of such Amlogic devices have only armv7 firmwares.

airsdk / Adobe-Runtime-Support