Memory leak when running hello_decode

[HelpMeHelpYou]

There is a memory leak. System info.

user@computer:~$ uname -a
Linux MC-VT-TB34 6.5.0-17-generic #17~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue Jan 16 14:32:32 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

user@computer:~$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04 LTS"

user@computer:~$ cat /proc/cpuinfo  | grep -i "model name" | head -n 1
model name      : 11th Gen Intel(R) Core(TM) i7-11700K @ 3.60GHz

user@computer:~/vivanov/onevpl/dev$ vainfo |  head -n 3
libva info: VA-API version 1.19.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_19
libva info: va_openDriver() returns 0
Trying display: wayland
vainfo: VA-API version: 1.19 (libva 2.19.0)
vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 23.2.4 ()

Steps to reproduce: compile hello_decode and run it.

If you are still not sure apply the patch to loop the source:

diff --git a/examples/api2x/hello-decode/src/hello-decode.cpp b/examples/api2x/hello-decode/src/hello-decode.cpp
index c842ddc..b014c31 100644
--- a/examples/api2x/hello-decode/src/hello-decode.cpp
+++ b/examples/api2x/hello-decode/src/hello-decode.cpp
@@ -13,7 +13,7 @@

 #include "util.hpp"

-#define OUTPUT_FILE                "out.raw"
+#define OUTPUT_FILE                "/dev/null"
 #define BITSTREAM_BUFFER_SIZE      2000000
 #define MAJOR_API_VERSION_REQUIRED 2
 #define MINOR_API_VERSION_REQUIRED 2
@@ -34,6 +34,7 @@ void Usage(void) {

 int main(int argc, char *argv[]) {
     // Variables used for legacy and 2.x
+   
     bool isDraining                 = false;
     bool isStillGoing               = true;
     bool isFailed                   = false;
@@ -47,6 +48,7 @@ int main(int argc, char *argv[]) {
     mfxStatus sts                   = MFX_ERR_NONE;
     Params cliParams                = {};
     mfxVideoParam decodeParams      = {};
+    size_t header_end_offset        =  0;

     // variables used only in 2.x version
     mfxConfig cfg[3];
@@ -118,6 +120,7 @@ int main(int argc, char *argv[]) {
     decodeParams.mfx.CodecId = MFX_CODEC_HEVC;
     decodeParams.IOPattern   = MFX_IOPATTERN_OUT_SYSTEM_MEMORY;
     sts                      = MFXVideoDECODE_DecodeHeader(session, &bitstream, &decodeParams);
+    header_end_offset        = bitstream.DataOffset;
     VERIFY(MFX_ERR_NONE == sts, "Error decoding header\n");

     // input parameters finished, now initialize decode
@@ -177,7 +180,12 @@ int main(int argc, char *argv[]) {
                 // The function requires more bitstream at input before decoding can
                 // proceed
                 if (isDraining)
-                    isStillGoing = false;
+                {
+                    fseek(source,header_end_offset,SEEK_SET);
+                    printf("loop the file!\n");
+                    fflush(stdout);
+                   isDraining = false;
+                }
                 break;
             case MFX_ERR_MORE_SURFACE:
                 // The function requires more frame surface at output before decoding
@@ -222,9 +230,14 @@ end:
     // Clean up resources - It is recommended to close components first, before
     // releasing allocated surfaces, since some surfaces may still be locked by
     // internal resources.
+   
+    printf("closing source\n");
+    fflush(stdout);
     if (source)
         fclose(source);

+    printf("closing sink\n");
+    fflush(stdout);
     if (sink)
         fclose(sink);

Watch umlimited memroy consumption (VmRss)

[shepark]

@HelpMeHelpYou Why don't you modify "ReadEncodedStream()" to reset file pointer in util.hpp? Something like this,

mfxStatus ReadEncodedStream(mfxBitstream &bs, FILE *f) {
    mfxU8 *p0 = bs.Data;
    mfxU8 *p1 = bs.Data + bs.DataOffset;
    if (bs.DataOffset > bs.MaxLength - 1) {
        return MFX_ERR_NOT_ENOUGH_BUFFER;
    }
    if (bs.DataLength + bs.DataOffset > bs.MaxLength) {
        return MFX_ERR_NOT_ENOUGH_BUFFER;
    }
    for (mfxU32 i = 0; i < bs.DataLength; i++) {
        *(p0++) = *(p1++);
    }

    bs.DataOffset = 0;

    do {
        bs.DataLength += (mfxU32)fread(bs.Data + bs.DataLength, 1, bs.MaxLength - bs.DataLength, f);

        if (bs.DataLength == 0) {
            if (fseek(f, 0, SEEK_SET) != 0) {
                printf("Error SEEK_SET 0\n");
                exit(1);
            }
            printf("\nSEEK_SET 0\n");
        }
    } while (bs.DataLength == 0)

    return MFX_ERR_NONE;
}

[HelpMeHelpYou]

@shepark Hello, thank for paying attention. Your snippet is pretty good. I think It'easy to understand how we make the bug more visible with your code. But still the bug is reproducible.