Memory usage after DeInit

[VizDock]

Hi I noticed in my projects in which I experimented, that memory after Init goes up, and after DeInit doesn't go down like before Init. It is arround 150MB more than before Init. I also tried the same thing in your recognizer app (I just moved readkey command to position after deInit). Consequently, 5 inits and deinits increase RAM usage to 1GB and more Is this normal behaviour?

[DoubangoTelecom]

Duplicate of #120

[DoubangoTelecom]

You should consider using OprnVINO instead of Tensorflow if not already the case. OpenVINO use less memory and is faster on CPU. https://www.doubango.org/SDKs/anpr/docs/Memory_management_design.html#memory-usage

If you check the logs you'll see that Tensorflow is configured to allocate up to 45% of the total memory per process. That make it slightly faster. It make sense having that configuration public to ease multi processing

[VizDock]

I am using OpenVINO I checked the link on issue you associated that it's the same, maybe I'm no expert but on Tensorflow github issue it states that it's harmless and that it is not a problem.. If I read and understand correctly. But I checked in resource manager and task manager, both showing 50-100 MB of more RAM usage on each deinit/init. So I don't understand what should I do. It's not a problem of how much RAM process take, its just problem it it increasing in same process on each deinit/init. I had a windows service that uses threads and another SDK to perform LPR. Each IP camera in it's own thread. Then I had problem implementing UltimateALPR because you said it uses static variables, so I had to make each camera running in separate process. I also implemented messaging between windows service and process that is running for specific camera, and implemented (in case of user changing parameters that are required to deInit and Init) that process for specific camera make it so. Now I have this problem. I like your SDK because it's better than the one we are using now but I don't have much time to make all changes, and when I stumble like this it demotivates :)

[DoubangoTelecom]

All objects in the sdk are tracked using reference counter (https://github.com/DoubangoTelecom/compv/blob/master/base/include/compv/base/compv_obj.h) and we always check for memory leaks before releasing the code. The probability to have a memory leak in our products is near zero. Unless you can share a simple code reproducing the issue (like what I have provided in issue #120), I'll keep the ticket closed

[VizDock]

In your c# recognizer project, replace code in Program.cs with this one and check memory after each Deinit

Program.zip

I added this command line parameters in Debug section of Visual Studio: --image ./assets/images/lic_us_1280x720.jpg --assets ./assets --charset latin --car_noplate_detect_enabled false --ienv_enabled false --openvino_enabled true --openvino_device CPU --klass_lpci_enabled false --klass_vcr_enabled false --klass_vmmr_enabled false --klass_vbsr_enabled false --parallel false

Edit.. I see now you wanted code in simple: Basically:

loop 1 to 500 initEngine Loop 1-20 LPR some image with plate Next Loop 1-100 deinit Engine pause 3 seconds for you to check RAM usage next loop 1-500

You will notice on pause point that RAM constantly increases

[DoubangoTelecom]

I made the loop infinite on a PC with 8G memory. The program started at 64Mo then it goes to 300Mo then it stays within [300-400]. That is exactly what we expect and it's clearly explained on the documentation at https://www.doubango.org/SDKs/anpr/docs/Memory_management_design.html#memory-pooling. Quote:

The SDK will allocate at maximum 1/20th of the available RAM during the application lifetime and manage it using a pool. For example, if the device have 8G memory, then it will start allocating 3M memory and depending on the malloc/free requests this amount will be increased with 400M (1/20th of 8G) being the maximum. Most of the time the allocated memory will never be more than 5M.

[DoubangoTelecom]

Another quote from the documentation:

We found it was interesting to add this section on the documentation so that the developers understand why the amount of allocated memory doesn’t automatically decrease when freed. You may think there are leaks but it’s probably not the case. Please also note that we track every allocated memory or object and can automatically detect leaks.

[DoubangoTelecom]

Also, the init()/deInit() functions are expected to be called once per process lifetime. If for any reason you call it 100 times, then you should never consume more than 1/20th of your RAM. The memory management using pooling is done using tbbMalloc from Intel. The source code is at https://github.com/DoubangoTelecom/compv/tree/master/base/include/compv/base/tbbmalloc. The 1/20th rule is enforced at https://github.com/DoubangoTelecom/compv/blob/2574d7df1b8dab53e6f90f6ee177bbba5a833c07/base/compv_base.cxx#L281. The library may allocate more than 1/20th but not that much. The SDK is used in multiple commercial products running 24/7

[VizDock]

This is the first time I read that init/deinit can be used once per process lifetime.

If I have a process that is performing LPR for some IP camera, and user wants to change parameters of lpr, I have to end process and start it again instead of just deinit engine and init it again with new parameters?

That is really weird, and if you're cleaning the memory after deinit, how come it goes like this in video? Not only that it increases on each init (after deinit), but also memory after deinit is becoming more occupied each time.

So, basically what you're telling me, this is normal (attached video link with wetransfer, please see)? I have 16GB of RAM and if I leave this sample running, in 20 minutes it came to 2GB. I stopped it then.. https://wetransfer.com/downloads/5c66680baaa4764ad59737de174511d220210603234055/3691820c805fd61b9379eeb6935bd4c220210603234525/afa590

This is after it has been running for 15 minutes:

[DoubangoTelecom]

"This is the first time I read that init/deinit can be used once per process lifetime."

The file you have shared literally have the following comment This function should be called once. at https://github.com/DoubangoTelecom/ultimateALPR-SDK/blob/0b2f9568d58abb9935e1a3fd71c80bc4ba4ce807/samples/csharp/recognizer/Program.cs#L378

I don't know why you're calling Init/DeInit multiple times but this is clearly not expected. For now we don't support changing the parameters while the app is running, you may need to call Init/DeInit to change the parameters once a day but not every 15 milliseconds for 25 minutes.

Basically we expect a program to call Init() once, then process() for each frame, then deInit() just before exiting the app.

As already said in my previous comment, tbbMalloc may allocate more memory than needed. Calling Init/DeInit for 200 times on a 8G PC I still remain bellow 300M.

[VizDock]

The file you have shared literally have the following comment This function should be called once.

It does, but it doesn't say "per process". I assumed that it means it should not be called multiple times without deinit.

Calling Init/DeInit for 200 times on a 8G PC I still remain bellow 300M.

If I do that, I also remain below 300M.

But if you do some LPR in between init and deinit, like in my video, it behaves differently.

Try this in your recognizer (code modification just between init and deinit) and you will see what I'm talking about

            Thread.Sleep(2000);
            for (int i = 0; i < 100; i++)
            { 
                UltAlprSdkResult result = CheckResult("Init", UltAlprSdkEngine.init(BuildJSON(charsetAkaAlphabet, assetsFolder, tokenDataBase64)));

            // Decode the JPEG/PNG/BMP file
            String file = parameters["--image"];
            if (!System.IO.File.Exists(file))
            {
                throw new System.IO.FileNotFoundException("File not found:" + file);
            }
            Bitmap image = new Bitmap(file);
            if (Image.GetPixelFormatSize(image.PixelFormat) == 24 && ((image.Width * 3) & 3) != 0)
            {
                //!\\ Not DWORD aligned -> the stride will be multiple of 4-bytes instead of 3-bytes
                // ultimateMICR requires stride to be in samples unit instead of in bytes
                Console.Error.WriteLine(String.Format("//!\\ The image width ({0}) not a multiple of DWORD.", image.Width));
                image = new Bitmap(image, new Size((image.Width + 3) & -4, image.Height));
            }
            int bytesPerPixel = Image.GetPixelFormatSize(image.PixelFormat) >> 3;
            if (bytesPerPixel != 1 && bytesPerPixel != 3 && bytesPerPixel != 4)
            {
                throw new System.Exception("Invalid BPP:" + bytesPerPixel);
            }

            // Extract Exif orientation
            const int ExifOrientationTagId = 0x112;
            int orientation = 1;
            if (Array.IndexOf(image.PropertyIdList, ExifOrientationTagId) > -1)
            {
                int orientation_ = image.GetPropertyItem(ExifOrientationTagId).Value[0];
                if (orientation_ >= 1 && orientation_ <= 8)
                {
                    orientation = orientation_;
                }
            }

            // Processing: Detection + recognition
            // First inference is expected to be slow (deep learning models mapping to CPU/GPU memory)
            BitmapData imageData = image.LockBits(new Rectangle(0, 0, image.Width, image.Height), ImageLockMode.ReadOnly, image.PixelFormat);
            try
            {
                    for (int j = 0; j < 100; j++)
                    {
                        // For packed formats (RGB-family): https://www.doubango.org/SDKs/anpr/docs/cpp-api.html#_CPPv4N15ultimateAlprSdk16UltAlprSdkEngine7processEK22ULTALPR_SDK_IMAGE_TYPEPKvK6size_tK6size_tK6size_tKi
                        // For YUV formats (data from camera): https://www.doubango.org/SDKs/anpr/docs/cpp-api.html#_CPPv4N15ultimateAlprSdk16UltAlprSdkEngine7processEK22ULTALPR_SDK_IMAGE_TYPEPKvPKvPKvK6size_tK6size_tK6size_tK6size_tK6size_tK6size_tKi
                        result = CheckResult("Process", UltAlprSdkEngine.process(
                        (bytesPerPixel == 1) ? ULTALPR_SDK_IMAGE_TYPE.ULTALPR_SDK_IMAGE_TYPE_Y : (bytesPerPixel == 4 ? ULTALPR_SDK_IMAGE_TYPE.ULTALPR_SDK_IMAGE_TYPE_BGRA32 : ULTALPR_SDK_IMAGE_TYPE.ULTALPR_SDK_IMAGE_TYPE_BGR24),
                        imageData.Scan0,
                        (uint)imageData.Width,
                        (uint)imageData.Height,
                        (uint)(imageData.Stride / bytesPerPixel),
                        orientation
                    ));
                        // Print result to console
                        Console.WriteLine("Result: {0}", result.json());
                    }
            }
            finally
            {
                image.UnlockBits(imageData);
            }

            // Write until user press a key

            // Now that you're done, deInit the engine before exiting
            CheckResult("DeInit", UltAlprSdkEngine.deInit());
            Thread.Sleep(3000);
            }```

[DoubangoTelecom]

Off course I was calling process between the init/deinit. I used your code but moved the image creation outside the loop. We don't call image.Dispose() as the process is expected to die but you should add that line in the end of your loop.

At the end of the day or next week I'll change the code to reclaim the memory from the thread pools, that should decrease the growth rate but you'll still have increase. As already said, init is expected to be called once or at least very few times. You should open a ticket requesting support for configuration change without restarting the engine

[VizDock]

Hi Thanks I noticed that amount of memory that is increasing is connected to how many times LPR do something between init and deinit. If you init, make one lpr, deinit, and repeat that several times, memory consumption increases with each deinit (compared to after previous deinit) by some small amount. If you make a lot of LPR between init and deinit, and repeat that, memory increase is much bigger between two deinits.

OK I will open support ticket for that functionality, besides configuration change I don't see any reason to make deinit.

Thank you

[VizDock]

We don't call image.Dispose() as the process is expected to die but you should add that line in the end of your loop.

Good point on Dispose().. I added it to recognizer test code above, but it's the same.. I was hoping it will help

[DoubangoTelecom]

The mem pools are per thread. For each thread it'll allocate the minimum memory then when you start processing images it'll grow up to a certain amount then stop. That explains why doing more process() will alloc more memory. If you set the num_threads param to 1 instead of -1 then you should see almost no diff.

I have changed the code to reclaim the mem per thread after deInit: https://github.com/DoubangoTelecom/compv/commit/46955fd254b154e74dc7b15dafedb2da1fc4e5d6

Try binary at https://www.doubango.org/SDKs/anpr/eval-windows/v3.3.3/ultimateALPR-SDK.dll, make sure the version displayed is 3.3.3, it should be better.

[VizDock]

I ran the same test like yesterday. Yesterday after 15 min deinit RAM usage was at 1.1GB and after init RAM usage was 1.3 Now after same time, deinit 400 and init 660 So I can say it is much better, for rare cases when user changes something. Thank you for fast response.