search

exadel-inc / CompreFace

Leading free and open-source face recognition system
https://exadel.com/accelerator-showcase/compreface/
Apache License 2.0
5.25k stars 722 forks source link

Memory consumption when POSTing multiple face files to recognition service #597

Closed Robert-IW closed 2 years ago

Robert-IW commented 3 years ago

Describe the bug Requesting identities for each face in directory of multiple faces (> 500 jpgs with max size 100 kB) consumes RAM/Swap (16 GB/2GB) until container crashes. I've tried this with the python SDK, pycurl and curl with the same results.

To Reproduce Steps to reproduce the behavior:

  1. activate conda environment
  2. docker-compose up -d
  3. run bash script with curl looping through image directory

Expected behavior return API request response for each image clear cache/memory

Screenshots image image

Desktop (please complete the following information):

Additional context Add any other context about the problem here.

pospielov commented 2 years ago

Hi Robert, sorry for the long reply. I just tried your script on my folder with 1100 images(size of images mostly more than 100kb ) and got a very different result: image What I recommend here, try to change the limit for compreface-api container. To do so, you need to open your .env file and change the value compreface_api_java_options, e.g.: compreface_api_java_options=-Xmx1g It will make Java work with a max of 2g memory consumption and run memory garbage collector more often.

pospielov commented 2 years ago

I made a mistake in the script, so there was no recognition... here are the stats after I fixed it: 2021-09-19_18-58 SO basically I have even more memory consumption on compreface-core, but not so big on compreface-api. The reason could be with the number of saved faces in face collection - I tested on one face. But it really depends on the number of faces. How many faces have you saved to face collection?

Robert-IW commented 2 years ago

Hi Posipielov Thanks for your response. The .env change (Xmx8g to Xmx2g) solved the issue. I have just over 7000 images in the database. image

pospielov commented 2 years ago

Because of the tricky behavior of Java memory properties, when you set Xmx2g, expect that the maximum memory consumption of this container may be up to 4Gb.

pankajsaini1987 commented 2 years ago

how i can change the .env file after docker install. where to change and how much maximum faces i can add in collection ?

pospielov commented 2 years ago

when you download CompreFace, there are two files in archive docker-compose.yml and .env. So you need to change .env file It should be changed before running docker-compose up -d if you already run CompreFace, you need to stop it first with docker-compose down, then change .env file, and then run CompreFace again with docker-compose up -d