search

asterics / AsTeRICS

The Assistive Technology Rapid Integration & Construction Set
http://www.asterics.eu
Other
53 stars 27 forks source link

Memory consumption of JavaCV / Facetracking #334

Open ChrisVeigl opened 4 years ago

ChrisVeigl commented 4 years ago

I just tested the performance of the facetracking on a RaspberryPi4 (4GB RAM, Raspbian buster).

I also investigated the memory utilisation on the RaspberryPi: After starting the ARE, it uses about 120MB of RAM when the autostart model (menu) is running (similar to the x86 version). After the "XFacetracker" model was started, RAM consumption went up steadily (about 5MB per second) until about 1,6GB of RAM were occupied. At this time, about 80 java threads existed (see screenshot of htop)

ScreenHunter 250

When the model was closed (and the menu was displayed again), the RAM was not released. After about 15 minutes, the garbage collector suddenly released about 500MB. I am not sure if the garbage collector would release unused RAM sooner (or more often if less RAM is available, eg. in the 1GB or 2GB version of the Raspi4)

Originally posted by @ChrisVeigl in https://github.com/asterics/AsTeRICS/pull/332#issuecomment-619282311

Memory consumption is also very high on PC platforms. I think it would make sense to investige this issue further (it seems to exist also in the master branch / previous releases) - maybe the garbage collection can be triggered "manually".

bmedicke commented 4 years ago

On macOS RAM consumption quickly rises to about 2GB and then the garbage collector kicks in and the cycle repeats (XFacetracker model).

ChrisVeigl commented 4 years ago

i suggest to call System.cg() in the frame grabber callback function, say every 300 frames (or better: every n frames - adjustable via a system- or plugin property). This introduces a small delay which is barely noticable but the memory consumption of the facetracking model stays around 50MB (tested on RPi 4).

deinhofer commented 4 years ago

i suggest to call System.cg() in the frame grabber callback function, say every 300 frames (or better: every n frames - adjustable via a system- or plugin property). This introduces a small delay which is barely noticable but the memory consumption of the facetracking model stays around 50MB (tested on RPi 4).

This is a typical optimization problem, using more RAM improves throughput and reduces GC pause time. I have tested it with a regular System.gc call but I did not like the short interruptions in mouse movement. So I tried to simply reduce the max heap size and also tried -XX:NewRatio which defines the ratio of the young generation objects (which frame objects should be). It worked quite well on my RPi3 and also on a normal Desktop. I have added comments to the start scripts so someone could even reduce the size own his own in case of a very low memory device.

ChrisVeigl commented 4 years ago

I'm not sure if I like this solution ;-) What happens when a plugin/model really needs more RAM - java.lang.OutOfMemoryError ? it seems to be unavoidable that a short delay occurs when the gc runs. having a parameter to control the gc period, you can decide to save RAM or to avoid lags. (a default could be set to several minutes so that the short delay is no problem).

is there really no efficient way in Java to free the unused image frame memory immeadely after processing?