Sanster / IOPaint

Image inpainting tool powered by SOTA AI Model. Remove any unwanted object, defect, people from your pictures or erase and replace(powered by stable diffusion) any thing on your pictures.
https://www.iopaint.com/
Apache License 2.0
18.8k stars 1.92k forks source link

[BUG] OOM - memory leak? #560

Open manuelkamp opened 4 weeks ago

manuelkamp commented 4 weeks ago

Model lama

Describe the bug installed via pip. made a systemd service to run it on my Ubunut 22.04 LXC.

[Unit]
Description=IOPaint
After=syslog.target network.target

[Service]
SuccessExitStatus=143

User=root
Group=root

Type=simple

WorkingDirectory=/root
ExecStart=/usr/local/bin/iopaint start --model=lama --device=cpu --host 0.0.0.0 --port=8080
ExecStop=/bin/kill -15 $MAINPID
Restart=on-failure

[Install]
WantedBy=multi-user.target

use a self coded windows application, using cefsharp (chromium browser engine) to call url and do some object removals. On every object removed (does not matter if it is the same picture or many different pictures), RAM usage rises until OOM. (the RAM usage rising is also replicated via regular Browser (Edge) and therefore not related to cefsharp usage)

Aug 16 22:33:12 srv-inpaint systemd[1]: iopaint.service: A process of this unit has been killed by the OOM killer.
Aug 16 22:33:13 srv-inpaint systemd[1]: iopaint.service: Main process exited, code=killed, status=9/KILL
Aug 16 22:33:13 srv-inpaint systemd[1]: iopaint.service: Failed with result 'oom-kill'.
Aug 16 22:33:13 srv-inpaint systemd[1]: iopaint.service: Consumed 16min 11.494s CPU time.
Aug 18 20:27:44 srv-inpaint systemd[1]: iopaint.service: Main process exited, code=killed, status=9/KILL
Aug 18 20:27:44 srv-inpaint systemd[1]: iopaint.service: Failed with result 'signal'.
Aug 18 20:27:44 srv-inpaint systemd[1]: iopaint.service: Consumed 4min 29.309s CPU time.
Aug 18 20:27:45 srv-inpaint systemd[1]: iopaint.service: Scheduled restart job, restart counter is at 5.
Aug 18 20:27:45 srv-inpaint systemd[1]: Stopped IOPaint.
Aug 18 20:27:45 srv-inpaint systemd[1]: iopaint.service: Consumed 4min 29.309s CPU time.
Aug 18 20:27:45 srv-inpaint systemd[1]: Started IOPaint.

System Info Platform: Linux-6.8.12-1-pve-x86_64-with-glibc2.35 Software version used

manuelkamp commented 4 weeks ago

related to #308 ?

manuelkamp commented 4 weeks ago

also related to #414 ?