Docker Memory Cache

[alfureu]

Hm, running Koozic in a docker, with 2GB RAM. I receive warnings from my NAS that the memory is being utilized by 90% whenever I start playing and transcoding music from Koozic -> DSub.

Is there a way how to limit it further? I think taking 50% of my Memory/Cache is a bit much for converting 10MB of songs sent out.

The problem with the two docker images (one for db and one for the app) is that whenever I touch these two through Portainer, they are unsuccessful to see each other anymore. Must be something with the docker-compose building process. However, it would be nice to be able to limit the Memory usage, eventually optimize the whole app.

[DocMarty84]

There is a performance issue in the case of folder scanning. This was improved in this commit, which will be available in the v2.2.0.

However, you mention it happens during the transcoding, which is weird. Indeed, we don't do anything fancy: we open a sub-process and call FFMpeg. It's surprising that there is such an increase in the memory. It would be interesting to see the memory usage of all processes in the system. Please note that the fact that a process uses memory doesn't mean it cannot be allocated to another process. So this might be a non-issue.

There is still an issue in the configuration you are running KooZic. It uses 4 processes, and each of them can use up to 2048 MiB without being flushed. This is the default configuration for multi-processing (see limit_memory_soft and limit_memory_hard). When installing through the regular installer, these values are tweaked based on the actual capacities of the machine. In Docker, the default values are used. You might want to use --workers=2 and limit_memory_soft=536870912 (536870912 = 512 1024 1024, therefore 512 MiB) in entrypoint.sh.

PS: on a side note, I think using Docker on a machine with limited resources is not always the best choice. I understand it's great for security and deployment easiness, but that also adds an extra layer for resource consumption. In such a use case, I recommend the regular installation with a dedicated user for the process.

[alfureu]

Thanks, this helped. Limited the workers and soft memory, and so far it seems a bit more normal. Will report back if I experience anything odd.

[alfureu]

Actually, one more thing. I set the workers to 2 but in the logs it still appears as 4, see screenshot below. The memory was my biggest concern, but it is strange that the amount of workers is set to 2 and it still appears 4.

[DocMarty84]

You should rebuild the docker image to take the change into account.

The starting command should look like:

# Start koozic
su - koozic -c "/usr/local/koozic/odoo-bin \
                --workers=2 \
                --limit-time-cpu=1800 \
                --limit-time-real=3600 \
                --limit-memory-soft=536870912 \
                -d koozic-v2 \
                --db-filter=koozic-v2 \
                --db_host=db \
                --db_port=5432 \
                --db_user=koozic \
                --db_password=koozic \
                --without-demo=all \
                --no-database-list \
                --log-level=warn"

[alfureu]

You are absolutely right, just did it and all si good now. Thanks!