huge disk usage by nginx logs

[presentpro]

Since successfully moving to the new image, I am having rapid and huge data use by the docker container. the below can be seen just a few minutes apart. This expands until the server itself crashes. Is there a setting I should be using to stop this occuring, is this something others are seeing?

presentprolondon@instance:~$ sudo du -h /var/lib | sort -rh | head -n 20
9.6G    /var/lib
9.5G    /var/lib/docker/overlay2
9.5G    /var/lib/docker
9.2G    /var/lib/docker/overlay2/8ec469dd85adf61d46d177a3fb57dd6ece1cc3517936ceb2c3c56cbb846522e7
4.7G    /var/lib/docker/overlay2/8ec469dd85adf61d46d177a3fb57dd6ece1cc3517936ceb2c3c56cbb846522e7/merged
4.6G    /var/lib/docker/overlay2/8ec469dd85adf61d46d177a3fb57dd6ece1cc3517936ceb2c3c56cbb846522e7/merged/var/log/nginx
4.6G    /var/lib/docker/overlay2/8ec469dd85adf61d46d177a3fb57dd6ece1cc3517936ceb2c3c56cbb846522e7/merged/var/log
4.6G    /var/lib/docker/overlay2/8ec469dd85adf61d46d177a3fb57dd6ece1cc3517936ceb2c3c56cbb846522e7/merged/var
4.6G    /var/lib/docker/overlay2/8ec469dd85adf61d46d177a3fb57dd6ece1cc3517936ceb2c3c56cbb846522e7/diff/var/log/nginx
4.6G    /var/lib/docker/overlay2/8ec469dd85adf61d46d177a3fb57dd6ece1cc3517936ceb2c3c56cbb846522e7/diff/var/log
4.6G    /var/lib/docker/overlay2/8ec469dd85adf61d46d177a3fb57dd6ece1cc3517936ceb2c3c56cbb846522e7/diff/var
4.6G    /var/lib/docker/overlay2/8ec469dd85adf61d46d177a3fb57dd6ece1cc3517936ceb2c3c56cbb846522e7/diff
140M    /var/lib/apt/lists
140M    /var/lib/apt
100M    /var/lib/docker/overlay2/d930f8a82147e55e017b5db9a353d295100028915dc9675079a7b9c8ce13e38d/diff
100M    /var/lib/docker/overlay2/d930f8a82147e55e017b5db9a353d295100028915dc9675079a7b9c8ce13e38d
100M    /var/lib/docker/overlay2/097f32cada0b2d40fa66a9b98f0dc09e0e9f1184968a63773ba3e0ffa5bcfccb/diff
100M    /var/lib/docker/overlay2/097f32cada0b2d40fa66a9b98f0dc09e0e9f1184968a63773ba3e0ffa5bcfccb
98M     /var/lib/docker/overlay2/d930f8a82147e55e017b5db9a353d295100028915dc9675079a7b9c8ce13e38d/diff/usr
98M     /var/lib/docker/overlay2/8ec469dd85adf61d46d177a3fb57dd6ece1cc3517936ceb2c3c56cbb846522e7/merged/usr

presentprolondon@instance:~$ sudo du -h /var/lib | sort -rh | head -n 20
11G     /var/lib/docker/overlay2/8ec469dd85adf61d46d177a3fb57dd6ece1cc3517936ceb2c3c56cbb846522e7
11G     /var/lib/docker/overlay2
11G     /var/lib/docker
11G     /var/lib
5.1G    /var/lib/docker/overlay2/8ec469dd85adf61d46d177a3fb57dd6ece1cc3517936ceb2c3c56cbb846522e7/merged
5.0G    /var/lib/docker/overlay2/8ec469dd85adf61d46d177a3fb57dd6ece1cc3517936ceb2c3c56cbb846522e7/merged/var/log/nginx
5.0G    /var/lib/docker/overlay2/8ec469dd85adf61d46d177a3fb57dd6ece1cc3517936ceb2c3c56cbb846522e7/merged/var/log
5.0G    /var/lib/docker/overlay2/8ec469dd85adf61d46d177a3fb57dd6ece1cc3517936ceb2c3c56cbb846522e7/merged/var
5.0G    /var/lib/docker/overlay2/8ec469dd85adf61d46d177a3fb57dd6ece1cc3517936ceb2c3c56cbb846522e7/diff/var/log/nginx
5.0G    /var/lib/docker/overlay2/8ec469dd85adf61d46d177a3fb57dd6ece1cc3517936ceb2c3c56cbb846522e7/diff/var/log
5.0G    /var/lib/docker/overlay2/8ec469dd85adf61d46d177a3fb57dd6ece1cc3517936ceb2c3c56cbb846522e7/diff/var
5.0G    /var/lib/docker/overlay2/8ec469dd85adf61d46d177a3fb57dd6ece1cc3517936ceb2c3c56cbb846522e7/diff
140M    /var/lib/apt/lists
140M    /var/lib/apt
100M    /var/lib/docker/overlay2/d930f8a82147e55e017b5db9a353d295100028915dc9675079a7b9c8ce13e38d/diff
100M    /var/lib/docker/overlay2/d930f8a82147e55e017b5db9a353d295100028915dc9675079a7b9c8ce13e38d
100M    /var/lib/docker/overlay2/097f32cada0b2d40fa66a9b98f0dc09e0e9f1184968a63773ba3e0ffa5bcfccb/diff
100M    /var/lib/docker/overlay2/097f32cada0b2d40fa66a9b98f0dc09e0e9f1184968a63773ba3e0ffa5bcfccb
98M     /var/lib/docker/overlay2/d930f8a82147e55e017b5db9a353d295100028915dc9675079a7b9c8ce13e38d/diff/usr
98M     /var/lib/docker/overlay2/8ec469dd85adf61d46d177a3fb57dd6ece1cc3517936ceb2c3c56cbb846522e7/merged/usr

presentprolondon@instance:~$ sudo du -h /var/lib | sort -rh | head -n 20
17G     /var/lib/docker/overlay2/8ec469dd85adf61d46d177a3fb57dd6ece1cc3517936ceb2c3c56cbb846522e7
17G     /var/lib/docker/overlay2
17G     /var/lib/docker
17G     /var/lib
8.2G    /var/lib/docker/overlay2/8ec469dd85adf61d46d177a3fb57dd6ece1cc3517936ceb2c3c56cbb846522e7/merged
8.1G    /var/lib/docker/overlay2/8ec469dd85adf61d46d177a3fb57dd6ece1cc3517936ceb2c3c56cbb846522e7/merged/var/log/nginx
8.1G    /var/lib/docker/overlay2/8ec469dd85adf61d46d177a3fb57dd6ece1cc3517936ceb2c3c56cbb846522e7/merged/var/log
8.1G    /var/lib/docker/overlay2/8ec469dd85adf61d46d177a3fb57dd6ece1cc3517936ceb2c3c56cbb846522e7/merged/var
8.1G    /var/lib/docker/overlay2/8ec469dd85adf61d46d177a3fb57dd6ece1cc3517936ceb2c3c56cbb846522e7/diff/var/log/nginx
8.1G    /var/lib/docker/overlay2/8ec469dd85adf61d46d177a3fb57dd6ece1cc3517936ceb2c3c56cbb846522e7/diff/var/log
8.1G    /var/lib/docker/overlay2/8ec469dd85adf61d46d177a3fb57dd6ece1cc3517936ceb2c3c56cbb846522e7/diff/var
8.1G    /var/lib/docker/overlay2/8ec469dd85adf61d46d177a3fb57dd6ece1cc3517936ceb2c3c56cbb846522e7/diff
140M    /var/lib/apt/lists
140M    /var/lib/apt
100M    /var/lib/docker/overlay2/d930f8a82147e55e017b5db9a353d295100028915dc9675079a7b9c8ce13e38d/diff
100M    /var/lib/docker/overlay2/d930f8a82147e55e017b5db9a353d295100028915dc9675079a7b9c8ce13e38d
100M    /var/lib/docker/overlay2/097f32cada0b2d40fa66a9b98f0dc09e0e9f1184968a63773ba3e0ffa5bcfccb/diff
100M    /var/lib/docker/overlay2/097f32cada0b2d40fa66a9b98f0dc09e0e9f1184968a63773ba3e0ffa5bcfccb
98M     /var/lib/docker/overlay2/d930f8a82147e55e017b5db9a353d295100028915dc9675079a7b9c8ce13e38d/diff/usr
98M     /var/lib/docker/overlay2/8ec469dd85adf61d46d177a3fb57dd6ece1cc3517936ceb2c3c56cbb846522e7/merged/usr

[Seji64]

Ouch, thats huge nginx logs. Didn't expect the will grow that large. Probably i should disable those nginx logs...

[Seji64]

Just a quick fix. You Can Test it with the latest build

[presentpro]

Super thank you! Initially it has stopped adding another gb in a few minutes which is great, I will check later to see that nothing re-occurs.

[presentpro]

This seems to have been resolved and now continues to perform well. there was also some large network requests I couldn't pin down that this also resolved. No idea how that would be connected, but a great fix.

[ControlCircuit]

I had the same problem and so I deleted Snidust Docker a week ago and reinstalled it. A week later, the nginx log files occupy 48GB of my hard drive again. I think the problem still exists.

sudo find / -size +500000  -exec ls -lahg {} \;
-rw-r--r-- 1 opc 17G Jul 30 06:49 /var/lib/docker/overlay2/3ac8cfc2998ab8ce459fee4f30c6ebdefeb5f13689650170ac1fb5fbc7b7726a/merged/var/log/nginx/error.log
-rw-r--r-- 1 opc 23G Jul 30 06:49 /var/lib/docker/overlay2/3ac8cfc2998ab8ce459fee4f30c6ebdefeb5f13689650170ac1fb5fbc7b7726a/merged/var/log/nginx/access.log
-rw-r--r-- 1 opc 17G Jul 30 06:49 /var/lib/docker/overlay2/3ac8cfc2998ab8ce459fee4f30c6ebdefeb5f13689650170ac1fb5fbc7b7726a/diff/var/log/nginx/error.log
-rw-r--r-- 1 opc 23G Jul 30 06:49 /var/lib/docker/overlay2/3ac8cfc2998ab8ce459fee4f30c6ebdefeb5f13689650170ac1fb5fbc7b7726a/diff/var/log/nginx/access.log

[Seji64]

Which docker image version you are using?

[ControlCircuit]

I use this line in my docker compose file: image: 'ghcr.io/seji64/snidust:latest'

Portianier shows me the follwing version:

[Seji64]

strange, maybe the default setting changed cause logging is not even specified in the nginx.conf

[Seji64]

grab the latest build/image and try again pls

[ControlCircuit]

I just re-pulled the latest build again on my server. I will report back in around a week if the logs stays low/empty. Thanks already for your fast fix.