search

littlebizzy / slickstack

Lightning-fast WordPress on Nginx
https://slickstack.io
GNU General Public License v3.0
629 stars 112 forks source link

ClamAV running out of RAM memory on small VMs during clamscan #87

Closed Bronislawsky closed 3 years ago

Bronislawsky commented 3 years ago

I have noticed that on low RAM vps that have no swap file ( linode / digital ocean at least ) clamscan fail because it is running out of ram

Here is a ref to help you easily create swapfile ;) https://linuxize.com/post/how-to-add-swap-space-on-ubuntu-20-04/

jessuppi commented 3 years ago

Thanks @Bronislawsky and please try to use more explanatory Issue titles --

As far as RAM usage in ClamAV it has been a problem for many years, and keeps getting worse, because their database of malware and "bad files" keeps growing larger.

Ref: https://unix.stackexchange.com/questions/114709/how-to-reduce-clamav-memory-usage Ref: https://forum.iredmail.org/topic13607-a-solution-to-clamav-consuming-too-much-memory.html Ref: https://www.linode.com/community/questions/9233/reducing-memory-usage-antivirus-recommendations Ref: https://www.howtoforge.com/community/threads/high-load-ram-clamd.82085/

Swap does not help the problem as far as I understand, and can make things worse. Besides, having SlickStack manage swap could end up conflicting with providers like Linode that manage swap themselves, etc.

Honestly I've never found ClamAV to be very effective, anyways, at least not for our LEMP stack. We might end up removing it completely from SlickStack for both of these reasons, unless a solution is found.

Using the "on-demand" version of ClamAV is also goofy, and does not seem like a serious solution...

Bronislawsky commented 3 years ago

Tested a few setup, from 2GB of RAM it seems ok but below 2GB most of the time the process gets killed

jessuppi commented 3 years ago

Tested a few setup, from 2GB of RAM it seems ok but below 2GB most of the time the process gets killed

An alternative might be checking how much RAM is available and conditionally installing ClamAV only on servers with 2GB or more of RAM memory available... perhaps also could include WPScan.

Ref: https://github.com/wpscanteam/wpscan

Bronislawsky commented 3 years ago

WPScan seems really great but is free for only non-commercial usage.. so I guess as soon as you run commercial stuff you need a license.. so it could not be enabled by default..

I tried for fun installing WPScan.. seems too bloated to me.. Installing Ruby, RubyGem. if everything was part of ubuntu repos that wouldnt be a problem..

jessuppi commented 3 years ago

An update to this discussion as SlickStack now includes a 2GB swapfile by default (e.g. when RAM is full):

https://github.com/littlebizzy/slickstack/blob/master/bash/ss-install-ubuntu-swap.txt

I'm hoping this alleviates the RAM exhaustion problem with ClamAV scans on smaller servers for now. I haven't done any tests however so feedback is always appreciated, thanks!

jessuppi commented 3 years ago

I tried for fun installing WPScan.. seems too bloated to me.. Installing Ruby, RubyGem. if everything was part of ubuntu repos that wouldnt be a problem..

P.S. interesting... yah, I guess replacing ClamAV with WPScan wouldn't be any "lighter" per se.

Probably a conversation for another time re: WPScan. Thanks!