clamonacc only uses 1 thread if --fdpass is given or tcp is used

arch-user-france1 commented 2 years ago

Describe the bug clamonacc (parameters --verbose -F --fdpass) is using only one thread (look at clamdtop) and may even crash the system if prevention is on and is busy with a large file

How to reproduce the problem put this in your clamd config:

OnAccessMaxFileSize 100M
OnAccessMaxThreads 5
OnAccessIncludePath /home/programmieren
#OnAccessIncludePath /
OnAccessExcludePath /proc
OnAccessExcludePath /run
OnAccessExcludePath /dev
OnAccessExcludePath /sys
#OnAccessExcludePath /var/www/html/data
OnAccessExtraScanning yes
OnAccessPrevention yes
OnAccessExcludeUname root
OnAccessDisableDDD no

start clamonacc by clamonacc --fdpass -F --verbose

Replace this text with specific steps needed to reproduce the issue.

Replace this text with the output from the ClamAV command: Checking configuration files in /etc/clamav

Config file: clamd.conf

PreludeAnalyzerName = "ClamAV" LogFile = "/var/log/clamav/clamav.log" LogFileMaxSize = "4294967295" LogTime = "yes" LogRotate = "yes" ExtendedDetectionInfo = "yes" LocalSocket = "/var/run/clamav/clamd.ctl" LocalSocketGroup = "clamav" LocalSocketMode = "666" MaxConnectionQueueLength = "50" MaxThreads = "12" ReadTimeout = "240" SendBufTimeout = "200" MaxQueue = "128" MaxDirectoryRecursion disabled SelfCheck = "3600" User = "clamav" BytecodeTimeout = "60000" MaxScanTime = "120000" MaxScanSize = "314572800" MaxFileSize = "262144000" MaxRecursion disabled PCREMatchLimit = "10000" PCRERecMatchLimit = "5000" OnAccessIncludePath = "/home/programmieren" OnAccessExcludePath = "/proc", "/run", "/dev", "/sys" OnAccessExcludeUname = "root" OnAccessMaxFileSize = "104857600" OnAccessPrevention = "yes" OnAccessExtraScanning = "yes"

Config file: freshclam.conf

LogFileMaxSize = "4294967295" LogTime = "yes" LogRotate = "yes" UpdateLogFile = "/var/log/clamav/freshclam.log" Checks = "24" DatabaseMirror = "db.local.clamav.net", "database.clamav.net" MaxAttempts = "5" SafeBrowsing is DEPRECATED

Config file: clamav-milter.conf

LogFile = "/var/log/clamav/clamav-milter.log" LogTime = "yes" LogRotate = "yes" PidFile = "/var/run/clamav/clamav-milter.pid" TemporaryDirectory = "/tmp" User = "clamav" ClamdSocket = "unix:/var/run/clamav/clamd.ctl" MilterSocket = "/var/run/clamav/clamav-milter.ctl" MilterSocketGroup = "clamav" MilterSocketMode = "666" AddHeader = "Replace" LogInfected = "Off" LogClean = "Off"

Software settings

Version: 0.103.2 Optional features supported: MEMPOOL IPv6 FRESHCLAM_DNS_FIX AUTOIT_EA06 BZIP2 LIBXML2 PCRE2 ICONV JSON

Database information

Database directory: /var/lib/clamav [3rd Party] winnow_bad_cw.hdb: 1 sig [3rd Party] phishtank.ndb: 12985 sigs [3rd Party] junk.ndb: 55802 sigs [3rd Party] bofhland_malware_URL.ndb: 4 sigs [3rd Party] doppelstern.hdb: 1 sig [3rd Party] crdfam.clamav.hdb: 1 sig [3rd Party] winnow.attachments.hdb: 182 sigs [3rd Party] bofhland_phishing_URL.ndb: 72 sigs [3rd Party] winnow_malware_links.ndb: 133 sigs [3rd Party] jurlbl.ndb: 2839 sigs [3rd Party] bofhland_cracked_URL.ndb: 40 sigs [3rd Party] sanesecurity.ftm: 170 sigs bytecode.cvd: version 333, sigs: 92, built on Mon Mar 8 16:21:51 2021 daily.cld: version 26369, sigs: 1948295, built on Tue Nov 30 10:18:45 2021 [3rd Party] bofhland_malware_attach.hdb: 1836 sigs [3rd Party] sigwhitelist.ign2: 12 sigs [3rd Party] blurl.ndb: 842 sigs [3rd Party] scam.ndb: 12750 sigs main.cvd: version 62, sigs: 6647427, built on Thu Sep 16 14:32:42 2021 [3rd Party] rogue.hdb: 575 sigs [3rd Party] porcupine.ndb: 6449 sigs [3rd Party] phish.ndb: 28057 sigs [3rd Party] winnow_extended_malware.hdb: 245 sigs [3rd Party] spamimg.hdb: 200 sigs [3rd Party] spamattach.hdb: 14 sigs [3rd Party] winnow_malware.hdb: 293 sigs Total number of signatures: 8719317

Platform information

uname: Linux 5.11.0-40-generic #44~20.04.2-Ubuntu SMP Tue Oct 26 18:07:44 UTC 2021 x86_64 OS: linux-gnu, ARCH: x86_64, CPU: x86_64 Full OS version: Ubuntu 20.04.3 LTS zlib version: 1.2.11 (1.2.11), compile flags: a9 platform id: 0x0a217b7b0800000000090300

Build information

GNU C: 9.3.0 (9.3.0) CPPFLAGS: -Wdate-time -D_FORTIFY_SOURCE=2 CFLAGS: -g -O2 -fdebug-prefix-map=/build/clamav-TW4JTf/clamav-0.103.2+dfsg=. -fstack-protector-strong -Wformat -Werror=format-security -Wall -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 CXXFLAGS: -g -O2 -fdebug-prefix-map=/build/clamav-TW4JTf/clamav-0.103.2+dfsg=. -fstack-protector-strong -Wformat -Werror=format-security -Wall -D_FILE_OFFSET_BITS=64 LDFLAGS: -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -Wl,--as-needed Configure: '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--runstatedir=/run' '--disable-maintainer-mode' '--disable-dependency-tracking' 'CFLAGS=-g -O2 -fdebug-prefix-map=/build/clamav-TW4JTf/clamav-0.103.2+dfsg=. -fstack-protector-strong -Wformat -Werror=format-security -Wall -D_FILE_OFFSET_BITS=64' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' 'CXXFLAGS=-g -O2 -fdebug-prefix-map=/build/clamav-TW4JTf/clamav-0.103.2+dfsg=. -fstack-protector-strong -Wformat -Werror=format-security -Wall -D_FILE_OFFSET_BITS=64' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -Wl,--as-needed' '--with-dbdir=/var/lib/clamav' '--sysconfdir=/etc/clamav' '--disable-clamav' '--disable-unrar' '--enable-milter' '--enable-dns-fix' '--with-libjson' '--with-system-libmspack' '--with-libcurl=/usr' '--with-gnu-ld' '--with-systemdsystemunitdir=/lib/systemd/system' 'build_alias=x86_64-linux-gnu' 'OBJCFLAGS=-g -O2 -fdebug-prefix-map=/build/clamav-TW4JTf/clamav-0.103.2+dfsg=. -fstack-protector-strong -Wformat -Werror=format-security' sizeof(void*) = 8 Engine flevel: 123, dconf: 123

Attachments

currently nothing available

Please note: clamonacc works if no --fdpass is given clamonacc doesn't work with tcp connections - the same happens

Bug clamav/clamonacc doesn't create new threads when fdpass is given

arch-user-france1 commented 2 years ago

nobody answered. WHY!

micahsnyder commented 2 years ago

Hi @debian-user-france1 sorry to hear you're seeing crashes with clamonacc. I see that you're using 0.103.2. There was a stability fix in 0.103.4 that may resolve the issue you're facing. Can you upgrade to the latest patch version?

https://github.com/Cisco-Talos/clamav/blob/rel/0.103/NEWS.md#01034

arch-user-france1 commented 2 years ago

Didn't I say that I have had one 104 too that had the same? I'll check it anyways. I wasn't sure if it's the same..

sezuan commented 2 years ago

Same here using clamad/clamonacc on Debian Buster and Bullseye, which is 0.103.5 based. Seems to use only one thread.

arch-user-france1 commented 2 years ago

There's nothing mentioned in the Changelog too, performance optimisations and other security stuff is there. Manjaro Linux uses 104.3 and it's still happening. I think it's a client sided issue (since the client clamomacc doesn't have to make threads).

sezuan commented 2 years ago

I try to summarize that (applies to rel/0.103):

It's currently not possible to do multithreaded on access scanning in a safe manner, because:

clamonacc with --fdpass can't pass more than one file handle to clamd, while clamdscan with --multipass is able to do so.
Not using --fdpass is not an option because CAP_DAC_READ_SEARCH doesn't work because of https://github.com/Cisco-Talos/clamav/pull/135 is not in rel/0.103
clamd running as root (which is pretty bad idead) would be a solution.

Is this correct?

arch-user-france1 commented 2 years ago

And tcp is not working too, not able to instruct clamd to scan parallel. Seems to be due to an unfinished code. It works but slow - clamonacc doesn't create new threads only uses one thread to pass files to clamd IF USING FDPASS OR TCP but the socket works.

arch-user-france1 commented 2 years ago

Running clamd as root is not a solution if tcp is used - for example: I've got a clamd dedicated server and want to use that with my computer - it won't work good because there's only one thread created - and one thread is slow as hell.

Only clamonacc is affected.

arch-user-france1 commented 2 years ago

@sezuan any news?

micahsnyder commented 2 years ago

@debian-user-france1 I'm sorry that you're frustrated.
clamonacc is notoriously difficult to debug to reproduce a crash or find the cause of a crash.

Does the crash happen every now and then?

Can you cause the crash to happen consistently by doing something specific in the monitored directories?

Can you share the stdout/stderr output from clamonacc with the --verbose option enabled when the crash occurs? This might show some warnings or errors before the crash that could help triage the issue. But may not show anything useful. I have no idea.

arch-user-france1 commented 2 years ago

@debian-user-france1 I'm sorry that you're frustrated. clamonacc is notoriously difficult to debug to reproduce a crash or find the cause of a crash.

Does the crash happen every now and then?

Can you cause the crash to happen consistently by doing something specific in the monitored directories?

Can you share the stdout/stderr output from clamonacc with the --verbose option enabled when the crash occurs? This might show some warnings or errors before the crash that could help triage the issue. But may not show anything useful. I have no idea.

This is not a report about a crash. The problem is that clamonacc is only using one clamd thread if it connects to a remote machine using tcp. The same seems to happen if the argument --fdpass is given.

This bug makes clamonacc unusable because the scanning, which the remote machine (clamd server) does, only happens using one thread, and clamonacc just adds scanns to the queue instead of asking the remote machine for a new thread.

Now, especially if I would instruct clamonacc to block unscanned files, the system would get stuck. Remember that the same happens if --fdpass is used, and so it is not possible to run clamd under non-root privileges.

Oh and clammonacc --verbose just shows that it's putting unscanned files to queue while waiting until clamd scans a file using one thread. If there are 1024 files in queue it just doesn't put new files to queue and the system could break (in case of on-access blocking).

micahsnyder commented 2 years ago

This is not a report about a crash. The problem is that clamonacc is only using one clamd thread if it connects to a remote machine using tcp. The same seems to happen if the argument --fdpass is given.

Sorry, my mistake. I understand, now.

I wonder if clamdscan is the same issue with clamd using a single thread when receiving a number of separate scan requests with --fdpass, vs --stream or just filepath based scanning.

arch-user-france1 commented 2 years ago

No, clamdscan works fine, it's only clamonacc. It occurs using tcp or --fdpass.

This is not a report about a crash. The problem is that clamonacc is only using one clamd thread if it connects to a remote machine using tcp. The same seems to happen if the argument --fdpass is given.

Sorry, my mistake. I understand, now.

I wonder if clamdscan is the same issue with clamd using a single thread when receiving a number of separate scan requests with --fdpass, vs --stream or just filepath based scanning.

arch-user-france1 commented 2 years ago

How to reproduce:

Install clamd etc on two servers (you can try it with one too).
Instruct clamd using your clamd config file to not use UNIX sockets and set up a tcp socket.
Run clamdtop on the clamd host
Run clamonacc
See result

arch-user-france1 commented 2 years ago

Hello?????

devurandom commented 2 years ago

I also run into this problem on Fedora 36. I made some modifications to the command line flags used to start clamd and clamonacc, compared to the official Fedora 36 packages -- cf. the systemd unit files quoted below.

The effect of clamonacc not sending multiple FDs to clamd (and thus queueing them and giving clamd a chance to scan multiple files in parallel) is that the whole system comes to a crawl when a larger application starts (e.g. starting IntelliJ IDEA blocks the system for several minutes) and it becomes even impossible to run other applications (like a terminal, for figuring out what is going on) until the first application is done, because this "one file at a time" is global in the system, across all applications, not just per application.

I first tried to solve this by de-nicing clamd (to -10, to give it higher priority -- cf. the systemd unit files quoted below) in order to prevent other processes from blocking clamd from scanning files, but it turned out fights over CPU resources are not the problem -- the system is mostly idle during the time it is blocked. Looking at glances showed that the actual problem is that clamd only uses a single CPU at a time (while the other 7 are mostly idle). Looking at clamdtop shows that at any given time there is only one FILEDES command in the queue.

Side question: It would be interesting to know whether this starvation is counted (or could be counted, using some configurable parameters) negatively towards the I/O budget of the process that originally opened the file, so that the kernel's I/O scheduler would give other processes precedence and the system could more quickly resume to being usable.

# clamconf -n
Checking configuration files in /etc

Config file: clamd.d/scan.conf
------------------------------
LocalSocket = "/run/clamav/clamd.ctl"
LocalSocketGroup = "virusgroup"
LocalSocketMode = "660"
User = "clamscan"
OnAccessIncludePath = "/home"
OnAccessExcludeRootUID = "yes"
OnAccessExcludeUname = "clamscan"
OnAccessPrevention = "yes"

Config file: freshclam.conf
---------------------------
DatabaseMirror = "database.clamav.net"

mail/clamav-milter.conf not found

Software settings
-----------------
Version: 0.103.7
Optional features supported: MEMPOOL IPv6 AUTOIT_EA06 BZIP2 LIBXML2 PCRE2 ICONV JSON 

Database information
--------------------
Database directory: /var/lib/clamav
bytecode.cvd: version 333, sigs: 92, built on Mon Mar  8 16:21:51 2021
main.cvd: version 62, sigs: 6647427, built on Thu Sep 16 14:32:42 2021
daily.cld: version 26658, sigs: 2002214, built on Wed Sep 14 09:58:32 2022
Total number of signatures: 8649733

Platform information
--------------------
uname: Linux 5.19.8-200.fc36.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 8 19:02:21 UTC 2022 x86_64
OS: linux-gnu, ARCH: x86_64, CPU: x86_64
zlib version: 1.2.11 (1.2.11), compile flags: a9
platform id: 0x0a21808008000000020c0101

Build information
-----------------
GNU C: 12.1.1 20220507 (Red Hat 12.1.1-1) (12.1.1)
CPPFLAGS: -I/usr/include/libprelude
CFLAGS: -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64  -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection  -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64
CXXFLAGS: -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64  -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection
LDFLAGS: -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1 -Wl,-dT,/builddir/build/BUILD/clamav-0.103.7/.package_note-clamav-0.103.7-1.fc36.x86_64.ld  -lprelude
Configure: '--build=x86_64-redhat-linux-gnu' '--host=x86_64-redhat-linux-gnu' '--program-prefix=' '--disable-dependency-tracking' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec' '--localstatedir=/var' '--sharedstatedir=/var/lib' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--enable-milter' '--disable-clamav' '--disable-static' '--disable-zlib-vcheck' '--disable-unrar' '--enable-id-check' '--enable-dns' '--with-dbdir=/var/lib/clamav' '--with-group=clamupdate' '--with-user=clamupdate' '--disable-rpath' '--disable-silent-rules' '--enable-clamdtop' '--enable-prelude' 'build_alias=x86_64-redhat-linux-gnu' 'host_alias=x86_64-redhat-linux-gnu' 'CXX=g++' 'CXXFLAGS=-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64  -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection' 'LDFLAGS=-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1 -Wl,-dT,/builddir/build/BUILD/clamav-0.103.7/.package_note-clamav-0.103.7-1.fc36.x86_64.ld' 'CC=gcc' 'CFLAGS=-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64  -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection' 'LT_SYS_LIBRARY_PATH=/usr/lib64:' 'PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig'
sizeof(void*) = 8
Engine flevel: 128, dconf: 128

# dnf info clamd clamav clamav-data clamav-filesystem clamav-lib clamav-update
Last metadata expiration check: 0:01:00 ago on Wed 14 Sep 2022 11:54:22 AM CEST.
Installed Packages
Name         : clamav
Version      : 0.103.7
Release      : 1.fc36
Architecture : x86_64
Size         : 13 M
Source       : clamav-0.103.7-1.fc36.src.rpm
Repository   : @System
From repo    : updates
Summary      : End-user tools for the Clam Antivirus scanner
URL          : https://www.clamav.net/
License      : GPLv2
Description  : Clam AntiVirus is an anti-virus toolkit for UNIX. The main purpose of this
             : software is the integration with mail servers (attachment scanning). The
             : package provides a flexible and scalable multi-threaded daemon, a command
             : line scanner, and a tool for automatic updating via Internet. The programs
             : are based on a shared library distributed with the Clam AntiVirus package,
             : which you can use with your own software. The virus database is based on
             : the virus database from OpenAntiVirus, but contains additional signatures
             : (including signatures for popular polymorphic viruses, too) and is KEPT UP
             : TO DATE.

Name         : clamav-data
Version      : 0.103.7
Release      : 1.fc36
Architecture : noarch
Size         : 219 M
Source       : clamav-0.103.7-1.fc36.src.rpm
Repository   : @System
From repo    : updates
Summary      : Virus signature data for the Clam Antivirus scanner
URL          : https://www.clamav.net/
License      : GPLv2
Description  : This package contains the virus-database needed by clamav. This
             : database should be updated regularly; the 'clamav-update' package
             : ships a corresponding cron-job. Use this package when you want a
             : working (but perhaps outdated) virus scanner immediately after package
             : installation.

Name         : clamav-filesystem
Version      : 0.103.7
Release      : 1.fc36
Architecture : noarch
Size         : 26 k
Source       : clamav-0.103.7-1.fc36.src.rpm
Repository   : @System
From repo    : updates
Summary      : Filesystem structure for clamav
URL          : https://www.clamav.net/
License      : GPLv2
Description  : This package provides the filesystem structure and contains the
             : user-creation scripts required by clamav.

Name         : clamav-lib
Version      : 0.103.7
Release      : 1.fc36
Architecture : x86_64
Size         : 2.2 M
Source       : clamav-0.103.7-1.fc36.src.rpm
Repository   : @System
From repo    : updates
Summary      : Dynamic libraries for the Clam Antivirus scanner
URL          : https://www.clamav.net/
License      : GPLv2
Description  : This package contains dynamic libraries shared between applications
             : using the Clam Antivirus scanner.

Name         : clamav-update
Version      : 0.103.7
Release      : 1.fc36
Architecture : x86_64
Size         : 163 M
Source       : clamav-0.103.7-1.fc36.src.rpm
Repository   : @System
From repo    : updates
Summary      : Auto-updater for the Clam Antivirus scanner data-files
URL          : https://www.clamav.net/
License      : GPLv2
Description  : This package contains programs which can be used to update the clamav
             : anti-virus database automatically. It uses the freshclam(1) utility for
             : this task. To activate it use, uncomment the entry in /etc/cron.d/clamav-update.
             : Use this package when you go updating the virus database regulary and
             : do not want to download a >160MB sized rpm-package with outdated virus
             : definitions.

Name         : clamd
Version      : 0.103.7
Release      : 1.fc36
Architecture : x86_64
Size         : 249 k
Source       : clamav-0.103.7-1.fc36.src.rpm
Repository   : @System
From repo    : updates
Summary      : The Clam AntiVirus Daemon
URL          : https://www.clamav.net/
License      : GPLv2
Description  : The Clam AntiVirus Daemon
             : See the README file how this can be done with a minimum of effort.
             : This package contains a generic system wide clamd service which is
             : e.g. used by the clamav-milter package.

Available Packages
Name         : clamav-lib
Version      : 0.103.7
Release      : 1.fc36
Architecture : i686
Size         : 1.0 M
Source       : clamav-0.103.7-1.fc36.src.rpm
Repository   : updates
Summary      : Dynamic libraries for the Clam Antivirus scanner
URL          : https://www.clamav.net/
License      : GPLv2
Description  : This package contains dynamic libraries shared between applications
             : using the Clam Antivirus scanner.

Name         : clamav-update
Version      : 0.103.7
Release      : 1.fc36
Architecture : i686
Size         : 96 k
Source       : clamav-0.103.7-1.fc36.src.rpm
Repository   : updates
Summary      : Auto-updater for the Clam Antivirus scanner data-files
URL          : https://www.clamav.net/
License      : GPLv2
Description  : This package contains programs which can be used to update the clamav
             : anti-virus database automatically. It uses the freshclam(1) utility for
             : this task. To activate it use, uncomment the entry in /etc/cron.d/clamav-update.
             : Use this package when you go updating the virus database regulary and
             : do not want to download a >160MB sized rpm-package with outdated virus
             : definitions.

# systemctl cat clamav-clamonacc.service
# /etc/systemd/system/clamav-clamonacc.service
# Adapted from https://github.com/Cisco-Talos/clamav/blob/8b6e53a08a31ea7c792fac2f2fb7a1775a6dc7e5/clamonacc/clamav-clamonacc.service.in
[Unit]
Description=ClamAV On-Access Scanner
Documentation=man:clamonacc(8) man:clamd.conf(5) https://docs.clamav.net/
Requires=clamav-daemon.service
After=clamav-daemon.service syslog.target network.target

[Service]
Type=simple
User=root
ExecStartPre=/bin/bash -c "while [ ! -S /run/clamav/clamd.ctl ]; do sleep 1; done"
ExecStart=/usr/sbin/clamonacc --foreground=true --fdpass=true --config-file=/etc/clamd.d/scan.conf

[Install]
WantedBy=multi-user.target

# systemctl cat clamav-daemon.service
# /etc/systemd/system/clamav-daemon.service
# Adapted from https://github.com/Cisco-Talos/clamav/blob/8b6e53a08a31ea7c792fac2f2fb7a1775a6dc7e5/clamd/clamav-daemon.service.in
[Unit]
Description=Clam AntiVirus userspace daemon
Documentation=man:clamd(8) man:clamd.conf(5) https://docs.clamav.net/
Requires=clamav-daemon.socket
# From Fedora's /usr/lib/systemd/system/clamd@.service:
After = syslog.target nss-lookup.target network.target
# Check for database existence
ConditionPathExistsGlob=/var/lib/clamav/main.{c[vl]d,inc}
ConditionPathExistsGlob=/var/lib/clamav/daily.{c[vl]d,inc}

[Service]
# Partially from Fedora's /usr/lib/systemd/system/clamd@.service:
ExecStart=/usr/sbin/clamd --foreground=true --config-file=/etc/clamd.d/scan.conf
# From Fedora's /usr/lib/systemd/system/clamd@.service:
Restart = on-failure
# Reload the database
ExecReload=/bin/kill -USR2 $MAINPID
TimeoutStartSec=420
# clamd regularly blocks other processes because it cannot keep up with the amount of work shoved its way by clamonacc:
Nice=-10

[Install]
WantedBy=multi-user.target
Also=clamav-daemon.socket

# systemctl cat clamav-daemon.socket
# /etc/systemd/system/clamav-daemon.socket
# Adapted from https://github.com/Cisco-Talos/clamav/blob/8b6e53a08a31ea7c792fac2f2fb7a1775a6dc7e5/clamd/clamav-daemon.socket.in
[Unit]
Description=Socket for Clam AntiVirus userspace daemon
Documentation=man:clamd(8) man:clamd.conf(5) https://docs.clamav.net/
# Check for database existence
ConditionPathExistsGlob=/var/lib/clamav/main.{c[vl]d,inc}
ConditionPathExistsGlob=/var/lib/clamav/daily.{c[vl]d,inc}

[Socket]
ListenStream=/run/clamav/clamd.ctl
#ListenStream=1024
SocketUser=clamscan
SocketGroup=virusgroup
RemoveOnStop=True

[Install]
WantedBy=sockets.target

amsnek commented 1 year ago

hmm could this explain the performance issues that are showing when clamonacc is used?

no matter which version, 0.103.7 or 0.105.7 -> once clamonacc is used with --fdpass (clamd runs as user clamav, clamonacc as root with --foreground and --fdpass) i constantly get these errors when the system is active IO wise:

clamav-clamonacc[1756008]: ERROR: ClamClient: Connection to clamd failed, Timeout was reached. clamav-clamonacc[1756008]: ClamClient: Connection to clamd re-established. clamav-clamonacc[1756008]: ERROR: ClamClient: Connection to clamd failed, Timeout was reached. clamav-clamonacc[1756008]: ClamClient: Connection to clamd re-established.

also, it seems 0.105.1 often does not free up space on filesystems it watches, interestingly enough lsof/fuser do not show the files as deleted or at all -> restarting clamav-daemon will free up the space

are there any settings or compilation opptions that can be used to make clamonacc usable?

arch-user-france1 commented 1 year ago

Honestly, clamonacc is trash and the team has no interest in fixing it at all. There are no settings for that and it's not a compiling problem, it's code that never really worked.

But now at least I know why I had to restart the system to get the space ^^

If you have the required knowledge, you could rewrite clamonacc or fork it; but I don't think that's your intention. This issue has been ignored since it was created, so I don't think you're ever going to use clamonacc anyway anymore. Use clamd as root, that would fix your problems.

Not even Windows works that way. Sure, it scans files as they are downloaded (created), but it doesn't block access to them I believe, and isn't that the intention of clamonacc?

I guess you should not care about AV systems on Linux, Viruses are mostly too fresh to be in the database of clamav. And clamav doesn't do machine learning. Either use propietary antivirus scanners; or forget about it. Clamav is not thought to be used for your purpose.

amsnek commented 1 year ago

Hmhm I digged a bit deeper now and it seems that at least my issue with

clamav-clamonacc[1756008]: ERROR: ClamClient: Connection to clamd failed, Timeout was reached.
clamav-clamonacc[1756008]: ClamClient: Connection to clamd re-established.

was caused by (stracing clamonacc):

read(3, 0x7ffcbd7a8140, 4096)           = -1 EMFILE (Too many open files)
nanosleep({tv_sec=3, tv_nsec=0}, 0x7ffcbd7a7860) = 0

-> greatly increased the max open files in the system unit file:

[Service]
LimitNOFILE=24000

While the open files increases up to ~22k it seems to remain stable now. I am using rkhunter on / to "simulate" aggressive load. Also using:

OnAccessMountPath /

Also increased the threads/maxthreads in clamd.conf and it does use more threads? though clamdtop does not reflect this 🤔 ? I am confused.

grep -i thread /etc/clamav/clamd.conf
MaxThreads 60
OnAccessMaxThreads 24

pstree -p 185920
clamonacc(185920)─┬─{clamonacc}(185922)
                  ├─{clamonacc}(185923)
                  ├─{clamonacc}(185924)
                  ├─{clamonacc}(185925)
                  ├─{clamonacc}(185926)
                  ├─{clamonacc}(185927)
                  ├─{clamonacc}(185928)
                  ├─{clamonacc}(185929)
                  ├─{clamonacc}(185930)
                  ├─{clamonacc}(185931)
                  ├─{clamonacc}(185932)
                  ├─{clamonacc}(185933)
                  ├─{clamonacc}(185934)
                  ├─{clamonacc}(185935)
                  ├─{clamonacc}(185936)
                  ├─{clamonacc}(185937)
                  ├─{clamonacc}(185938)
                  ├─{clamonacc}(185939)
                  ├─{clamonacc}(185940)
                  ├─{clamonacc}(185941)
                  ├─{clamonacc}(185942)
                  ├─{clamonacc}(185943)
                  ├─{clamonacc}(185944)
                  ├─{clamonacc}(185945)
                  └─{clamonacc}(185946)

sadly, there dont seem (non cloud based) onaccess scanners for linux (anymore), or at least none that i am aware about.

arch-user-france1 commented 1 year ago

First: there is no antivirus which is open source, not stealing you, other than clamav. At least no other popular.

And the remaining stuff It doesn't use more threads because you either used --fdpass or used TCP. That's what this issue is about, you've just stumbled about it. Don't use clamonacc - or fix it, we would be thankful.

amsnek commented 1 year ago

I do not have the required skills to fix it, otherwise I would attempt todo so. But it seems, at least from pstree viewpoint, that it does use threads up to the defined number in clamd.conf. but yeah, clamdtop says otherwise...

Cisco-Talos / clamav