search

Cisco-Talos / clamav

ClamAV - Documentation is here: https://docs.clamav.net
https://www.clamav.net/
GNU General Public License v2.0
4.27k stars 692 forks source link

OnAccess scanning crashes with "malloc(): invalid next size (unsorted)" #1292

Open ocient-jlarson opened 3 months ago

ocient-jlarson commented 3 months ago

Describe the bug

After roughly two days clamav onaccess scanning will crash with the following message:

# systemctl status clamav-onaccess
● clamav-onaccess.service - ClamAV On Access Scanner
     Loaded: loaded (/etc/systemd/system/clamav-onaccess.service; enabled; vendor preset: enabled)
     Active: failed (Result: signal) since Wed 2024-06-19 14:12:39 UTC; 5 days ago
   Main PID: 1038358 (code=killed, signal=ABRT)

Jun 17 16:14:56 myhost clamonacc[1038358]: ClamInotif: excluding '/opt/ocient' (and all sub-directories)
Jun 17 16:14:56 myhost clamonacc[1038358]: ClamInotif: excluding '/home/ansible/.ansible' (and all sub-directories)
Jun 17 16:14:56 myhost clamonacc[1038358]: ClamInotif: excluding '/root/quarantine' (and all sub-directories)
Jun 17 16:14:56 myhost clamonacc[1038358]: ERROR: ClamInotif: could not add element to hash table for /opt/ocient
Jun 17 16:14:56 myhost clamonacc[1038358]: ERROR: ClamInotif: issue when adding watch for /opt/ocient
Jun 17 16:14:56 myhost clamonacc[1038358]: ERROR: ClamInotif: could not watch path '/opt', Invalid argument passed to function
Jun 17 16:14:56 myhost clamonacc[1038358]:  ClamInotif: extra scanning on inotify events enabled
Jun 19 14:12:38 myhost clamonacc[1038358]: malloc(): invalid next size (unsorted)
Jun 19 14:12:39 myhost systemd[1]: clamav-onaccess.service: Main process exited, code=killed, status=6/ABRT
Jun 19 14:12:39 myhost systemd[1]: clamav-onaccess.service: Failed with result 'signal'.

clamav-onaccess service file:

[Unit]
Description=ClamAV On Access Scanner
Requires=clamav-daemon.service
After=clamav-daemon.service syslog.target network.target

[Service]
Type=simple
User=root
ExecStartPre=/bin/bash -c "while [ ! -S /var/run/clamav/clamd.sock ]; do sleep 1; done"
ExecStart=/usr/local/sbin/clamonacc --verbose --fdpass -F --config-file=/etc/clamav/clamd.conf --log=/var/log/clamav/clamonacc.log

[Install]
WantedBy=multi-user.target

These are beefy physical boxes. Dual Xeon 6230R's with 1.5TB of memory. There are a single VM in the mix where this is also crashed. Thats just a simple 2vcpu and 8GB of memory VM. The systems with this issue represent a single "cluster" in a sense. All boxes have clamonacc in a failed state. Both freshclam and the daemon are running happily. Freshclam runs as clamav and the daemon along with clamonacc run as root.

Another smaller cluster with this also installed is fine. This is a smaller cluster of 8 nodes, same version of Ubuntu. A key difference between them this smaller cluster hasn't had the CIS benchmark ran against it for hardening. I'm beginning to suspect that is a culprit. I'm looking to see what check may be making it crabby

I also have scattered VM's running this, both 22.04 and 24.04. No issues.

I have restarted OnAccess with verbose messaging to see if I can shake anything else out.

How to reproduce the problem

Start ClamAv OnAccess scanning. Crashes after 2 days

clamconf --config-dir /etc/clamav/

Checking configuration files in /etc/clamav/

Config file: clamd.conf

AlertExceedsMax disabled CacheSize = "65536" PreludeEnable disabled PreludeAnalyzerName disabled LogFile = "/var/log/clamav/clamav.log" LogFileUnlock disabled LogFileMaxSize = "1048576" LogTime = "yes" LogClean disabled LogSyslog disabled LogFacility = "LOG_LOCAL6" LogVerbose disabled LogRotate = "yes" ExtendedDetectionInfo = "yes" PidFile disabled TemporaryDirectory = "/var/lib/clamav/tmp" DatabaseDirectory = "/var/lib/clamav" OfficialDatabaseOnly disabled FailIfCvdOlderThan disabled LocalSocket = "/run/clamav/clamd.sock" LocalSocketGroup = "clamav" LocalSocketMode = "660" FixStaleSocket = "yes" TCPSocket disabled TCPAddr disabled MaxConnectionQueueLength = "200" StreamMaxLength = "104857600" StreamMinPort = "1024" StreamMaxPort = "2048" MaxThreads = "10" ReadTimeout = "120" CommandReadTimeout = "30" SendBufTimeout = "500" MaxQueue = "100" IdleTimeout = "30" ExcludePath = "^/proc/", "^/sys/", "^/var/opt/ocient/", "^/opt/ocient/", "^/snap/", "^/dev/", "^/run/", "^/var/lib/lxcfs/cgroup/", "^/var/snap/lxd/common/", "^/var/spool/postfix/dev/", "^/home/ansible/.ansible/", "^/tmp/ansible" MaxDirectoryRecursion = "20" FollowDirectorySymlinks disabled FollowFileSymlinks disabled CrossFilesystems = "yes" SelfCheck = "600" ConcurrentDatabaseReload = "yes" DisableCache disabled VirusEvent disabled ExitOnOOM disabled AllowAllMatchScan = "yes" Foreground disabled Debug disabled LeaveTemporaryFiles disabled GenerateMetadataJson disabled User = "root" Bytecode = "yes" BytecodeSecurity = "TrustSigned" BytecodeTimeout = "10000" BytecodeUnsigned disabled BytecodeMode = "Auto" DetectPUA disabled ExcludePUA disabled IncludePUA disabled ScanPE = "yes" ScanELF = "yes" ScanMail = "yes" ScanPartialMessages disabled PhishingSignatures = "yes" PhishingScanURLs = "yes" HeuristicAlerts = "yes" HeuristicScanPrecedence disabled StructuredDataDetection disabled StructuredMinCreditCardCount = "3" StructuredMinSSNCount = "3" StructuredSSNFormatNormal = "yes" StructuredSSNFormatStripped disabled ScanHTML = "yes" ScanOLE2 = "yes" AlertBrokenExecutables disabled AlertBrokenMedia disabled AlertEncrypted disabled StructuredCCOnly disabled AlertEncryptedArchive disabled AlertEncryptedDoc disabled AlertOLE2Macros disabled AlertPhishingSSLMismatch disabled AlertPhishingCloak disabled AlertPartitionIntersection disabled ScanPDF = "yes" ScanSWF = "yes" ScanXMLDOCS = "yes" ScanHWP3 = "yes" ScanOneNote = "yes" ScanArchive = "yes" ForceToDisk disabled MaxScanTime disabled MaxScanSize = "419430400" MaxFileSize = "104857600" MaxRecursion = "17" MaxFiles = "10000" MaxEmbeddedPE = "41943040" MaxHTMLNormalize = "41943040" MaxHTMLNoTags = "8388608" MaxScriptNormalize = "20971520" MaxZipTypeRcg = "1048576" MaxPartitions = "50" MaxIconsPE = "100" MaxRecHWP3 = "16" PCREMatchLimit = "100000" PCRERecMatchLimit = "2000" PCREMaxFileSize = "104857600" OnAccessMountPath disabled OnAccessIncludePath = "/home", "/root", "/tmp", "/var/tmp", "/opt", "/usr/local/bin", "/usr/bin", "/usr/sbin" OnAccessExcludePath = "/var/opt/ocient", "/opt/ocient", "/snap", "/opt/cvdupdate", "/dev", "/run", "/sys", "/proc", "/home/ansible/.ansible", "/root/quarantine" OnAccessExcludeRootUID disabled OnAccessExcludeUID disabled OnAccessExcludeUname = "clamav", "ansible" OnAccessMaxFileSize = "5242880" OnAccessDisableDDD disabled OnAccessPrevention disabled OnAccessExtraScanning = "yes" OnAccessCurlTimeout = "5000" OnAccessMaxThreads = "5" OnAccessRetryAttempts disabled OnAccessDenyOnError disabled DevACOnly disabled DevACDepth disabled DevPerformance disabled DevLiblog disabled DisableCertCheck disabled AlgorithmicDetection = "yes" BlockMax disabled PhishingAlwaysBlockSSLMismatch disabled PhishingAlwaysBlockCloak disabled PartitionIntersection disabled OLE2BlockMacros disabled ArchiveBlockEncrypted disabled

Config file: freshclam.conf

LogFileMaxSize = "1048576" LogTime = "yes" LogSyslog = "yes" LogFacility = "LOG_LOCAL6" LogVerbose disabled LogRotate = "yes" PidFile disabled DatabaseDirectory = "/var/lib/clamav" Foreground disabled Debug disabled UpdateLogFile = "/var/log/clamav/freshclam.log" DatabaseOwner = "clamav" Checks = "24" DNSDatabaseInfo = "current.cvd.clamav.net" DatabaseMirror = "https://clamav-updates.mydomain.com" PrivateMirror disabled MaxAttempts = "3" ScriptedUpdates = "yes" TestDatabases = "yes" CompressLocalDatabase disabled ExtraDatabase disabled ExcludeDatabase disabled DatabaseCustomURL disabled HTTPProxyServer disabled HTTPProxyPort disabled HTTPProxyUsername disabled HTTPProxyPassword disabled HTTPUserAgent disabled NotifyClamd = "/etc/clamav/clamd.conf" OnUpdateExecute disabled OnErrorExecute disabled OnOutdatedExecute disabled LocalIPAddress disabled ConnectTimeout = "30" ReceiveTimeout = "60" Bytecode = "yes"

clamav-milter.conf not found

Software settings

Version: 1.3.1 Optional features supported: MEMPOOL AUTOIT_EA06 BZIP2 LIBXML2 PCRE2 ICONV JSON RAR

Database information

Database directory: /var/lib/clamav daily.cld: version 27317, sigs: 2063416, built on Tue Jun 25 08:26:12 2024 main.cvd: version 62, sigs: 6647427, built on Thu Sep 16 12:32:42 2021 bytecode.cvd: version 335, sigs: 86, built on Tue Feb 27 15:37:24 2024 Total number of signatures: 8710929

Platform information

uname: Linux 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 OS: Linux, ARCH: x86_64, CPU: x86_64 Full OS version: Ubuntu 20.04.6 LTS zlib version: 1.3.1 (1.3.1), compile flags: a9 platform id: 0x0a21c9c90800000000070500

Build information

GNU C: 7.5.0 (7.5.0) sizeof(void*) = 8 Engine flevel: 201, dconf: 201