search

Cisco-Talos / clamav

ClamAV - Documentation is here: https://docs.clamav.net
https://www.clamav.net/
GNU General Public License v2.0
4.48k stars 708 forks source link

Freshclam crash with DatabaseCustomURL for a CVD and also other files, affects versions 1.3.1, 1.3.2, 1.4.0 and 1.4.1 #1364

Open gotspatel opened 2 months ago

gotspatel commented 2 months ago

Describe the bug ---------------- I had version 1.3.0 installed and perfectly running on window server 2019 VM, Yesterday tried updating it to 1.4.0 but found issues with freshclam service (it stopped abruptly and immediately on start generating error in eventlog as below)

Faulting application name: freshclam.exe, version: 1.4.0.0, time stamp: 0x66bd0724
Faulting module name: ucrtbase.dll, version: 10.0.17763.6189, time stamp: 0xbc3e3f37
Exception code: 0xc0000005
Fault offset: 0x0000000000025990
Faulting process id: 0x2c68
Faulting application start time: 0x01daff5f791b2503
Faulting application path: C:\Program Files\ClamAV\freshclam.exe
Faulting module path: C:\Windows\System32\ucrtbase.dll
Report Id: 730971f8-e1fc-4528-a917-98a91128208a
Faulting package full name: 
Faulting package-relative application ID:

Then I tried Fresh Install of Version 1.4.0 Again, but same issue

Later I tried fresh install of 1.4.1, but same issue So I again tested with 1.3.1, 1.3.2 version also but same issue

I reverted back to 1.3.0 again and there is no problem for freshclam service it works flawlessly as before and updated the signatures

How to reproduce the problem ----------------------------

Try installing it on Windows Server 2019 with all VC Libs installed using abbodi1406 script

I did fresh install on a fresh VM and I was able to reproduce the same for all version 1.3.1, 1.3.2, 1.40. and 1.4.1

C:\Program Files\ClamAV>clamconf -n Checking configuration files in C:\Program Files\ClamAV

Config file: clamd.conf

LogFile = "C:\Program Files\ClamAV\logs\clamd.log" LogTime = "yes" LogVerbose = "yes" LogRotate = "yes" ExtendedDetectionInfo = "yes" TemporaryDirectory = "C:\temp\CLAMTemp" TCPSocket = "3310" TCPAddr = "127.0.0.1" ExcludePath = "C:\Windows", "C:\Scripts" SelfCheck = "1800" AlertBrokenExecutables = "yes" MaxRecursion = "40"

Config file: freshclam.conf

LogTime = "yes" LogRotate = "yes" Foreground = "yes" UpdateLogFile = "C:\Program Files\ClamAV\logs\freshclam.log" Checks = "24" DatabaseMirror = "database.clamav.net" DatabaseCustomURL = <<<< REMOVED AS IT HAS SENSITIVE INFORMATION >>>>

clamav-milter.conf not found

Software settings

Version: 1.3.0 Optional features supported: MEMPOOL AUTOIT_EA06 BZIP2 LIBXML2 PCRE2 JSON RAR

Database information

Database directory: C:\Program Files\ClamAV\database [3rd Party] badmacro.ndb: 706 sigs [3rd Party] blurl.ndb: 1953 sigs [3rd Party] bofhland_cracked_URL.ndb: 40 sigs [3rd Party] bofhland_malware_attach.hdb: 1836 sigs [3rd Party] bofhland_malware_URL.ndb: 4 sigs [3rd Party] bofhland_phishing_URL.ndb: 72 sigs bytecode.cvd: version 335, sigs: 86, built on Tue Feb 27 21:07:24 2024 daily.cld: version 27387, sigs: 2066357, built on Tue Sep 3 14:08:04 2024 daily.cvd: version 27389, sigs: 2066461, built on Thu Sep 5 14:03:25 2024 [3rd Party] foxhole.ign2: 6 sigs [3rd Party] foxhole_all.cdb: 149 sigs [3rd Party] foxhole_all.ndb: 101 sigs [3rd Party] foxhole_filename.cdb: 3609 sigs [3rd Party] foxhole_generic.cdb: 215 sigs [3rd Party] foxhole_js.cdb: 48 sigs [3rd Party] foxhole_js.ndb: 4 sigs [3rd Party] foxhole_mail.cdb: 37 sigs [3rd Party] hackingteam.hsb: 435 sigs [3rd Party] ignore_list.ign2: 1 sig [3rd Party] interserver256.hdb: 28766 sigs [3rd Party] interservertopline.db: 1138 sigs [3rd Party] javascript.ndb: 10557 sigs [3rd Party] junk.ndb: 55064 sigs [3rd Party] jurlbl.ndb: 29699 sigs [3rd Party] lott.ndb: 2337 sigs main.cvd: version 62, sigs: 6647427, built on Thu Sep 16 18:02:42 2021 [3rd Party] malware.expert.hdb: 1 sig [3rd Party] malwarehash.hsb: 1031 sigs [3rd Party] MiscreantPunch099-Low.ldb: 1199 sigs [3rd Party] phish.ndb: 30709 sigs [3rd Party] phishtank.ndb: 1 sig [3rd Party] porcupine.hsb: 183 sigs [3rd Party] porcupine.ndb: 1607 sigs [3rd Party] rogue.hdb: 7287 sigs [3rd Party] sanesecurity.ftm: 185 sigs [3rd Party] Sanesecurity_sigtest.yara: 54 sigs [3rd Party] Sanesecurity_spam.yara: 46 sigs [3rd Party] scam.ndb: 13097 sigs [3rd Party] securiteinfo.hdb: 49086 sigs [3rd Party] securiteinfo.ign2: 222 sigs [3rd Party] securiteinfoandroid.hdb: 29652 sigs [3rd Party] securiteinfoascii.hdb: 36181 sigs [3rd Party] securiteinfohtml.hdb: 32966 sigs [3rd Party] securiteinfoold.hdb: 4145583 sigs [3rd Party] securiteinfopdf.hdb: 3408 sigs [3rd Party] shell.hdb: 4277 sigs [3rd Party] shell.ldb: 57 sigs [3rd Party] shellb.db: 292 sigs [3rd Party] shelter.ldb: 62 sigs [3rd Party] sigwhitelist.ign2: 18 sigs [3rd Party] spam.ldb: 2 sigs [3rd Party] spamattach.hdb: 14 sigs [3rd Party] spamimg.hdb: 233 sigs [3rd Party] spam_marketing.ndb: 37626 sigs [3rd Party] spear.ndb: 1 sig [3rd Party] spearl.ndb: 1 sig [3rd Party] urlhaus.ndb: 10705 sigs [3rd Party] whitelist.fp: 3081 sigs [3rd Party] winnow.attachments.hdb: 1 sig [3rd Party] winnow_bad_cw.hdb: 1 sig [3rd Party] winnow_extended_malware.hdb: 1 sig [3rd Party] winnow_malware.hdb: 1 sig [3rd Party] winnow_malware_links.ndb: 133 sigs [3rd Party] winnow_phish_complete.ndb: 53 sigs Total number of signatures: 15326165

Platform information

uname: Microsoft Windows Server 6.2 SP0.0 Build 9200 OS: Windows, ARCH: AMD64, CPU: AMD64 zlib version: 1.3.1 (1.3.1), compile flags: 65 platform id: 0x1025c8c80800000000000792

Build information

Microsoft Visual C++: (0.7.146) sizeof(void*) = 8 Engine flevel: 200, dconf: 200

C:\Program Files\ClamAV>

micahsnyder commented 2 months ago

Can you confirm if you're the same person to report this issue through Discord?

I haven't heard of any compatibility issues on Windows with 1.3.1 or with Windows versions 8 or newer. So I am very surprised by the issue you're facing.

For ClamAV 1.4.1, 1.3.2, and 1.0.7 I think the will fail with a similar "0xc0000005 application error" on Windows 7.

With ClamAV 1.4.0 and 1.4.1 we provide PDB debugging symbol files, added to try to triage this issue on Windows 7. We didn't solve it, and with no requirement to continue support for Windows 7 we accepted the compatibility issue.

If you want to dig in deeper, you could try starting freshclam.exe from the 1.4.1 or 1.4.0 versions with WinDbg to see if it gives a stack trace or some better explanation for the application error.

gotspatel commented 2 months ago

YES Same Person

I was even able to replicate the same issues on windows 10 and windows 11 (fresh VM Created for testing Purpose)

I tested and verified same issue on below OS Server 2019, Server 2016, Windows 10 LTSC, Windows 11 Pro

I will try to use WinDbg and provide further details.

Thanks

gotspatel commented 2 months ago

Found the Culprit

the conf file I had untill now for 1.3.0 had CRLF and I changed it to LF and it works without any other modifications and without any issue for all versions under windows, especially in freshclam.conf

hope to get a solution from next update onward to allow both CRLF and LF in config files under windows, please

Thanks

micahsnyder commented 2 months ago

@gotspatel that's wild! Let's reopen this issue and rename it. That is absolutely a bug.

gotspatel commented 1 month ago

OK Again I Tried today to install clamAV 1.4.1 on a Fresh VM Windows Server 2019 Standard and the freshclam service is still failing

it crashes with ucrtbase.dll and nt.dll attached the evenviewer details and logs to investigate same happens in the old VM also, (We had reverted to 1.3.0 as it was a production VM and didn't want issues in it)

this VM was also supposed to be production but I wanted to try again, let me know if more details required, I really hope to get it working on windows please, No problem whatsoever with clamd.exe, clamscan.exe, clamdscan.exe

EventViewer.zip

gotspatel commented 1 month ago

@micahsnyder

I have pinpointed the issues in the freshclam.conf as below, Hope Now you can check what has changed that the versions after 1.3.0 freshclam config doesn't like blank lines and comments in between the urls list

Previously untill 1.3.0 version my freshclam had this EXACTLY IN THIS ORDER and with some blank lines and comment line in between (AND Was and still is WORKING FINE in 1.3.0)

DatabaseCustomURL https://urlhaus.abuse.ch/downloads/urlhaus.ndb
DatabaseCustomURL http://sigs.interserver.net/interserver256.hdb
DatabaseCustomURL http://sigs.interserver.net/interservertopline.db
DatabaseCustomURL http://sigs.interserver.net/shell.ldb
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/securiteinfo.hdb
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/securiteinfo.ign2
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/javascript.ndb
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/spam_marketing.ndb
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/securiteinfohtml.hdb
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/securiteinfoascii.hdb
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/securiteinfoandroid.hdb
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/securiteinfoold.hdb
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/securiteinfopdf.hdb
DatabaseCustomURL http://database.clamav.net/daily.cvd

# http://rbluri.interserver.net/usage.php  http://rbluri.interserver.net/

DatabaseCustomURL http://sigs.interserver.net/interserver256.hdb
DatabaseCustomURL http://sigs.interserver.net/interservertopline.db
DatabaseCustomURL http://sigs.interserver.net/shell.ldb
DatabaseCustomURL http://sigs.interserver.net/whitelist.fp
DatabaseCustomURL http://sigs.interserver.net/shellb.db
DatabaseCustomURL http://sigs.interserver.net/shell.hdb

I changed it as below, removing the blank and comment line from between (ORDER IS NOT IMPORTANT) it works with any order of url but there should not be blank line or comment line in between and it works with 1.3.1, 1.4.0 and 1.4.1

DatabaseCustomURL http://database.clamav.net/daily.cvd
DatabaseCustomURL http://sigs.interserver.net/shellb.db
DatabaseCustomURL http://sigs.interserver.net/shell.ldb
DatabaseCustomURL http://sigs.interserver.net/shell.hdb
DatabaseCustomURL http://sigs.interserver.net/whitelist.fp
DatabaseCustomURL https://urlhaus.abuse.ch/downloads/urlhaus.ndb
DatabaseCustomURL http://sigs.interserver.net/interserver256.hdb
DatabaseCustomURL http://sigs.interserver.net/interservertopline.db
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/javascript.ndb
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/securiteinfo.hdb
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/securiteinfo.ign2
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/spam_marketing.ndb
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/securiteinfoold.hdb
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/securiteinfopdf.hdb
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/securiteinfohtml.hdb
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/securiteinfoascii.hdb
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/securiteinfoandroid.hdb
micahsnyder commented 1 month ago

It is not a CRLF issue.

I was able to reproduce the issue with this smaller config:

DatabaseMirror http://localhost:8000
DatabaseCustomURL http://localhost:8000/extra.wdb
DatabaseCustomURL http://localhost:8000/daily.cvd

I'm hosting databases on the same system with port 8000 so as not to rate limit myself.

I'll have a fix for it shortly.