Freshclam DatabaseCustomURL failure has confusing "Blocked by CDN" message and stops downloading files

armanmisdar commented 2 years ago

Dear Admin, I'm having trouble updating my FreshClam signatures. Here's the error that I got:

WARNING: fc_download_url_databases: fc_download_url_database failed: Forbidden; Blocked by CDN (17) ERROR: Database update process failed: Forbidden; Blocked by CDN WARNING: FreshClam was forbidden from downloading a database.

Please advise

ragusaa commented 2 years ago

Hi,

Do you happen to have a Russian IP? https://github.com/Cisco-Talos/clamav/issues/500

Thanks, Andy

armanmisdar commented 2 years ago

Thanks Andy. My IP is located in Asia, to be more precise - Malaysia

ragusaa commented 2 years ago

Hi, Sorry I missed your reply yesterday. What version of clam are you running? Older versions are blocked from downloading signatures. Could you post the output of 'clamconf -n'?

Thanks, Andy

armanmisdar commented 2 years ago

Thanks Andy. My ClamAV version is 0.103.6

And below here is the output of clamconf -n

Config file: clamd.conf

PreludeAnalyzerName = "ClamAV" LogFile = "/var/log/clamav/clamav.log" LogTime = "yes" LogRotate = "yes" PidFile = "/run/clamav/clamd.pid" TemporaryDirectory = "/tmp" LocalSocket = "/run/clamav/clamd.ctl" LocalSocketGroup = "clamav" LocalSocketMode = "660" CommandReadTimeout = "5" ExcludePath = "^/proc/", "^/sys/", "^/var/hotcopy/", "/usr/share/doc/clamav-*/test/" SelfCheck = "3600" VirusEvent = "no" User = "clamav" BytecodeTimeout = "60000" DetectPUA = "yes" ExcludePUA = "PUA.Win.Packer", "PUA.Win.Trojan.Packed", "PUA.Win.Trojan.Molebox", "PUA.Win.Packer.Upx", "PUA.Doc.Packed" ScanPartialMessages = "yes" AlertBrokenExecutables = "yes" MaxScanTime = "120000" MaxRecursion = "16" MaxZipTypeRcg = "3145728"

Config file: freshclam.conf

clamav-milter.conf not found

Software settings

Version: 0.103.6 Optional features supported: MEMPOOL IPv6 FRESHCLAM_DNS_FIX AUTOIT_EA06 BZIP2 LIBXML2 PCRE2 ICONV JSON

Database information

Database directory: /var/lib/clamav [3rd Party] winnow_phish_complete_url.ndb: 54 sigs [3rd Party] bofhland_phishing_URL.ndb: 72 sigs [3rd Party] securiteinfo.ign2: 138 sigs [3rd Party] phishtank.ndb: 4588 sigs [3rd Party] phish.ndb: 28608 sigs [3rd Party] foxhole_filename.cdb: 3182 sigs [3rd Party] junk.ndb: 56107 sigs [3rd Party] securiteinfohtml.hdb: 70637 sigs [3rd Party] porcupine.ndb: 2722 sigs [3rd Party] badmacro.ndb: 642 sigs [3rd Party] lott.ndb: 2338 sigs [3rd Party] spearl.ndb: 1 sig [3rd Party] scam.ndb: 12825 sigs [3rd Party] spamimg.hdb: 216 sigs [3rd Party] foxhole_js.ndb: 4 sigs [3rd Party] winnow_spam_complete.ndb: 26 sigs [3rd Party] winnow.attachments.hdb: 1 sig [3rd Party] shelter.ldb: 49 sigs [3rd Party] winnow_malware_links.ndb: 133 sigs [3rd Party] bofhland_cracked_URL.ndb: 40 sigs [3rd Party] securiteinfoandroid.hdb: 84401 sigs [3rd Party] blurl.ndb: 4954 sigs [3rd Party] spamattach.hdb: 14 sigs [3rd Party] jurlbla.ndb: 1406 sigs bytecode.cvd: version 333, sigs: 92, built on Mon Mar 8 23:21:51 2021 [3rd Party] bofhland_malware_URL.ndb: 4 sigs [3rd Party] securiteinfopdf.hdb: 3408 sigs [3rd Party] foxhole_generic.cdb: 213 sigs [3rd Party] winnow_bad_cw.hdb: 1 sig [3rd Party] jurlbl.ndb: 6143 sigs [3rd Party] hackingteam.hsb: 435 sigs [3rd Party] winnow_malware.hdb: 1 sig [3rd Party] winnow_extended_malware.hdb: 1 sig [3rd Party] rogue.hdb: 3161 sigs [3rd Party] malwarehash.hsb: 771 sigs [3rd Party] bofhland_malware_attach.hdb: 1836 sigs [3rd Party] securiteinfoascii.hdb: 108001 sigs [3rd Party] javascript.ndb: 43708 sigs [3rd Party] spear.ndb: 1 sig [3rd Party] local.ign2: 0 sig [3rd Party] winnow_extended_malware_links.ndb: 1 sig [3rd Party] urlhaus.ndb: 4790 sigs [3rd Party] securiteinfo.hdb: 122131 sigs main.cvd: version 62, sigs: 6647427, built on Thu Sep 16 20:32:42 2021 daily.cld: version 26636, sigs: 1998238, built on Tue Aug 23 15:52:45 2022 [3rd Party] foxhole_js.cdb: 48 sigs Total number of signatures: 9213569

Platform information

uname: Linux 5.4.0-122-generic #138-Ubuntu SMP Wed Jun 22 15:00:31 UTC 2022 x86_64 OS: linux-gnu, ARCH: x86_64, CPU: x86_64 Full OS version: Ubuntu 20.04.4 LTS zlib version: 1.2.11 (1.2.11), compile flags: a9 platform id: 0x0a217f7f0800000000090400

Build information

GNU C: 9.4.0 (9.4.0) CPPFLAGS: -Wdate-time -D_FORTIFY_SOURCE=2 CFLAGS: -g -O2 -fdebug-prefix-map=/build/clamav-1RbSOK/clamav-0.103.6+dfsg=. -fstack-protector-strong -Wformat -Werror=format-security -Wall -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 CXXFLAGS: -g -O2 -fdebug-prefix-map=/build/clamav-1RbSOK/clamav-0.103.6+dfsg=. -fstack-protector-strong -Wformat -Werror=format-security -Wall -D_FILE_OFFSET_BITS=64 LDFLAGS: -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -Wl,--as-needed Configure: '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--runstatedir=/run' '--disable-maintainer-mode' '--disable-dependency-tracking' 'CFLAGS=-g -O2 -fdebug-prefix-map=/build/clamav-1RbSOK/clamav-0.103.6+dfsg=. -fstack-protector-strong -Wformat -Werror=format-security -Wall -D_FILE_OFFSET_BITS=64' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' 'CXXFLAGS=-g -O2 -fdebug-prefix-map=/build/clamav-1RbSOK/clamav-0.103.6+dfsg=. -fstack-protector-strong -Wformat -Werror=format-security -Wall -D_FILE_OFFSET_BITS=64' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -Wl,--as-needed' '--with-dbdir=/var/lib/clamav' '--sysconfdir=/etc/clamav' '--disable-clamav' '--disable-unrar' '--enable-milter' '--enable-dns-fix' '--with-libjson' '--with-system-libmspack' '--with-libcurl=/usr' '--with-gnu-ld' '--with-systemdsystemunitdir=/lib/systemd/system' 'build_alias=x86_64-linux-gnu' 'OBJCFLAGS=-g -O2 -fdebug-prefix-map=/build/clamav-1RbSOK/clamav-0.103.6+dfsg=. -fstack-protector-strong -Wformat -Werror=format-security' sizeof(void*) = 8 Engine flevel: 127, dconf: 127

Thanks

ragusaa commented 2 years ago

I am able to reproduce the issue. It looks like maybe a bug where freshclam encounters an error and stops downloading files. When you are not on cooldown, would you mind trying with "https://www.securiteinfo.com/get/signatures/13f93f439ee9610229122f864bad794e9fb755d8c11eeb3ea004bed5fa219eceb260894eb90797a72adb153d9ba12226edbe5d8a7fb28f00a25d2af64398aaca/securiteinfo.mdb" removed?

Thanks, Andy

armanmisdar commented 2 years ago

Thanks Andy, I will give it a try soon and get back to you!

armanmisdar commented 2 years ago

Hi Andy, this is what I got:

Forbidden You don't have permission to access this resource.

ragusaa commented 2 years ago

Was that for one of the clamav databases, or one of the other ones?

Are you able to download the clamav databases now that you are no longer in the cooldown period?

armanmisdar commented 1 year ago

Hi Andy, sorry I misunderstand your previous instructions. I've removed the line you mentioned from my /etc/clamav/freshclam.conf and it seems to be working now:

daily database available for update (local version: 26642, remote version: 26644) Testing database: '/var/lib/clamav/tmp.dc4bb4d085/clamav-d7da04e45dbceb516543d21f2bc9c78d.tmp-daily.cld' ... Database test passed. daily.cld updated (version: 26644, sigs: 1999707, f-level: 90, builder: raynman) main.cvd database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr)

Will there be any harm for removing "https://www.securiteinfo.com/get/signatures/13f93f439ee9610229122f864bad794e9fb755d8c11eeb3ea004bed5fa219eceb260894eb90797a72adb153d9ba12226edbe5d8a7fb28f00a25d2af64398aaca/securiteinfo.mdb" from the conf file?

Thanks for your help!

ragusaa commented 1 year ago

That database doesn't seem to be available, so I don't see any harm in removing it. It may be worth contacting securiteinfo to ask if it has moved. This appears to be a bug in freshclam, so thanks for reporting!

Thanks, Andy

armanmisdar commented 1 year ago

You're most welcome Andy, without your help I can't figure it out. Thanks again!

Cisco-Talos / clamav