search

Cisco-Talos / clamav

ClamAV - Documentation is here: https://docs.clamav.net
https://www.clamav.net/
GNU General Public License v2.0
4.43k stars 706 forks source link

ALLMATCHES scan is aborted by checkURLs #320

Closed eriklax closed 2 years ago

eriklax commented 3 years ago

If scanning an .eml with two MIME parts, one contains a Heuristics.Phishing.Email.SpoofedDomain and the second a Win.Test.EICAR_HDB-1 (EICAR), the engine will abort on the first MIME part, and won't continue to the second MIME part (where the EICAR is) only reporting the Heuristics.Phishing.Email.SpoofedDomain match.

https://github.com/Cisco-Talos/clamav/blob/10260c77b3162b5dafbf4221e22437f52b053b54/libclamav/mbox.c#L4349-L4356

I believe it has something to do with the rc argument being set to virus, and the ALLMATCHES flag isn't taken into consideration.

eriklax commented 3 years ago

The issues with the ALLMATCHES flag being sticky #319, it will break regular scannings as well (missing the virus).

# printf "zMULTISCAN /test.eml\0" | nc -U /tmp/clamd.socket 
/test.eml: Win.Test.EICAR_HDB-1 FOUND
# printf "zALLMATCHSCAN /test.eml\0" | nc -U /tmp/clamd.socket
/test.eml: Heuristics.Phishing.Email.SpoofedDomain FOUND
/test.eml: Heuristics.Phishing.Email.SpoofedDomain FOUND
# printf "zMULTISCAN /test.eml\0" | nc -U /tmp/clamd.socket 
/test.eml: Heuristics.Phishing.Email.SpoofedDomain FOUND
/test.eml: Heuristics.Phishing.Email.SpoofedDomain FOUND
mjbroekman commented 3 years ago

I'm seeing a similar behavior when scanning PE binaries that should be matching multiple PE Section Hash signatures. Only the first PE section that matches is getting reported.

Example:

sigtool --mdb 3fd4aa339bdfee23684ff495d884aa842165e61af85fd09411abfd64b9780146.exe 
LibClamAV debug: cli_peheader: SizeOfHeader is not aligned to the SectionAlignment
LibClamAV debug: Section{0}: 89088:dfb44642756ad7031bf4c862af14ffc8
LibClamAV debug: Section{1}: 512:59ea7ed8c8616ea4f7a407d5f5d9e766
LibClamAV debug: Section{2}: 3072:3610f22b6bcf30a7498462ef738b0190
LibClamAV debug: hashtab: Freeing hashset, elements: 0, capacity: 0
LibClamAV debug: Cleaning up phishcheck
LibClamAV debug: Phishcheck cleaned up
cat ~/clamdb/mytest.mdb
89088:dfb44642756ad7031bf4c862af14ffc8:Test.PESection.0
512:59ea7ed8c8616ea4f7a407d5f5d9e766:Test.PESection.1
3072:3610f22b6bcf30a7498462ef738b0190:Test.PESection.2
clamscan -z 3fd4aa339bdfee23684ff495d884aa842165e61af85fd09411abfd64b9780146.exe
/Users/mbroekman/Security/abuse.ch/3fd4aa339bdfee23684ff495d884aa842165e61af85fd09411abfd64b9780146.exe: Win.Dropper.KpotStealer-9322564-1 FOUND
/Users/mbroekman/Security/abuse.ch/3fd4aa339bdfee23684ff495d884aa842165e61af85fd09411abfd64b9780146.exe: Test.PESection.0.UNOFFICIAL FOUND

----------- SCAN SUMMARY -----------
Known viruses: 8574049
Engine version: 0.103.3
Scanned directories: 0
Scanned files: 1
Infected files: 1
Data scanned: 0.09 MB
Data read: 0.09 MB (ratio 1.00:1)
Time: 9.356 sec (0 m 9 s)
Start Date: 2021:11:11 18:27:25
End Date:   2021:11:11 18:27:34
micahsnyder commented 3 years ago

I'm seeing a similar behavior when scanning PE binaries that should be matching multiple PE Section Hash signatures. Only the first PE section that matches is getting reported.

The PE section hash one is resolved in this PR https://github.com/Cisco-Talos/clamav/pull/326

Regarding the original issue, I don't plan to fix this as-is. The allmatch system as a whole is really janky and poorly implemented. I have a plan to re-do it so it's less error prone and am itching to work on it but have other tasks I must work on first. We can keep this ticket open though until that's done, so we can at least verify that the "big fix" actually works for the reported issues once we have a "big fix".

micahsnyder commented 2 years ago

This issue should now be fixed in the main branch and in the 1.0.0 release candidate.

As I don't have a copy, I am going to trust that it is resolved by the all-match mode overhaul and close the issue. @eriklax if you still have the sample that caused the originally described problem and you find it is not fixed, do not hesitate to comment and reopen on this issue.