office document detection

[vince68]

Great job for you application, I want to use it in production on week 15 or 16. I have a question is you application compatible with the premium api from virustotal ? Maybe I would buy to scan all files. Because we receive a lot of javascript that contains the lockit virus and now you application dont scan it because amavis decompress the rar or zip with pXXX name and not the original one. I just add this line, because for me some office document are detected like this : Composite Document File V2 Document. So I modify your code to handle it :

            return any((f(resource) for f in [
                                    lambda r: r.mime_type.startswith('application/'),
                                    lambda r: r.mime_type.startswith('Composite'),
                                    lambda r: r.mime_type in ('text/x-shellscript', 'text/x-perl', 'text/x-ruby', 'text/x-python'),
                                    lambda r: re.search(r"\.(exe|com|zip|tar\.[\w\d]+|doc\w?|xls\w?|ppt\w?|pdf|js|bat|cmd|rtf|ttf|html?)$", r.basen$
            ]))

ps : I dont be a python programmer at all.

[ercpe]

AmavisVT detects files based on their mime types and, under a few circumstances, the files extension. Pure javascript files are notoriously hard to guess by their content and as such often recognized as text/plain.

Adding r.mime_type.startswith('Composite') won't work because Composite Document File V2 Document isn't a valid mime type. I will think about a way to specify a filter for mime parts of a mail.

As for private/public API: I don't have access to the private API and as such not tested against it. AmavisVT may work with the private API if the only difference is the url to the api endpoint and the API key.

[vince68]

Composite works on my case. Maybe the office document that contains a virus was especially malformed to bypass security filters.

[ercpe]

That's strange. Could you please run this snippet in a python console?

import magic m = magic.Magic(flags=magic.MAGIC_MIME_TYPE) print(m.id_buffer(open('/path/to/attachment', 'r').read()))

Does it really print Composite Document File V2 Document?

[vince68]

Sorry for the delay, I was on holiday last week.

It's wierd, I see that my modification wont work. See my logs,

amavis log : Apr 12 10:35:17 m4frlx21 amavis[5904]: (05904-16) run_av: /usr/local/bin/amavisvt exit 0, p003: Not scanned by virustotal

and debug mode from amavisvt : DEBUG:amavisvt.client:Examine /var/lib/amavis/tmp/amavis-20160412T095016-05904/parts/p003 DEBUG:amavisvt.client:----> p003, c067d5de1ab4b6e25fbdf7ecb73799a8, 5c060ffe0430a22d1f9f23b74e2f62d0784d5048, 96db66b79b02d9ea00aeebb3410335166c1ae14ed1af11115bc2f55ddf88100f: Composite Document File V2 Document, corrupt: Can't read SAT

And now the line you tell me to test :

import magic m = magic.Magic(flags=magic.MAGIC_MIME_TYPE) print(m.id_buffer(open('/var/lib/amavis/tmp/amavis-20160412T095016-05904/parts/p003', 'r').read())) application/msword exit()

[ercpe]

So it's working as intended. libmagic detects the attachment as application/msword which gets send to virustotal. "Not scanned by virustotal" means, that this exact file wasn't uploaded to virustotal before. AmavisVT will not send any file content to Virustotal by itself.