search

google / minions

Distributed filesystem scanner
Apache License 2.0
128 stars 12 forks source link

Build a Virus-Total minion #3

Open paradoxengine opened 6 years ago

paradoxengine commented 6 years ago

Sending binaries to virus total seems like a reasonable thing to do.

The public API is documented here: https://www.virustotal.com/es/documentation/public-api/ The first scan is bound to take a long while as the minion builds a cache of "harmless files" with all the binaries found in a healthy OS, but after that it should be reasonably fast. I wonder if there is a way to optimize this entirely by having some form of pre-check on the hashes of well known good binaries - it seems a trivial enough idea that I'm sure it must exist already - otherwise, maybe it would be an interesting project to just download all the rpms/apts and generate hashes of those binaries?

Of course, this plays very nicely with the idea of controlling the scope of goblins re the binaries they serve.

paradoxengine commented 6 years ago

Seems like NIST has what I need in terms of list of files - this looks like a great pre-filter before feeding anything to virus total. https://www.nist.gov/itl/ssd/software-quality-group/nsrl-download/current-rds-hash-sets

paradoxengine commented 6 years ago

Worth mentioning also this malice plugin which has the great idea of using a Bloom filter: https://github.com/malice-plugins/nsrl