search

alexandreborges / malwoverview

Malwoverview is a first response tool used for threat hunting and offers intel information from Virus Total, Hybrid Analysis, URLHaus, Polyswarm, Malshare, Alien Vault, Malpedia, Malware Bazaar, ThreatFox, Triage, InQuest and it is able to scan Android devices against VT.
https://github.com/alexandreborges/malwoverview
GNU General Public License v3.0
2.85k stars 439 forks source link

Multiprocessing or threading for bulk file scans #3

Closed deadbits closed 5 years ago

deadbits commented 5 years ago

Right now the app only performs one operation at a time and even with a private key for VirusTotal it is very time consuming to lookup say 100 files or even more so 5000 files.

It seems like the logic of what to do with a file/URL (which functions and checks to run) should be outside of main in their own Class(es) or Functions with the arguments on which functions to run passed along as kwargs or whatnot, and then let main handle dispatching files/URLs for multiprocessing or multithreading if the number of files is above some arbitrary or user defined limit.

Is this something you’d be interested in implementing? I can help with (or directly submit) the PR. Just think it’d be super useful to speed up private key lookups

alexandreborges commented 5 years ago

Adam,

Good evening. How are you?

Yes, you are correct: one operation per time. The reason is that, when I wrote this tool, its first idea was to use it with a public VT key and, according to the VT rules, it wouldn’t be allowed (and neither possible) parallels connections.

After few weeks, I realized that many people with private keys were use the tool. Thus, I’ve started writing a new option only for using private key, because the current one is enough to public keys. Additionally, I removed some code from the main function. Of course, this new option will be using multi-thread. :)

Honestly, I have about 14 new features to implement and I haven’t done it because my first semester was complicated (many conferences) and my talk in the next Def Con made things more complex yet. Some of these features are ready and I’ll release soon.

I appreciate your time in making this very valuable comments and, soon after DEF CON, I hope I can release it. No doubts, constructive comments such yours are very appropriate (and rare).

Have an excellent day, Adam.

Warm regards,

Alexandre.

PS: if it's possible, send me a message in private (I believe you have my LinkedIn). Let's keep in touch. :)

deadbits commented 5 years ago

First off, congrats on the DefCon talk! That’s really great!

Also, If you ever feel like putting any of your outstanding tasks/features that you want to implement into this tool as issues I could always try to work on them in my free time to help lighten the load a bit