search

diogo-fernan / malsub

A Python RESTful API framework for online malware analysis and threat intelligence services.
Other
368 stars 80 forks source link

The "-all" arg attempts to run on services without an API key, causing errors #12

Closed AHewitt closed 5 years ago

AHewitt commented 5 years ago

For example if you run:

python3 malsub.py -a all -ri 8.8.8.8

Malsub will try to retrieve the IP Address report using all possible services, even those without an API Key present. This causes some errors in the output of the command. To fix this we should check which services do not have an API key present and exclude them.

diogo-fernan commented 5 years ago

This should not be the case regardless of the operation due to the check of an API key here: https://github.com/diogo-fernan/malsub/blob/master/malsub/core/main.py#L125-L126

Perhaps there is another condition (e.g., whitespace characters) missed out in the if?

AHewitt commented 5 years ago

Maybe it should not, but there are issues. You can reproduce this by running malsub with a default apikey.yaml file and executing: "python3 malsub.py -a all -ri 1.2.3.4"

Sample output includes 4 tracebacks: https://gist.github.com/AHewitt/85aa76c1810321b1bbd6f014789c9ab3

Tracebacks are from: Maltracker, Metadefender, Threatstream, and VirusTotal with errors like "403 Client Error: FORBIDDEN for url: http://api.maltracker.net:4700/c2/ip/12.2.3.40/?apikey=%3Capikey%3E" and "401 Client Error: Unauthorized for url: https://api.metadefender.com/v3/ip/12.2.3.40"

I would like to take a closer look at the code to see how to fix this.

AHewitt commented 5 years ago

So I fixed a simple logic bug (commit here) where operation would continue even with a missing API Key. Malsub now correctly does an exit(1) when an API Key is missing. However when running with the -all arg it will exit upon the first missing API Key. I think it would be best for the user if Malsub simply skipped running that service and output something like Missing key for <service>, skipping. So I'll leave this open and look into that.

AHewitt commented 5 years ago

With a closer look at this issue and a review of the readme, it is by design that the script user subtracts the services without valid keys. For example:

Running this with some keys missing will cause a "missing valid key" error: python3 malsub.py -a all -ri 8.8.8.8

However I can just subtract those services without keys like this: python3 malsub.py -a all,-mw,-js,-ts -ri 8.8.8.8

and it works fine. So I'm not sure this issue is worth fixing for this specific scenario.

diogo-fernan commented 5 years ago

I think it would be best for the user if Malsub simply skipped running that service and output something like Missing key for <service>, skipping. So I'll leave this open and look into that.

This makes sense!