Build Python based VTScript

[nigleweber]

Let's build a VirusTotal search tool that can be used to search for arbitrary things using the vtapi.

To Do:

• [ ] Plan program structure

  • [ ] Main
  • [ ] Core Helper
  • [ ] API Helper

• [x] Setup editor of choice (VSCode and Atom are popular)

• [x] Install packages for formatting

• [x] Add a python linter

• [x] Obtain Virustotal account + key <- this is free

• [x] SHA-256 to test with c4556a383a40a97900e110fdb0e24ac01868ea9accaed6e9b76bc9b0d0b55d85

• [x] Create api key loading function

  • [x] Create a getter function for loading the api key from a file and returning it
  • [x] File read should be handled by a generic helper function

• [x] Accept cmdline input for a SHA to search for

  • [x] Use the argparse library to make the argument parsing easier

• [x] Create function to make api request

  • [x] Virustotal has this tool to test your searches with https://developers.virustotal.com/reference/search-1
  • [x] The python code is already written for you to make the request
  • [x] Output is JSON
  • [x] Check the HTTP response status

• [x] Return the JSON output and pass it to a JSON parser function

• [x] Let's get a count on the number of detections and a listing of each detecting engine

Protips

1. If you use magic numbers as you go add a comment like #MAGIC NUMBER so you can easily find it
2. When printing debug statements add a comment like #TESTING CODE so you can easily (un)comment it in the future
3. No global variables if at all possible
4. Functions ideally less than 20 lines
5. Make use of your linter as much as possible. Linters are like automated teachers for code quality

[nigleweber]

Additions:

• [x] Let's add pulling file information if detection count is over 3
• [x] Add sleep timer if the number of API requests exceeds 4 per minute

URL for VT File API: https://developers.virustotal.com/reference/file-info

[nigleweber]

Suggestions for "UI":

• [x] API input file: prompt should mention what the file is
• [x] Write all data into a single file so you don't have to prompt multiple times

Improvements:

• [x] De-duplicate code in main if file_arg: and its corresponding else statement
• [x] Move parsing outside of write_file_reputation_report
• [x] Write all data into file, regardless of malicious or "benign"
• [x] Re-structure main
  • [x] Create a new argument parsing function that checks `len(sys.argv) and returns args, check should happen before instantiating the parser
  • [x] Add print statement if api_key is None
  • [x] Take all code from file_reputation = False onwards and split into new function

[nigleweber]

Priority 1 improvements so that I can review faster:

• [x] Put comments on the end on lines
• [x] Put print statements all on 1 line
• [x] Remove the empty lines within functions where possible. For this round there shouldn't be any lines by themselves

[nigleweber]

@v3r1t4s You missed a couple of the previous action items:

Missed:

• Remove the empty lines within functions where possible. For this round there shouldn't be any lines by themselves
  • Unless absolutely necessary, remove empty lines within functions
• Create a new argument parsing function
  • My item was vague. The function you created already is good, we need another for the actual argument parsing itself
• Put all print statements on a single line. There's still a lot of them that are on 2 lines

[nigleweber]

To do for next iteration:

• [x] Reduce calls to append_file

  • File writes are slow and expensive calls. We should do this AT MOST once per item, and realistically only once per run of the script

• [x] Split parse_json function into two. End parse_json at analysis_stat = res_dic and return analysis_stat

  • Name second function something like data_processing for now, we can improve it later

• [x] Please post the output (the txt/json file) of a successful run

  • Output is looking very unfocused based on the number of calls to append_file <- I could be mistaken on this

• [x] For logic_func let's de-duplicate bunch of code. Line 227-236 are/should be identical to 239-247. Let's split this off to a separate function

[nigleweber]

Looking better and better! 👍

Comments/Questions:

1. We need to update the returns from parse_json. Why is it returning res_dic twice?
2. Let's start compress the report_data_dic.append statements. Multi line appends are superfluous
   • If this is for formatting, it should not be done within a function like parse_json

To do for next iteration:

• [x] Make parse_json a generic/helper function. Pass in (potential json string) and a key to search for/parse out
• [x] Rename parse_filename to reference user input
• [x] Make parse_filename generic/helper
• [x] Let's make append_report a generic/helper
• [x] As a suggestion, rather than return true/false for errors, return an error message or "" to denote no error

Bonus:

• [x] Let's start using the Helpers.py file
• [x] Move read_api_key, check_if_filepath_exists and exception_handler to the Helper module (file)
• [x] Make verify_at_least_one_command_line_argument more generic, to check if we have X minimum arguments, where X is an argument to the function also rename the function as appropriate