doomedraven / VirusTotalApi

VirusTotal Full api
MIT License
290 stars 85 forks source link

Rescan URL's when "The requested resource is not among the finished, queued or pending scans" #2

Closed darrenmcentee closed 8 years ago

darrenmcentee commented 8 years ago

Hi, I have a URL list i want to scan with VT API. My current command is:

vt.py -r --rescan -ur urls_for_scan.txt > GP-000-Results.txt

This works well, and returns back the report from the URL scan. But within the report, about 1/3 (a third) of all the URLs scanned, come back with the response:

[!] Status : The requested resource is not among the finished, queued or pending scans

See attached. 2016-02-19_10-31-48

Many are not scanned for me, so we are missing out on many URL scan reports.

Example out put in my:

Searching for url(s) report: http://www.xyz.com
[!] Status : The requested resource is not among the finished, queued or pending scans

Am i implementing this rescan incorrectly? Thanks, Darren.

doomedraven commented 8 years ago

Hello @darrenmcentee, for start you can't rescan what is not scanned, so you need check for that verbose msg, and if you got it, upload to scan, and add to pend queue or any other way prefered by you to check later for report for that url

Small explanation of commands, as i saw you use short and long command in past too

-r is short for --rescan -ur is short for --url-report

so don't use both options

you need to do one think, rescan or get report

vt.py -r urls_for_scan.txt > GP-000-Results.txt or vt.py -ur urls_for_scan.txt > GP-000-Results.txt

utilities:

My suggestion is to use vt script as library and script it, see wiki for automatization

doomedraven commented 8 years ago

reopen this if any extra help needed

hakangs commented 8 years ago

Hi, I am requesting ~50 file scan results, but for some I am getting "The requested resource is not among the finished, queued or pending scans", but when I view VT website for those SHA-1s I can see the results.

I am using this request :

https://www.virustotal.com/vtapi/v2/file/report

What can be the problem ?

doomedraven commented 8 years ago

from documentation Before performing your submissions we encourage you to retrieve the latest report on the files

so i would suggest first do -fs or -rai depend on your api type, and if not found then request for scan, as basically you asking for rescan sample

if you still have issue, can you share your list of samples to test?

hakangs commented 8 years ago

This is code and samples :

`import json import urllib import urllib2

def get_detection_rate(sha1):

initialize result dict

dr = {"detection_rate": "", "vt_link": ""}
url = "https://www.virustotal.com/vtapi/v2/file/report"
api_key_list = []
api_key_num = 0
# ask VT with every api key
while api_key_num < 4:
    try:
        parameters = {"resource": sha1, "apikey": api_key_list[api_key_num]}
        data = urllib.urlencode(parameters)
        req = urllib2.Request(url, data)
        response = urllib2.urlopen(req,timeout=3)
        json_string = response.read()
        parsed_json = json.loads(json_string)
        if str(parsed_json["response_code"]) != "0":
            # update detection rate
            dr["detection_rate"] = str(parsed_json["positives"]) + '/' + str(parsed_json["total"])
            # update vt link
            dr["vt_link"] = parsed_json['permalink']
            break
    except:
        api_key_num = api_key_num + 1
if not dr["detection_rate"] and not dr["vt_link"]:
    print "empty for " + sha1
else:
    print "found for " + sha1
return dr

sha1s = "04bd1321551036e1738dbd9064f99c63a115becf,054efef9644b63f4b3bb9c664ac92a0698aef590,06d9283eaff05a4e6d3a04d9e7294a7f07ddd351,093a87031d311e937e813422e94231260b30b7aa,10829b63a496fc5fab389c9423641d787ef0fa3f,11dc931857eac2e9ed4177fd78d1986b0668597a,1235104c5ea47bf7ca8bd60aa166d748af1ed581,1744d6522fc9abdb9ee4f2255b430285496dd749,1b9604955d398784b4386088ec4c3ce8ef0b3052,1edf77ba99c7c973d461f5b62cb6c34e1a5d7be1,27ea339dac2c8abda9810a3b45162e459af2c903,311a0fb06232f905adc01df3817ccf7dfd6db73a,31a0a77adc4aed82570e6ab5dc279f1d3e6030e3,34b8a69c15dc3aa28fccd4303fc5f59bffc10458,3e4ad8c30599fa6fbaf0e1ef6c6685e010eab878,422ea500275d8818061d32b693b4cc96ee75828b,4fb0b8dc5208e1ee390f3391eb06add6409feea5,57b8e9e712d515d829c8cef52690beef11905456,5c26315243d40db66d02e274841631de1565c13a,5d4c9a96cdc888833c7317d37bb74bb0ad991b0e,61cf877819b344761bb1c5b3622a80705bd92378,6b1d0a34e9a050815e2fcfc0111a3a711b059e20,6e5fa6caabd84ecde727611dffa8918d589c122a,7016e316e82e084c5f44590a39af8e70e772ad62,7100689ac720bea999d4e94fc80027fcf86bd3e4,7260bf8b656a9b7766b200c391e0a7d035a805d1,72c26d9f0b9e69202bfe0871113d7df7f0407c19,7931f6cea232d82e913d57cc057d9db1328ca496,7a0351452a9c2e299c4c63f8af236afde94f4d89,7a08a87dee32ba340201a5c2ec2d85a4c81aeee2,7eb3b901aeb1ec12889440f6d6690873798069ae,8033d2370169342809d43c687b77ee0e0e4b9773,80bd9904d9f39cfa8fc65179b57ab9bf58a02d3f,81e701789acf9f2a432fe2527560c59f50b156a8,843d77d0e69c0cd7af8af313264ca12c9d5eb2c8,88f0d2fccae4142349731e378ea4931d5e930fa3,89ec25b15b7e7dd2f01178eac502122da4db5963,905df7ca339f5751a310f5b0cfc513276d08aa5d,95c12b0bf57d6034245c4b2a6726255f486e7141,98deb8979be90576c0fe154a02cc862cf92c7c0d,9f5b716a84a869592c492deafc6eedf8d71fb66a,a2fead7092fd67c0e4f9a2bf644dd7860b10bdd8,b51a6d25db9f8c6dd96805938c5d4f1c22ffb8e7,bfb6c6c2fda6a6914772ed6be845eadd4a5fbb86,c659339103d99f33dbc3304cc9d41bfebbcf4337,c91e02f3065711488ee2e4af25d4e0ba5aff630a,c9ab1a84afcc2d8cb5e5b341d16b9ce7a28752b8,d550051c10ad6e3106fb74dc63c4ac5b36df5b5d,d63413668a2ca7b2ca1ae51fdc0339ac"

for sha1 in sha1s.split(","): get_detection_rate(sha1) `

doomedraven commented 8 years ago

this is not related to my script, so i can't help

hakangs commented 8 years ago

Sorry, thanks.