search

diogo-fernan / malsub

A Python RESTful API framework for online malware analysis and threat intelligence services.
Other
368 stars 80 forks source link

does not work for me for HA and VT, is API key in URL still supported ? #19

Closed beamzer closed 3 years ago

beamzer commented 4 years ago

I know the API keys work, because i use them with HA VxAPI en for VT with Curl (curl ... -F apikey=$VTAPI ...) but malsub gives me errors. Could this be because the API keys are used in the URL and this method is not supported anymore ?

% python3 malsub.py -a ha -q -v

            ####             ##    
              ##             ##    
####   ###    ##  #### ## ## ####  
# # #    ##   ## ##    ## ## ## ## 
# # #  ####   ## ####  ## ## ## ## 
# # # ## ##   ##  #### ## ## ## ## 
# # # ## ##   ##    ## ## ## ## ## 
# # #  ## #   ## ####   ## # ####

   malsub v1.3
   https://github.com/diogo-fernan/malsub

[!] warning Wed 16 Sep 2020 16:24:56.559932 +0000 UTC: "HybridAnalysis" -- "quota" error: 403 Client Error: Forbidden for url: https://www.hybrid-analysis.com/api/quota?apikey=<<MyAPIkey>>&secret=%3Capiuser%3E
Traceback (most recent call last):
  File "/home/remnux/GIT/malsub/malsub/core/work.py", line 27, in exec
    data = f.result()
  File "/usr/lib/python3.6/concurrent/futures/_base.py", line 425, in result
    return self.__get_result()
  File "/usr/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/remnux/GIT/malsub/malsub/service/hybrid-analysis.py", line 79, in quota
    data, _ = request(self.api_quot)
  File "/home/remnux/GIT/malsub/malsub/core/web.py", line 97, in request
    apispec.verify, bin, json)
  File "/home/remnux/GIT/malsub/malsub/core/web.py", line 86, in get
    json_req, param, verify, bin, json)
  File "/home/remnux/GIT/malsub/malsub/core/web.py", line 49, in _request
    res.raise_for_status()
  File "/home/remnux/.local/lib/python3.6/site-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://www.hybrid-analysis.com/api/quota?apikey=<<MyAPIkey>>&secret=%3Capiuser%3E

[*] verbose Wed 16 Sep 2020 16:24:56.562787 +0000 UTC: malsub finished with results:
+---+-------+-----------------+
| # | input | Hybrid Analysis |
+---+-------+-----------------+
| 1 | —     | unsuccessful    |
+---+-------+-----------------+
AHewitt commented 4 years ago

Hi beamzer and thanks for bringing this to our attention. I think your assumption is correct and we need to make some changes to fix it.

diogo-fernan commented 4 years ago

Hey beamzer,

You are missing the API user as per &secret=%3Capiuser%3E from the URI. Hybrid Analysis calls it a secret (check https://github.com/diogo-fernan/malsub/blob/master/data/apikey.yaml#L13).

Cheers, Diogo

beamzer commented 4 years ago

Hey beamzer,

You are missing the API user as per &secret=%3Capiuser%3E from the URI. Hybrid Analysis calls it a secret (check https://github.com/diogo-fernan/malsub/blob/master/data/apikey.yaml#L13).

Cheers, Diogo

Hi Diogo,

This is wat i thought in the beginning too. but as far as i can see/find, Hybrid-Analysis and VT don't use a "secret". I know some others do, but for HA and VT there is no "secret" on my API profile page and also the other API methods i use work fine with only the API key string.

thanks, Ewald....

diogo-fernan commented 4 years ago

It could be that the API specification changed or that the /api/quota endpoint is no longer available at Hybrid Analysis. Also try removing the apiuser parameter for HA from the YAML configuration file since you do not have one.

beamzer commented 4 years ago

Hi Diogo, that does not seem to help:

/home/remnux/GIT/malsub% python3 malsub.py -a ha -q -v

         _,  |\  ,       |)
/|/|/|  / |  |/ / \_|  | |/\_
 | | |_/\/|_/|_/ \/  \/|_/\/

   malsub v1.3
   https://github.com/diogo-fernan/malsub

[!] warning Mon 21 Sep 2020 20:23:36.998482 +0000 UTC: "HybridAnalysis" -- "quota" error: 403 Client Error: Forbidden for url: https://www.hybrid-analysis.com/api/quota?apikey=<<MyAPIkey>>
Traceback (most recent call last):
  File "/home/remnux/GIT/malsub/malsub/core/work.py", line 27, in exec
    data = f.result()
  File "/usr/lib/python3.6/concurrent/futures/_base.py", line 425, in result
    return self.__get_result()
  File "/usr/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/remnux/GIT/malsub/malsub/service/hybrid-analysis.py", line 79, in quota
    data, _ = request(self.api_quot)
  File "/home/remnux/GIT/malsub/malsub/core/web.py", line 97, in request
    apispec.verify, bin, json)
  File "/home/remnux/GIT/malsub/malsub/core/web.py", line 86, in get
    json_req, param, verify, bin, json)
  File "/home/remnux/GIT/malsub/malsub/core/web.py", line 49, in _request
    res.raise_for_status()
  File "/usr/local/lib/python3.6/dist-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://www.hybrid-analysis.com/api/quota?apikey=<<MyAPIkey>>

[*] verbose Mon 21 Sep 2020 20:23:36.999654 +0000 UTC: malsub finished with results:
+---+-------+-----------------+
| # | input | Hybrid Analysis |
+---+-------+-----------------+
| 1 | —     | unsuccessful    |
+---+-------+-----------------+

thanks, Ewald...

diogo-fernan commented 4 years ago

It may be that the /api/quota API endpoint is no longer available or that your API key lacks permissions. Have you tried other functions?

beamzer commented 4 years ago

Hi Diogo,

Submitting to HA doesn't show an error, but it also doesn't show that submission in the HA webinterface.

% python3 malsub.py -a ha -s september-16-919948-2020.doc 

 ##  ## ###### ###    ###### ##  ## ######  
 ###### ###### ###    ###    ##  ## ##  ##  
 ###### ##  ## ###    ###### ##  ## #####   
 ##  ## ###### ###        ## ##  ## ##  ##  
 ##  ## ##  ## ###    ###### ###### ######  
 ##  ## ##  ## ###### ###### ###### ######

   malsub v1.3
   https://github.com/diogo-fernan/malsub

When i try to receive malware data based on the file hash, i also an error. When i try the same with the HA vxapi.py (and the same API key), it does give me the report.

% python3 malsub.py -a ha -r 44d6c7f1c3536454a19ef188240a6f58c415fb2da863308610e7e27a66e562cb

 _  _ ____ _    ____ _  _ ___ 
 |\/| |--| |___ ==== |__| |==]

   malsub v1.3
   https://github.com/diogo-fernan/malsub

[!] warning Wed 23 Sep 2020 13:22:49.114182 +0000 UTC: "HybridAnalysis" -- "report_file" error: 403 Client Error: Forbidden for url: https://www.hybrid-analysis.com/api/scan/44d6c7f1c3536454a19ef188240a6f58c415fb2da863308610e7e27a66e562cb?apikey=<<MyAPIkey>>
Traceback (most recent call last):
  File "/home/remnux/GIT/malsub/malsub/core/work.py", line 27, in exec
    data = f.result()
  File "/usr/lib/python3.6/concurrent/futures/_base.py", line 425, in result
    return self.__get_result()
  File "/usr/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/remnux/GIT/malsub/malsub/service/hybrid-analysis.py", line 41, in report_file
    data, _ = request(self.api_repf)
  File "/home/remnux/GIT/malsub/malsub/core/web.py", line 97, in request
    apispec.verify, bin, json)
  File "/home/remnux/GIT/malsub/malsub/core/web.py", line 86, in get
    json_req, param, verify, bin, json)
  File "/home/remnux/GIT/malsub/malsub/core/web.py", line 49, in _request
    res.raise_for_status()
  File "/usr/local/lib/python3.6/dist-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://www.hybrid-analysis.com/api/scan/44d6c7f1c3536454a19ef188240a6f58c415fb2da863308610e7e27a66e562cb?apikey=<<MyAPIkey>>

And via vxapi.py

% ../VxAPI/vxapi.py search_hash 44d6c7f1c3536454a19ef188240a6f58c415fb2da863308610e7e27a66e562cb
[
    {
        "analysis_start_time": "2020-09-19T09:29:16+00:00",
        "av_detect": 55,
etc...
kkochanski commented 4 years ago

Hello @beamzer, Hybrid Analysis dev here.

Hybrid Analysis provides two API versions. They mostly differ in the way of doing authentication, parameters and the form of responses.

Here, what I see, malsub is using the first version (that is deprecated and going to be removed in the future - I would recommend using API v2 instead). In order to be able to use that API version, you have to pass API Key and Secret. (all of it, you're getting at the moment of creating/regenerating API Key)

In contrary to malsub, the latest version of vxapi is using API v2. That API version requires providing only API Key in the request header.

To resolve your problem, I would recommend you regenerating your API Key at Hybrid Analysis. Then you will receive the API Key and the Secret that you will be able to use in both, malsub (passing API Key and Secret) and vxapi (passing just API Key). You will see it in the success message.

image