krmaxwell / maltrieve

A tool to retrieve malware directly from the source for security researchers.
GNU General Public License v3.0
563 stars 183 forks source link

Viper intergration problem #176

Closed zam89 closed 8 years ago

zam89 commented 8 years ago

Well, I try to integrate maltrieve and viper. I use this command:

python maltrieve.py -v -d pull -l maltrieve.log

which -v is for dump to VIPER instance. But after that I encountered this error:

Traceback (most recent call last):
  File "maltrieve.py", line 515, in <module>
    main()
  File "maltrieve.py", line 504, in main
    if save_malware(each, cfg):
  File "maltrieve.py", line 328, in save_malware
    stored = upload_viper(response, md5, cfg) or stored
  File "maltrieve.py", line 290, in upload_viper
    response = requests.post(url, headers=headers, files=files, data=tags)
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 109, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 50, in request
    response = session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 451, in request
    prep = self.prepare_request(req)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 382, in prepare_request
    hooks=merge_hooks(request.hooks, self.hooks),
  File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 304, in prepare
    self.prepare_url(url, params)
  File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 362, in prepare_url
    to_native_string(url, 'utf8')))
requests.exceptions.MissingSchema: Invalid URL 'True/file/add': No schema supplied. Perhaps you meant http://True/file/add?

Something wrong with the url. Any guide? Thanks.

jrespeto commented 8 years ago

Hi,

Maltrieve is an awesome tool!

I'm really new to python programing but I got a friend of mine from work look at the error with me. We traced the issues to the "class config(object):"

from the error log requests.exceptions.MissingSchema: Invalid URL u'True/file/add': No schema supplied.

line 289 => url = "{srv}/file/add".format(srv=cfg.viper)

.format(srv=cfg.viper) is returning True


119 # TODO: Merge these 120 self.vxcage = args.vxcage or self.configp.has_option('Maltrieve', 'vxcage') 121 self.cuckoo = args.cuckoo or self.configp.has_option('Maltrieve', 'cuckoo') 122 self.viper = args.viper or self.configp.has_option('Maltrieve', 'viper')


Should be this like in the # CRITs section.

    # TODO: Merge these
    if args.vxcage or self.configp.has_option('Maltrieve', 'vxcage'):
        self.vxcage = self.configp.get('Maltrieve', 'vxcage')
    if args.cuckoo or self.configp.has_option('Maltrieve', 'cuckoo'):
        self.cuckoo = self.configp.get('Maltrieve', 'cuckoo')
    if args.viper or self.configp.has_option('Maltrieve', 'viper'):
        self.viper = self.configp.get('Maltrieve', 'viper')

Will let you know if I run into other things.

Thank you,

Jonathan

zam89 commented 8 years ago

Hi,

I've try your solution and it works perfectly. Output from maltrieve:

$ python maltrieve.py -v -d pull -l maltrieve.log
Processing source URLs
Completed source processing
Downloading samples, check log for details
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/gevent/threadpool.py", line 203, in _worker
    value = func(*args, **kwargs)
gaierror: [Errno -2] Name or service not known
(<ThreadPool at 0x2a8a790 10/10/10>, <built-in function getaddrinfo>) failed with gaierror

Completed downloads

Output from Viper API:

python api.py
Bottle v0.12.8 server starting up (using WSGIRefServer())...
Listening on http://localhost:8080/
Hit Ctrl-C to quit.

127.0.0.1 - - [25/Jul/2015 15:37:45] "POST /file/add HTTP/1.1" 200 26
[!] File exists already
127.0.0.1 - - [25/Jul/2015 15:37:45] "POST /file/add HTTP/1.1" 500 41
[!] File exists already
127.0.0.1 - - [25/Jul/2015 15:37:46] "POST /file/add HTTP/1.1" 500 41

Hope this will help for those that have similar problem.

+1 nice one @jrespeto