search

CheckPointSW / Cuckoo-AWS

Extension to Cuckoo Sandbox open source projects, adds support to AWS cloud functionalities and enables running emulation on auto-scaling infrastructure
Other
136 stars 30 forks source link

Launch error #5

Closed littlejob closed 5 years ago

littlejob commented 5 years ago

My issue is: After configuring auto-scaling option, upon cuckoo launch errors logged. My Cuckoo version and operating system are: Cuckoo v.2.0.6 - Ubuntu 18 This can be reproduced by: Simply following the instructions on the readme page..

The log, error, files etc can be found:

`2019-03-29 13:40:43,973 [cuckoo.core.scheduler] INFO: Using "aws" as machine manager 2019-03-29 13:40:44,093 [cuckoo.machinery.aws] INFO: connecting to AWS:us-west-1 Traceback (most recent call last):

File "/home/ubuntu/apps/Cuckoo-AWS/venv/bin/cuckoo", line 11, in load_entry_point('Cuckoo', 'console_scripts', 'cuckoo')()

File "/home/ubuntu/apps/Cuckoo-AWS/venv/lib/python2.7/site-packages/click-6.6-py2.7.egg/click/core.py", line 716, in call return self.main(*args, **kwargs)

File "/home/ubuntu/apps/Cuckoo-AWS/venv/lib/python2.7/site-packages/click-6.6-py2.7.egg/click/core.py", line 696, in main rv = self.invoke(ctx)

File "/home/ubuntu/apps/Cuckoo-AWS/venv/lib/python2.7/site-packages/click-6.6-py2.7.egg/click/core.py", line 1037, in invoke return Command.invoke(self, ctx)

File "/home/ubuntu/apps/Cuckoo-AWS/venv/lib/python2.7/site-packages/click-6.6-py2.7.egg/click/core.py", line 889, in invoke return ctx.invoke(self.callback, **ctx.params)

File "/home/ubuntu/apps/Cuckoo-AWS/venv/lib/python2.7/site-packages/click-6.6-py2.7.egg/click/core.py", line 534, in invoke return callback(*args, **kwargs)

File "/home/ubuntu/apps/Cuckoo-AWS/venv/lib/python2.7/site-packages/click-6.6-py2.7.egg/click/decorators.py", line 17, in new_func return f(get_current_context(), *args, **kwargs)

File "/home/ubuntu/apps/Cuckoo-AWS/cuckoo/main.py", line 242, in main sys.stderr.write(exception_message())

File "/home/ubuntu/apps/Cuckoo-AWS/cuckoo/common/utils.py", line 262, in exception_message for package in pip.get_installed_distributions() AttributeError: 'module' object has no attribute 'get_installed_distributions' `

chkp-orank commented 5 years ago

Hi @littlejob

This is an issue I have already experienced, and it actually appears in the README.md under "Problems and solutions".

In the case that the installation fails or if the following exception appears: ” 'module' object has no attribute 'get_installed_distributions' ”, try downgrading pip:

pip install --force-reinstall pip==9.0.3

After you do so, try reinstalling cuckoo.

sfoxdev commented 5 years ago

I had the same error at beginning, so I downgraded as was recommend pip install --force-reinstall pip==9.0.3 after that I got TypeError 

2019-04-04 11:12:36,709 [cuckoo.core.scheduler] INFO: Using "aws" as machine manager
2019-04-04 11:12:36,814 [cuckoo.machinery.aws] INFO: connecting to AWS:eu-west-1
Oops! Cuckoo failed in an unhandled exception!
Sometimes bugs are already fixed in the development release, it is therefore recommended to retry with the latest development release available https://github.com/cuckoosandbox/cuckoo
If the error persists please open a new issue at https://github.com/cuckoosandbox/cuckoo/issues

=== Exception details ===
Cuckoo version: 2.0.6
OS version: posix
OS release: Ubuntu 18.04 bionic
Python version: 2.7.15rc1
Python implementation: CPython
Machine arch: x86_64
Modules: alembic:0.8.8 androguard:3.0.1 asn1crypto:0.24.0 attrs:19.1.0 beautifulsoup4:4.5.3 boto3:1.9.126 botocore:1.12.126 capstone:3.0.5rc2 cffi:1.12.2 chardet:2.3.0 click:6.6 colorama:0.3.7 cryptography:2.6.1 cuckoo:2.0.6.2 django-extensions:1.6.7 django:1.8.4 docutils:0.14 dpkt:1.8.7 ecdsa:0.13 egghatch:0.2.3 elasticsearch:5.3.0 enum34:1.1.6 flask-sqlalchemy:2.1 flask:0.12.2 functools32:3.2.3.post2 future:0.17.1 futures:3.2.0 httpreplay:0.2.5 idna:2.8 ipaddress:1.0.22 itsdangerous:1.1.0 jinja2:2.9.6 jmespath:0.9.4 jsbeautifier:1.6.2 jsonschema:3.0.1 mako:1.0.8 markupsafe:1.1.1 olefile:0.43 oletools:0.51 peepdf:0.4.2 pefile2:1.2.11 pillow:3.2.0 pip:9.0.3 pkg-resources:0.0.0 pycparser:2.19 pycrypto:2.6.1 pyelftools:0.24 pyguacamole:0.6 pymisp:2.4.54 pymongo:3.0.3 pyopenssl:19.0.0 pyrsistent:0.14.11 python-dateutil:2.4.2 python-editor:1.0.4 python-magic:0.4.12 pythonaes:1.0 requests:2.13.0 roach:0.1.2 s3transfer:0.2.0 scapy:2.3.2 setuptools:40.9.0 sflock:0.3.9 six:1.12.0 sqlalchemy:1.0.8 tlslite-ng:0.6.0 unicorn:1.0.1 urllib3:1.24.1 wakeonlan:0.2.2 werkzeug:0.15.1 wheel:0.33.1 yara-python:3.6.3

2019-04-04 11:12:37,836 [cuckoo] ERROR: TypeError: 'NoneType' object is not iterable
Traceback (most recent call last):
  File "/opt/cuckoo/cuckoo/main.py", line 234, in main
    cuckoo_main(maxcount)
  File "/opt/cuckoo/cuckoo/main.py", line 182, in cuckoo_main
    sched.start()
  File "/opt/cuckoo/cuckoo/core/scheduler.py", line 903, in start
    self.initialize()
  File "/opt/cuckoo/cuckoo/core/scheduler.py", line 838, in initialize
    machinery.initialize(machinery_name)
  File "/opt/cuckoo/cuckoo/common/abstracts.py", line 178, in initialize
    self._initialize_check()
  File "/opt/cuckoo/cuckoo/machinery/aws.py", line 53, in _initialize_check
    if self._is_autoscaled(instance):
  File "/opt/cuckoo/cuckoo/machinery/aws.py", line 311, in _is_autoscaled
    for item in instance.tags:
TypeError: 'NoneType' object is not iterable

How can I solve this issue?

sfoxdev commented 5 years ago

After some investigation found missed libs:

apt install pkg-config python-lzma liblzma-dev
pip install pyliblzma

Now cockoo runs OK:

2019-04-04 11:55:36,340 [cuckoo.core.scheduler] INFO: Using "aws" as machine manager
2019-04-04 11:55:36,449 [cuckoo.machinery.aws] INFO: connecting to AWS:eu-west-1
2019-04-04 11:55:37,131 [cuckoo.machinery.aws] INFO: All machines are busy, allocating new machine
2019-04-04 11:55:38,431 [cuckoo.core.scheduler] INFO: Loaded 1 machine/s
2019-04-04 11:55:38,443 [cuckoo.core.scheduler] INFO: Waiting for analysis tasks.
chkp-orank commented 5 years ago

thanks @sfoxdev

that's awesome

sfoxdev commented 5 years ago

That error appear again:

Oops! Cuckoo failed in an unhandled exception!
Sometimes bugs are already fixed in the development release, it is therefore recommended to retry with the latest development release available https://github.com/cuckoosandbox/cuckoo
If the error persists please open a new issue at https://github.com/cuckoosandbox/cuckoo/issues

=== Exception details ===
Cuckoo version: 2.0.6
OS version: posix
OS release: Ubuntu 18.04 bionic
Python version: 2.7.15rc1
Python implementation: CPython
Machine arch: x86_64
Modules: alembic:0.8.8 androguard:3.0.1 asn1crypto:0.24.0 attrs:19.1.0 beautifulsoup4:4.5.3 boto3:1.9.130 botocore:1.12.130 cairocffi:0.9.0 cairosvg:1.0.22 capstone:3.0.5rc2 cffi:1.12.2 chardet:2.3.0 click:6.6 col
orama:0.3.7 cryptography:2.6.1 cssselect:1.0.3 cuckoo:2.0.6.2 distorm3:3.4.1 distribute:0.7.3 django-extensions:1.6.7 django:1.8.4 docutils:0.14 dpkt:1.8.7 ecdsa:0.13 egghatch:0.2.3 elasticsearch:5.3.0 enum34:1.1.
6 et-xmlfile:1.0.1 flask-sqlalchemy:2.1 flask:0.12.2 functools32:3.2.3.post2 future:0.17.1 futures:3.2.0 html5lib:1.0.1 httpreplay:0.2.5 idna:2.8 ipaddress:1.0.22 itsdangerous:1.1.0 jdcal:1.4 jinja2:2.9.6 jmespath
:0.9.4 jsbeautifier:1.6.2 jsonschema:3.0.1 lxml:4.3.3 m2crypto:0.24.0 mako:1.0.8 markupsafe:1.1.1 olefile:0.43 oletools:0.51 openpyxl:2.6.2 peepdf:0.4.2 pefile2:1.2.11 pillow:3.2.0 pip:9.0.3 pkg-resources:0.0.0 ps
ycopg2:2.8 pycparser:2.19 pycrypto:2.6.1 pydeep:0.4 pyelftools:0.24 pyguacamole:0.6 pyliblzma:0.5.3 pymisp:2.4.54 pymongo:3.0.3 pyopenssl:19.0.0 pyphen:0.9.5 pyrsistent:0.14.11 python-dateutil:2.4.2 python-editor:
1.0.4 python-magic:0.4.12 pythonaes:1.0 requests:2.13.0 roach:0.1.2 s3transfer:0.2.0 scapy:2.3.2 setuptools:41.0.0 sflock:0.3.9 six:1.12.0 sqlalchemy:1.0.8 tinycss:0.4 tlslite-ng:0.6.0 ujson:1.35 unicorn:1.0.1 url
lib3:1.24.1 volatility:2.6.1 wakeonlan:0.2.2 weasyprint:0.36 webencodings:0.5.1 werkzeug:0.15.2 wheel:0.33.1 yara-python:3.6.3

2019-04-10 08:27:37,425 [cuckoo] ERROR: .[31m.[31mTypeError: 'NoneType' object is not iterable.[0m.[0m
Traceback (most recent call last):
  File "/opt/cuckoo/cuckoo/main.py", line 234, in main
    cuckoo_main(maxcount)
  File "/opt/cuckoo/cuckoo/main.py", line 182, in cuckoo_main
    sched.start()
  File "/opt/cuckoo/cuckoo/core/scheduler.py", line 903, in start
    self.initialize()
  File "/opt/cuckoo/cuckoo/core/scheduler.py", line 838, in initialize
    machinery.initialize(machinery_name)
  File "/opt/cuckoo/cuckoo/common/abstracts.py", line 178, in initialize
    self._initialize_check()
  File "/opt/cuckoo/cuckoo/machinery/aws.py", line 53, in _initialize_check
    if self._is_autoscaled(instance):
  File "/opt/cuckoo/cuckoo/machinery/aws.py", line 311, in _is_autoscaled
    for item in instance.tags:
TypeError: 'NoneType' object is not iterable
sfoxdev commented 5 years ago

Reinstalled from scratch. Looks it's very unstable version.

chkp-orank commented 5 years ago

@sfoxdev

thank you for letting us know. this bug probably occurred as a result of iterating over an empty list of tags.

we pushed a fix 7 days ago to aws.py.