search

gcrahay / otx_misp

Imports Alienvault OTX pulses to a MISP instance
Other
52 stars 42 forks source link

Statement: from otx_misp.cli import main gives an error #4

Closed icepaule closed 7 years ago

icepaule commented 8 years ago

Hello,

after having installed the otx-misp extension by "pip install otx-misp" I get below error. Could someone please advive me what I'm doing wrong?

Thanks a lot for your help.

Marcus

me@misp:~# otx-misp --help Traceback (most recent call last): File "/usr/local/bin/otx-misp", line 7, in from otx_misp.cli import main File "/usr/local/lib/python2.7/dist-packages/otx_misp/init.py", line 16, in requests.packages.urllib3.disable_warnings() AttributeError: 'module' object has no attribute 'packages'

gcrahay commented 8 years ago

Hello,

Thank you for this report.

What is your Python version? Could you paste the output of this command: pip freeze?

Thanks,

Gaetan

icepaule commented 8 years ago

Hi Gaetan,

wow, what a fast action... ;-)

Please find the output below.

Thanks a lot for your help

Marcus

@misp:/opt/otx-misp-1.0.0# pip freeze adns-python==1.2.1 apt-xapian-index==0.45 asnhistory==2.0.4 backports-abc==0.4 backports.shutil-get-terminal-size==1.0.0 backports.ssl-match-hostname==3.5.0.1 certifi==2016.8.8 chardet==2.0.1 click==6.6 colorama==0.2.5 configobj==4.7.2 crcmod==1.7 cybox==2.1.0.13 cycler==0.10.0 cymru-services==1.4 Cython==0.20.1.post0 DateTime==4.1.1 dateutils==0.6.6 decorator==4.0.10 dnspython==1.14.0 dnspython3==1.12.0 DomainClassifier==0.5 ez-setup==0.9 Flask==0.11.1 future==0.15.2 html5lib==0.999 ipaddress==1.0.16 ipasn-redis==2.0 IPy==0.83 ipython==5.1.0 ipython-genutils==0.1.0 itsdangerous==0.24 Jinja2==2.8 Landscape-Client==14.12 langid==1.1.6 Logbook==1.0.0 lxml==3.6.4 MarkupSafe==0.23 matplotlib==1.5.2 mixbox==1.0.0 mmh3==2.3.1 networkx==1.11 nltk==3.2.1 numpy==1.8.2 ordered-set==2.0.1 otx-misp==1.0.0 PAM==0.4.2 passivetotal==1.0.30 Paste==2.0.3 pathlib2==2.1.0 pexpect==4.2.0 pickleshare==0.7.4 Pillow==2.3.0 prompt-toolkit==1.0.6 ptyprocess==0.5.1 pubsublogger==1.2.2 pybloomfiltermmap==0.3.15 pycountry==1.20 pycrypto==2.6.1 pyeupi==1.0 Pygments==2.1.3 pymisp==2.2 pyOpenSSL==0.13 pyparsing==2.1.8 pypdns==1.3 pypssl==2.1 pyserial==2.6 PySocks==1.5.7 pytesseract==0.1.6 python-apt===0.9.3.5ubuntu1 python-dateutil==2.5.3 python-debian===0.1.21-nmu2ubuntu2 python-magic==0.4.12 pytz==2016.6.1 pyzmq==15.4.0 redis==2.10.5 requests==2.11.1 requests-cache==0.4.12 simplegeneric==0.8.1 singledispatch==3.4.0.3 six==1.10.0 ssh-import-id==3.21 stix==1.2.0.1 textblob==0.11.1 texttable==0.8.4 tornado==4.4.1 traitlets==4.2.2 Twisted-Core==13.2.0 url-normalize==1.3.1 urlarchiver==0.2 urllib3==1.7.1 uwhoisd==0.0.7 virtualenv==1.11.4 wcwidth==0.1.7 weakrefmethod==1.0.3 Werkzeug==0.11.10 Whoosh==2.7.4 zope.interface==4.0.5 /usr/local/lib/python2.7/dist-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#snimissingwarning. SNIMissingWarning /usr/local/lib/python2.7/dist-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning. InsecurePlatformWarning

On 01.09.2016 19:01, Gaetan wrote:

Hello,

Thank you for this report.

What is your Python version? Could you paste the output of this command: |pip freeze|?

Thanks,

Gaetan

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gcrahay/otx_misp/issues/4#issuecomment-244143917, or mute the thread https://github.com/notifications/unsubscribe-auth/AUKYu73jy218oU9l6nmTd3HbX-_moczYks5qlwTZgaJpZM4Jy5v1.

This mail was scanned by the Astaro Security Gateway of paulis.net.

Der Inhalt dieser E-Mail (einschliesslich etwaiger beigefuegter Dateien) ist vertraulich und nur fuer den Empfaenger bestimmt. Wenn Sie nicht der bestimmungsgemaesse Empfaenger sind oder diese E-Mail irrtuemlich erhalten haben, informieren Sie bitte sofort den Absender und loeschen Sie diese E-Mail (einschliesslich etwaiger beigefuegter Dateien) von Ihrem System. Jegliche Offenlegung, Vervielfaeltigung, Weitergabe oder Nutzung des Inhalts dieser E-Mail sind nicht gestattet. Vielen Dank.

The contents of this e-mail (including any attachments) are confidential and may be legally privileged. If you are not the intended recipient of this e-mail or have received this e-mail in error, please notify the sender immediately and then delete it (including any attachments) from your system. Any disclosure, copying, distribution or use of the material in this e-mail is strictly prohibited. Thank you.

Marcus Pauli Security Analyst Schwalbenstrasse 66 85521 Ottobrunn Tel. 0176/57695212

gcrahay commented 8 years ago

OK, it seems you're using an "old" Ubuntu version.

I uploaded a new version to Pypi (1.0.1).

Can you update your package (pip install --upgrade otx-misp)?

Hope this fix your issue.

Gaetan

icepaule commented 8 years ago

Good morning Gaetan,

thanks a lot for that quick action. The upgrade worked nicely and I can now call up the otx-misp to see the help pages and stuff.

BUT, trying to get in touch with OTX gives me below problem:

root@misp:~# otx-misp --otx xxxxxx --server http://10.10.0.4/ --misp xxxxxx -v -a --discover-tags -d

'unicode' does not have the buffer interface

root@misp:~#

Have you got any idea of what I could do to resolve this?

Cheers

Marcus

M.Pauli

Security Analyst

Schwalbenstrasse 66

85521 Ottobrunn

http://www.mpauli.de/ http://www.mpauli.de

mailto:info@mpauli.de info@mpauli.de

From: Gaetan [mailto:notifications@github.com] Sent: Donnerstag, 1. September 2016 22:30 To: gcrahay/otx_misp otx_misp@noreply.github.com Cc: icepaule info@mpauli.de; Author author@noreply.github.com Subject: Re: [gcrahay/otx_misp] Statement: from otx_misp.cli import main gives an error (#4)

OK, it seems you're using an "old" Ubuntu version.

I uploaded a new version to Pypi (1.0.1).

Can you update your package (pip install --upgrade otx-misp)?

Hope this fix your issue.

Gaetan

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gcrahay/otx_misp/issues/4#issuecomment-244202377 , or mute the thread https://github.com/notifications/unsubscribe-auth/AUKYu2POU9iGqqe-qtbjpsNgD2P1VUoiks5qlzXSgaJpZM4Jy5v1 . https://github.com/notifications/beacon/AUKYu-7L-AIyZfFUMxMhrUV9rL-eZaX0ks5qlzXSgaJpZM4Jy5v1.gif

< pre> -- This mail was scanned by the Astaro Security Gateway of paulis.net. Der Inhalt dieser E-Mail (einschliesslich etwaiger beigefuegter Dateien) ist vertraulich und nur fuer den Empfaenger bestimmt. Wenn Sie nicht der bestimmungsgemaesse Empfaenger sind oder diese E-Mail irrtuemlich erhalten haben, informieren Sie bitte sofort den Absender und loeschen Sie diese E-Mail (einschliesslich etwaiger beigefuegter Dateien) von Ihrem System. Jegliche Offenlegung, Vervielfaeltigung, Weitergabe oder Nutzung des Inhalts dieser E-Mail sind nicht gestattet. Vielen Dank. The contents of this e-mail (including any attachments) are confidential and may be legally privileged. If you are not the intended recipient of this e-mail or have received this e-mail in error, please notify the sender immediately and then delete it (including any attachments) from your system. Any disclosure, copying, distribution or use of the material in this e-mail is strictly prohibited. Thank you.

gcrahay commented 8 years ago

Hi Marcus,

I set up a Ubuntu 14.4 VM and found the bug. It should be fixed with the new version of the package on Pypi (1.0.2). I've launched several imports from this VM and had no error. It should work now!

Gaetan

icepaule commented 8 years ago

Hi Gaetan,

worked like a charm… :)

Excellent, thanks a lot for your kind help.

If I may bother you with one more question? Is there a “best practice” to call the job in cron but respect the last import date to avoid double events in MISP?

Again, thanks a lot for your kind and quick help.

Really appreciated.

Cheers

Marcus

M.Pauli

Security Analyst

Schwalbenstrasse 66

85521 Ottobrunn

http://www.mpauli.de/ http://www.mpauli.de

mailto:info@mpauli.de info@mpauli.de

From: Gaetan [mailto:notifications@github.com] Sent: Freitag, 2. September 2016 19:33 To: gcrahay/otx_misp otx_misp@noreply.github.com Cc: icepaule info@mpauli.de; Author author@noreply.github.com Subject: Re: [gcrahay/otx_misp] Statement: from otx_misp.cli import main gives an error (#4)

Hi Marcus,

I set up a Ubuntu 14.4 VM and found the bug. It should be fixed with the new version of the package on Pypi (1.0.2). I've launched several imports from this VM and had no error. It should work now!

Gaetan

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gcrahay/otx_misp/issues/4#issuecomment-244439149 , or mute the thread https://github.com/notifications/unsubscribe-auth/AUKYu87gIg74XA86YzmOPWzWfU_DbNQhks5qmF3igaJpZM4Jy5v1 . https://github.com/notifications/beacon/AUKYu-oVIXs6cCEJ0EChjx_yex43lgcCks5qmF3igaJpZM4Jy5v1.gif

This mail was scanned by the Astaro Security Gateway of paulis.net.

Der Inhalt dieser E-Mail (einschliesslich etwaiger beigefuegter Dateien) ist vertraulich und nur fuer den Empfaenger bestimmt. Wenn Sie nicht der bestimmungsgemaesse Empfaenger sind oder diese E-Mail irrtuemlich erhalten haben, informieren Sie bitte sofort den Absender und loeschen Sie diese E-Mail (einschliesslich etwaiger beigefuegter Dateien) von Ihrem System. Jegliche Offenlegung, Vervielfaeltigung, Weitergabe oder Nutzung des Inhalts dieser E-Mail sind nicht gestattet. Vielen Dank.

The contents of this e-mail (including any attachments) are confidential and may be legally privileged. If you are not the intended recipient of this e-mail or have received this e-mail in error, please notify the sender immediately and then delete it (including any attachments) from your system. Any disclosure, copying, distribution or use of the material in this e-mail is strictly prohibited. Thank you.

gcrahay commented 8 years ago

Hi Marcus,

Yes, the documentation needs to be improved. You can use a configuration file with the '-c' option:

root@misp:~# otx-misp --otx xxxxxx --server http://10.10.0.4/ --misp xxxxxx -v -a --discover-tags -c otx-misp.ini -w -u

('-w', writes the ini file, '-u', updates the timestamp of last import in this file)

Next calls would be:

root@misp:~# otx-misp -c otx-misp.ini

Gaetan

icepaule commented 8 years ago

Hello Gaetan,

sorry to bother you again …

The initial call of otx-misp worked like a charm and build the ini-file.

But the second call just fails with an error.

But have a look yourself:

root@misp:~# cat otx-misp.ini

[otx_misp]

update_timestamp = yes

to_ids = yes

otx = xxxxxxx

discover_tags = yes

misp = xxxxx

tlp = yes

author = yes

publish = no

server = https://10.10.0.4/

distribution = 0

timestamp = 2016-09-06T12:37:13.639554

root@misp:~# otx-misp -c otx-misp.ini

get() takes at least 3 arguments (2 given)

Again, thanks a lot for all your help and enjoy the weekend. :)

Marcus

M.Pauli

Security Analyst

Schwalbenstrasse 66

85521 Ottobrunn

Tel. +49 89 209 31840

Fax: +49 89 209 67458

D2: +49 176 5769 5212

http://www.mpauli.de/ http://www.mpauli.de

mailto:info@mpauli.de info@mpauli.de

From: Gaetan [mailto:notifications@github.com] Sent: Montag, 5. September 2016 12:07 To: gcrahay/otx_misp otx_misp@noreply.github.com Cc: icepaule info@mpauli.de; Author author@noreply.github.com Subject: Re: [gcrahay/otx_misp] Statement: from otx_misp.cli import main gives an error (#4)

Hi Marcus,

Yes, the documentation needs to be improved. You can use a configuration file with the '-c' option:

root@misp:~# otx-misp --otx xxxxxx --server http://10.10.0.4/ http://10.10.0.4/ --misp xxxxxx -v -a --discover-tags -c otx-misp.ini -w -u

('-w', writes the ini file, '-u', updates the timestamp of last import in this file)

Next calls would be:

root@misp:~# otx-misp -c otx-misp.ini

Gaetan

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gcrahay/otx_misp/issues/4#issuecomment-244709974 , or mute the thread https://github.com/notifications/unsubscribe-auth/AUKYuy-jNAHXPFvL0e8PkY3FZF78pFLsks5qm-mzgaJpZM4Jy5v1 . https://github.com/notifications/beacon/AUKYu-zwBvSYQThyUgc6aUY9drf4biBeks5qm-mzgaJpZM4Jy5v1.gif

This mail was scanned by the Astaro Security Gateway of paulis.net.

Der Inhalt dieser E-Mail (einschliesslich etwaiger beigefuegter Dateien) ist vertraulich und nur fuer den Empfaenger bestimmt. Wenn Sie nicht der bestimmungsgemaesse Empfaenger sind oder diese E-Mail irrtuemlich erhalten haben, informieren Sie bitte sofort den Absender und loeschen Sie diese E-Mail (einschliesslich etwaiger beigefuegter Dateien) von Ihrem System. Jegliche Offenlegung, Vervielfaeltigung, Weitergabe oder Nutzung des Inhalts dieser E-Mail sind nicht gestattet. Vielen Dank.

The contents of this e-mail (including any attachments) are confidential and may be legally privileged. If you are not the intended recipient of this e-mail or have received this e-mail in error, please notify the sender immediately and then delete it (including any attachments) from your system. Any disclosure, copying, distribution or use of the material in this e-mail is strictly prohibited. Thank you.

gcrahay commented 8 years ago

Hi Marcus,

Thanks for the report! I think the new version in Pypi should fix your issue.

Have a nice weekend!

Gaetan

icepaule commented 8 years ago

Hi Gaetan,

wow, you’re my hero…

Worked perfectly well after upgrading.

Thaaanks so much and enjoy the rest of the weekend. :)

Cheers

Marcus

P.S. Whenever you’re coming to Munich – I owe you at least one beer. ;-)

M.Pauli

Security Analyst

Schwalbenstrasse 66

85521 Ottobrunn

Tel. +49 89 209 31840

Fax: +49 89 209 67458

D2: +49 176 5769 5212

http://www.mpauli.de/ http://www.mpauli.de

mailto:info@mpauli.de info@mpauli.de

From: Gaetan [mailto:notifications@github.com] Sent: Samstag, 10. September 2016 12:43 To: gcrahay/otx_misp otx_misp@noreply.github.com Cc: icepaule info@mpauli.de; Author author@noreply.github.com Subject: Re: [gcrahay/otx_misp] Statement: from otx_misp.cli import main gives an error (#4)

Hi Marcus,

Thanks for the report! I think the new version in Pypi should fix your issue.

Have a good weekend!

Gaetan

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gcrahay/otx_misp/issues/4#issuecomment-246104984 , or mute the thread https://github.com/notifications/unsubscribe-auth/AUKYuzXcTHBzDyAKZg7EhNoY03tFpRRHks5qoonLgaJpZM4Jy5v1 . https://github.com/notifications/beacon/AUKYu5vC8XmXZAsyRh5WH_65PEhf5u-Xks5qoonLgaJpZM4Jy5v1.gif

This mail was scanned by the Astaro Security Gateway of paulis.net.

Der Inhalt dieser E-Mail (einschliesslich etwaiger beigefuegter Dateien) ist vertraulich und nur fuer den Empfaenger bestimmt. Wenn Sie nicht der bestimmungsgemaesse Empfaenger sind oder diese E-Mail irrtuemlich erhalten haben, informieren Sie bitte sofort den Absender und loeschen Sie diese E-Mail (einschliesslich etwaiger beigefuegter Dateien) von Ihrem System. Jegliche Offenlegung, Vervielfaeltigung, Weitergabe oder Nutzung des Inhalts dieser E-Mail sind nicht gestattet. Vielen Dank.

The contents of this e-mail (including any attachments) are confidential and may be legally privileged. If you are not the intended recipient of this e-mail or have received this e-mail in error, please notify the sender immediately and then delete it (including any attachments) from your system. Any disclosure, copying, distribution or use of the material in this e-mail is strictly prohibited. Thank you.