amuehlem / MISP-RPM

RPM packages for MISP
34 stars 15 forks source link

Warning during the starting of MISP-Modules #36

Closed Tyrell20 closed 3 years ago

Tyrell20 commented 4 years ago

Hello, during the start of misp-module 's service I am faced to several warning, listed below:

Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,599 - misp-modules - WARNING - MISP modules apiosintds failed due to No module named 'apiosintDS' Sep 17 12:09:41 misp-modules: WARNING [__init__.py:50 - <module>() ] Unable to load pymisp properly: No module named 'deprecated' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,602 - misp-modules - WARNING - MISP modules apivoid failed due to cannot import name 'MISPAttribute' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,630 - misp-modules - WARNING - MISP modules assemblyline_query failed due to No module named 'assemblyline_client' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,634 - misp-modules - WARNING - MISP modules assemblyline_submit failed due to No module named 'assemblyline_client' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,635 - misp-modules - WARNING - MISP modules vulners failed due to No module named 'vulners' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,638 - misp-modules - WARNING - MISP modules bgpranking failed due to No module named 'pybgpranking' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,662 - misp-modules - WARNING - MISP modules xforceexchange failed due to cannot import name 'MISPAttribute' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,663 - misp-modules - WARNING - MISP modules xlsx_enrich failed due to No module named 'np' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,671 - misp-modules - WARNING - MISP modules circl_passivedns failed due to cannot import name 'MISPAttribute' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,678 - misp-modules - WARNING - MISP modules circl_passivessl failed due to cannot import name 'MISPAttribute' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,684 - misp-modules - WARNING - MISP modules cve_advanced failed due to cannot import name 'MISPEvent' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,686 - misp-modules - WARNING - MISP modules docx_enrich failed due to No module named 'np' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,772 - misp-modules - WARNING - MISP modules geoip_country failed due to No module named 'geoip2' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,773 - misp-modules - WARNING - MISP modules intel471 failed due to No module named 'pyintel471' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,774 - misp-modules - WARNING - MISP modules ipasn failed due to No module named 'pyipasnhistory' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,774 - misp-modules - WARNING - MISP modules joesandbox_query failed due to No module named 'jbxapi' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,775 - misp-modules - WARNING - MISP modules joesandbox_submit failed due to No module named 'jbxapi' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,777 - misp-modules - WARNING - MISP modules macaddress_io failed due to No module named 'maclookup' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,778 - misp-modules - WARNING - MISP modules ocr_enrich failed due to No module named 'cv2' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,778 - misp-modules - WARNING - MISP modules ods_enrich failed due to No module named 'np' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,779 - misp-modules - WARNING - MISP modules odt_enrich failed due to No module named 'np' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,782 - misp-modules - WARNING - MISP modules pdf_enrich failed due to No module named 'np' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,782 - misp-modules - WARNING - MISP modules pptx_enrich failed due to No module named 'np' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,783 - misp-modules - WARNING - MISP modules qrcode failed due to No module named 'pyzbar' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,783 - misp-modules - WARNING - MISP modules ransomcoindb failed due to No module named 'expansion._ransomcoindb' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,785 - misp-modules - WARNING - MISP modules securitytrails failed due to No module named 'dnstrails' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,785 - misp-modules - WARNING - MISP modules virustotal_public failed due to cannot import name 'MISPAttribute' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,824 - misp-modules - WARNING - MISP modules urlhaus failed due to cannot import name 'MISPAttribute' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,825 - misp-modules - WARNING - MISP modules virustotal failed due to cannot import name 'MISPAttribute' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,827 - misp-modules - WARNING - MISP modules goamlexport failed due to cannot import name 'MISPEvent' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,829 - misp-modules - WARNING - MISP modules pdfexport failed due to cannot import name 'MISPEvent' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,835 - misp-modules - WARNING - MISP modules csvimport failed due to cannot import name 'MISPEvent' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,837 - misp-modules - WARNING - MISP modules cuckooimport failed due to cannot import name 'MISPEvent' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,842 - misp-modules - WARNING - MISP modules email_import failed due to cannot import name 'MISPObject' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,845 - misp-modules - WARNING - MISP modules goamlimport failed due to cannot import name 'MISPEvent' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,846 - misp-modules - WARNING - MISP modules joe_import failed due to cannot import name 'MISPAttribute' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,848 - misp-modules - WARNING - MISP modules openiocimport failed due to cannot import name 'MISPObject' Sep 17 12:09:41 misp-modules: 2020-09-17 12:09:41,849 - misp-modules - WARNING - MISP modules stiximport failed due to cannot import name 'MISPObject'

The result of this is that on MISP I am not able to use all module. Is it possible that MISP missed some dependencies?

0x7fff9 commented 4 years ago

Hi, same issue here! I installed most of the missing python modules with pip and although that is just a warning and isn't something that would stop some of the modules to start I did it anyway. The modules just won't start. From /var/log/messages it even prints INFO - MISP modules server started on localhost port 6666 however nothing is listening on 6666 if you check with ss -tunlp

any idea of what can be missing to start these modules?

thanks in advance and great work on making this project!!

cheers.

amuehlem commented 4 years ago

Hi 0x7fff9

Do you see a misp-modules process? ps -ef|grep misp-modules should show at least the python process.

The dependencies are quite a hassle, I'm working on a new misp-modules RPM which has all the dependencies in a python virtual env, but some of them are not automatically installed.

Cheers

amuehlem commented 4 years ago

Hi Tyrell20

I've created a new misp-modules-2.4.121-1.el7.x86_64 RPM which can be found in the repository. I've moved all misp-modules and dependencies into a python virtual environment. Like this the logs are clean and show no errors.

Cheers

Tyrell20 commented 4 years ago

Hello @amuehlem many thanks for awesome job for this project.

I tried to download the new misp-module rpm but I am facing to this issue:

Loaded plugins: product-id, rhnplugin, search-disabled-repos, subscription-manager This system is receiving updates from RHN Classic or Red Hat Satellite. Resolving Dependencies --> Running transaction check ---> Package misp-modules.noarch 0:1.0-8.el7 will be updated ---> Package misp-modules.x86_64 0:2.4.121-1.el7 will be an update --> Processing Dependency: libpoppler-cpp.so.10()(64bit) for package: misp-modules-2.4.121-1.el7.x86_64 --> Finished Dependency Resolution Error: Package: misp-modules-2.4.121-1.el7.x86_64 (misp) Requires: libpoppler-cpp.so.10()(64bit) You could try using --skip-broken to work around the problem You could try running: rpm -Va --nofiles --nodigest

Could you please check? If you need an help to investigate I am at your disposal.

Many thanks

0x7fff9 commented 4 years ago

Hey! sorry to come back at this. but the behavior is more os less the same. basically it prints that the modules started on 6666 to /var/log/messages but no listener is seen or modules are available on MISP UI.

INFO - misp-modules:287 - MISP modules server started on localhost port 6666

ss -tunlp |grep LISTEN
tcp    LISTEN     0      128       *:443                   *:*                   users:(("httpd",pid=31415,fd=4),("httpd",pid=30350,fd=4),("httpd",pid=30297,fd=4),("httpd",pid=23215,fd=4),("httpd",pid=23201,fd=4),("httpd",pid=13367,fd=4),("httpd",pid=13018,fd=4),("httpd",pid=12865,fd=4),("httpd",pid=12238,fd=4),("httpd",pid=9005,fd=4),("httpd",pid=2566,fd=4))
tcp    LISTEN     0      128    127.0.0.1:6379                  *:*                   users:(("redis-server",pid=12599,fd=4))
tcp    LISTEN     0      128       *:80                    *:*                   users:(("httpd",pid=31415,fd=3),("httpd",pid=30350,fd=3),("httpd",pid=30297,fd=3),("httpd",pid=23215,fd=3),("httpd",pid=23201,fd=3),("httpd",pid=13367,fd=3),("httpd",pid=13018,fd=3),("httpd",pid=12865,fd=3),("httpd",pid=12238,fd=3),("httpd",pid=9005,fd=3),("httpd",pid=2566,fd=3))
tcp    LISTEN     0      80     [::]:3306               [::]:*                   users:(("mysqld",pid=12546,fd=18))
ps auxf |grep misp-modules
apache   31244  0.5  0.3 1052656 126824 ?      Ssl  14:54   0:02 /opt/misp-modules-venv/bin/python3 /opt/misp-modules-venv/bin/misp-modules
curl -v http://127.0.0.1:6666
* About to connect() to 127.0.0.1 port 6666 (#0)
*   Trying 127.0.0.1...
* Connection refused
* Failed connect to 127.0.0.1:6666; Connection refused
* Closing connection 0
curl: (7) Failed connect to 127.0.0.1:6666; Connection refused

Any idea on what can be happening here? From the ps auxf output, could it be that is misisng the -l 127.0.0.1 -s arguments? thanks! cheers.

EDIT: I just tested, if manually run /opt/misp-modules-venv/bin/python3 /opt/misp-modules-venv/bin/misp-modules -l 127.0.0.1 -s all works well.

curl -v http://127.0.0.1:6666
* About to connect() to 127.0.0.1 port 6666 (#0)
*   Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 6666 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 127.0.0.1:6666
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Server: TornadoServer/6.0.4
< Content-Type: text/html; charset=UTF-8
< Date: Thu, 01 Oct 2020 15:14:38 GMT
< Content-Length: 69

cheers.

0x7fff9 commented 4 years ago

Hi @Tyrell20 you will need to install poppler-cpp-devel and due to how some packages are built also glibc.i686 cheers

Tyrell20 commented 4 years ago

Hi @Tyrell20 you will need to install poppler-cpp-devel and due to how some packages are built also glibc.i686 cheers

Hi @0x7fff9 many thanks. Normal installing does not work. Trying to install the package via RPM I obtaining the error below:

[root@ws301cip opt]# rpm -Uhv poppler-cpp-devel-0.26.5-42.el7.x86_64.rpm warning: poppler-cpp-devel-0.26.5-42.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY error: Failed dependencies: libpoppler-cpp.so.10()(64bit) is needed by poppler-cpp-devel-0.26.5-42.el7.x86_64 pkgconfig(poppler) = 0.26.5 is needed by poppler-cpp-devel-0.26.5-42.el7.x86_64 poppler-cpp(x86-64) = 0.26.5-42.el7 is needed by poppler-cpp-devel-0.26.5-42.el7.x86_64 poppler-devel(x86-64) = 0.26.5-42.el7 is needed by poppler-cpp-devel-0.26.5-42.el7.x86_64

Tyrell20 commented 4 years ago

Hey! sorry to come back at this. but the behavior is more os less the same. basically it prints that the modules started on 6666 to /var/log/messages but no listener is seen or modules are available on MISP UI.

INFO - misp-modules:287 - MISP modules server started on localhost port 6666

ss -tunlp |grep LISTEN
tcp    LISTEN     0      128       *:443                   *:*                   users:(("httpd",pid=31415,fd=4),("httpd",pid=30350,fd=4),("httpd",pid=30297,fd=4),("httpd",pid=23215,fd=4),("httpd",pid=23201,fd=4),("httpd",pid=13367,fd=4),("httpd",pid=13018,fd=4),("httpd",pid=12865,fd=4),("httpd",pid=12238,fd=4),("httpd",pid=9005,fd=4),("httpd",pid=2566,fd=4))
tcp    LISTEN     0      128    127.0.0.1:6379                  *:*                   users:(("redis-server",pid=12599,fd=4))
tcp    LISTEN     0      128       *:80                    *:*                   users:(("httpd",pid=31415,fd=3),("httpd",pid=30350,fd=3),("httpd",pid=30297,fd=3),("httpd",pid=23215,fd=3),("httpd",pid=23201,fd=3),("httpd",pid=13367,fd=3),("httpd",pid=13018,fd=3),("httpd",pid=12865,fd=3),("httpd",pid=12238,fd=3),("httpd",pid=9005,fd=3),("httpd",pid=2566,fd=3))
tcp    LISTEN     0      80     [::]:3306               [::]:*                   users:(("mysqld",pid=12546,fd=18))
ps auxf |grep misp-modules
apache   31244  0.5  0.3 1052656 126824 ?      Ssl  14:54   0:02 /opt/misp-modules-venv/bin/python3 /opt/misp-modules-venv/bin/misp-modules
curl -v http://127.0.0.1:6666
* About to connect() to 127.0.0.1 port 6666 (#0)
*   Trying 127.0.0.1...
* Connection refused
* Failed connect to 127.0.0.1:6666; Connection refused
* Closing connection 0
curl: (7) Failed connect to 127.0.0.1:6666; Connection refused

Any idea on what can be happening here? From the ps auxf output, could it be that is misisng the -l 127.0.0.1 -s arguments? thanks! cheers.

EDIT: I just tested, if manually run /opt/misp-modules-venv/bin/python3 /opt/misp-modules-venv/bin/misp-modules -l 127.0.0.1 -s all works well.

curl -v http://127.0.0.1:6666
* About to connect() to 127.0.0.1 port 6666 (#0)
*   Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 6666 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 127.0.0.1:6666
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Server: TornadoServer/6.0.4
< Content-Type: text/html; charset=UTF-8
< Date: Thu, 01 Oct 2020 15:14:38 GMT
< Content-Length: 69

cheers.

Hi @0x7fff9 could you try to disable the IPv6? We solved the issue on this way

0x7fff9 commented 4 years ago

Hey! sorry to come back at this. but the behavior is more os less the same. basically it prints that the modules started on 6666 to /var/log/messages but no listener is seen or modules are available on MISP UI. INFO - misp-modules:287 - MISP modules server started on localhost port 6666

ss -tunlp |grep LISTEN
tcp    LISTEN     0      128       *:443                   *:*                   users:(("httpd",pid=31415,fd=4),("httpd",pid=30350,fd=4),("httpd",pid=30297,fd=4),("httpd",pid=23215,fd=4),("httpd",pid=23201,fd=4),("httpd",pid=13367,fd=4),("httpd",pid=13018,fd=4),("httpd",pid=12865,fd=4),("httpd",pid=12238,fd=4),("httpd",pid=9005,fd=4),("httpd",pid=2566,fd=4))
tcp    LISTEN     0      128    127.0.0.1:6379                  *:*                   users:(("redis-server",pid=12599,fd=4))
tcp    LISTEN     0      128       *:80                    *:*                   users:(("httpd",pid=31415,fd=3),("httpd",pid=30350,fd=3),("httpd",pid=30297,fd=3),("httpd",pid=23215,fd=3),("httpd",pid=23201,fd=3),("httpd",pid=13367,fd=3),("httpd",pid=13018,fd=3),("httpd",pid=12865,fd=3),("httpd",pid=12238,fd=3),("httpd",pid=9005,fd=3),("httpd",pid=2566,fd=3))
tcp    LISTEN     0      80     [::]:3306               [::]:*                   users:(("mysqld",pid=12546,fd=18))
ps auxf |grep misp-modules
apache   31244  0.5  0.3 1052656 126824 ?      Ssl  14:54   0:02 /opt/misp-modules-venv/bin/python3 /opt/misp-modules-venv/bin/misp-modules
curl -v http://127.0.0.1:6666
* About to connect() to 127.0.0.1 port 6666 (#0)
*   Trying 127.0.0.1...
* Connection refused
* Failed connect to 127.0.0.1:6666; Connection refused
* Closing connection 0
curl: (7) Failed connect to 127.0.0.1:6666; Connection refused

Any idea on what can be happening here? From the ps auxf output, could it be that is misisng the -l 127.0.0.1 -s arguments? thanks! cheers. EDIT: I just tested, if manually run /opt/misp-modules-venv/bin/python3 /opt/misp-modules-venv/bin/misp-modules -l 127.0.0.1 -s all works well.

curl -v http://127.0.0.1:6666
* About to connect() to 127.0.0.1 port 6666 (#0)
*   Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 6666 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 127.0.0.1:6666
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Server: TornadoServer/6.0.4
< Content-Type: text/html; charset=UTF-8
< Date: Thu, 01 Oct 2020 15:14:38 GMT
< Content-Length: 69

cheers.

Hi @0x7fff9 could you try to disable the IPv6? We solved the issue on this way

ohh that makes sense since maybe it doesn't know where to bind. cool!

0x7fff9 commented 4 years ago

Hi! Ok, so this looks much better now (after adding the -l parameters) and I can see modules that I enabled.

MISP1

however I am experiencing the following now: 1- I am not getting any box/prompt asking for PDF. 2- Onjobs it states Enrichment finished, but no attributes added.
MISP2

3- The logfile prints this message that I can't make any sense of

misp-modules: 2020-10-01 16:04:47,200 - INFO    - tornado.access:2250 - 200 GET /modules (127.0.0.1) 2.14ms
misp-modules: 'data'
misp-modules: Couldn't fetch attachment (JSON 'data' is empty). Are you using the 'Query enrichment' action?

Any idea of what can be happening here?

thanks! cheers.

0x7fff9 commented 4 years ago

Hi @Tyrell20 you will need to install poppler-cpp-devel and due to how some packages are built also glibc.i686 cheers

Hi @0x7fff9 many thanks. Normal installing does not work. Trying to install the package via RPM I obtaining the error below:

[root@ws301cip opt]# rpm -Uhv poppler-cpp-devel-0.26.5-42.el7.x86_64.rpm warning: poppler-cpp-devel-0.26.5-42.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY error: Failed dependencies: libpoppler-cpp.so.10()(64bit) is needed by poppler-cpp-devel-0.26.5-42.el7.x86_64 pkgconfig(poppler) = 0.26.5 is needed by poppler-cpp-devel-0.26.5-42.el7.x86_64 poppler-cpp(x86-64) = 0.26.5-42.el7 is needed by poppler-cpp-devel-0.26.5-42.el7.x86_64 poppler-devel(x86-64) = 0.26.5-42.el7 is needed by poppler-cpp-devel-0.26.5-42.el7.x86_64

Hey @Tyrell20 looks like you still miss some deps. a lot of libs for both i686 and x64 are required to make this new modules RPM install successfully.

cheers.

amuehlem commented 4 years ago

poppler-cpp is now included as a dependeny in the misp-modules RPM.

Tyrell20 commented 4 years ago

poppler-cpp is now included as a dependeny in the misp-modules RPM.

Many thanks @amuehlem unfortunatelly the issue persists:

[root@XXX yyyy]# yum clean all Loaded plugins: product-id, rhnplugin, search-disabled-repos, subscription-manager Cleaning repos: mariadb misp rhel7-x86_64-server rhel7-x86_64-spacewalktools rhel7_x86_64-epel [root@ XXX yyyy]# yum update Loaded plugins: product-id, rhnplugin, search-disabled-repos, subscription-manager This system is receiving updates from RHN Classic or Red Hat Satellite. mariadb | 2.9 kB 00:00:00 misp | 2.9 kB 00:00:00 (1/2): misp/primary_db | 152 kB 00:00:00 (2/2): mariadb/primary_db | 49 kB 00:00:00 rhel7-x86_64-server | 1.3 kB 00:00:00 rhel7-x86_64-server/group | 631 kB 00:00:00 rhel7-x86_64-server/updateinfo | 3.5 MB 00:00:00 rhel7-x86_64-server/primary | 41 MB 00:00:00 rhel7-x86_64-server 29431/29431 rhel7-x86_64-spacewalktools | 871 B 00:00:00 rhel7-x86_64-spacewalktools/primary | 67 kB 00:00:00 rhel7-x86_64-spacewalktools 160/160 rhel7_x86_64-epel | 1.3 kB 00:00:00 rhel7_x86_64-epel/group | 389 kB 00:00:00 rhel7_x86_64-epel/updateinfo | 3.1 MB 00:00:00 rhel7_x86_64-epel/primary | 5.9 MB 00:00:00 rhel7_x86_64-epel 21195/21195 Resolving Dependencies --> Running transaction check ---> Package misp-modules.noarch 0:1.0-8.el7 will be updated ---> Package misp-modules.x86_64 0:2.4.121-2.el7 will be an update --> Processing Dependency: poppler-cpp for package: misp-modules-2.4.121-2.el7.x86_64 --> Processing Dependency: libpoppler-cpp.so.10()(64bit) for package: misp-modules-2.4.121-2.el7.x86_64 --> Finished Dependency Resolution Error: Package: misp-modules-2.4.121-2.el7.x86_64 (misp) Requires: libpoppler-cpp.so.10()(64bit) Error: Package: misp-modules-2.4.121-2.el7.x86_64 (misp) Requires: poppler-cpp You could try using --skip-broken to work around the problem You could try running: rpm -Va --nofiles --nodigest [root@ XXX yyyy]# yum install *poppler-cpp* Loaded plugins: product-id, rhnplugin, search-disabled-repos, subscription-manager This system is receiving updates from RHN Classic or Red Hat Satellite. mariadb/filelists_db | 262 kB 00:00:00 misp/filelists_db | 2.0 MB 00:00:00 rhel7-x86_64-server/filelists | 53 MB 00:00:00 rhel7-x86_64-spacewalktools/filelists | 102 kB 00:00:00 rhel7_x86_64-epel/filelists | 42 MB 00:00:00 No package *poppler-cpp* available. Error: Nothing to do

amuehlem commented 4 years ago

The package poppler-cpp is in the repository rhel-7-server-optional-rpms/x86_64, it seems it is not activated on your system.

0x7fff9 commented 4 years ago

@amuehlem should I open an issue for the matter I am experiencing? I was assuming it was similar to @Tyrell20 's one but now starting not to look like that.

cheers.

amuehlem commented 4 years ago

The issue with the PDF checkbox?

0x7fff9 commented 4 years ago

no, in this case I think I got the idea wrong of how it works. I thought it would ask for a PDF but that is supose to enrich from an attachment of an event.

but if I have an event witth attributes like IPs attached and then try to enrich this event from this attachment nothing happens and on jobs I see: Event ID: 6494 modules: ["pdf_enrich"] | Enrichment finished, but no attributes added. and on logs:

misp-modules: 'data'
misp-modules: Couldn't fetch attachment (JSON 'data' is empty). Are you using the 'Query enrichment' action?
misp-modules: 2020-10-02 10:57:48,056 - INFO    - tornado.access:2250 - 200 POST /query (127.0.0.1) 0.73ms

cheers.

amuehlem commented 4 years ago

It doesn't work for me neither, but I think this could be problem with the module itself

https://github.com/MISP/misp-modules/blob/main/misp_modules/modules/expansion/pdf_enrich.py

0x7fff9 commented 4 years ago

well if you attach a PDF to an event and then go here: MISP3 the freetxt parsing works flawlessly. cheers.

amuehlem commented 4 years ago

Same here, I think this might be an issue with the module itself and should be reported there. Or "Enrich Event" is a different function than "Query enrichment". From the module code, the "Query Enrichment" should be used by the module

"Couldn't fetch attachment (JSON 'data' is empty). Are you using the 'Query enrichment' action?"

0x7fff9 commented 4 years ago

ye, that error is pretty vague to me! I think I'll open an issue on MISP itself and see what happens!

thanks once again for suppot and this project that is very helpful when git and open internet is not a possibility! 👍

amuehlem commented 4 years ago

I've asked in the MISP Support Channel on gitter, no answer so far :-/

0x7fff9 commented 4 years ago

oh, thanks for that!!