MISP-Modules

[cyamal1b4]

Hello, so I just ran the initial download and install of the container, great set up btw so thank you for getting it together. I see how it installed MISP-Modules in the docker-file, however, port 6666 was never exposed and it isn't listening or running the MISP modules service on my host. I saw where querying the http://dockerhostip:6666/modules is supposed to work, it didn’t for me (Or I could just not have the right IP, I pulled it from a docker inspect). Honestly, I have tried for a while now to get misp-modules running via docker or at least with a docker instance of MISP but its killing me. On the MISP UI, I get connection refused, connection timed out, and OK but it never works when I’m running it through docker. Right now the old way of curl -s http://127.0.0.1:6666/modules doesn't work either with this setup, is that correct?. I saw the other issues on here with the modules but apparently it was supposed to crank up with the supervisord file. Any help would be appreciated. Thanks!!

Note: I have had a hosted MISP instance as well as the modules all as services on this host previously. I then moved to another instance of MISP via docker on this host with those modules still running on local host:6666. I couldn’t get them to communicate (I believe it was a docker network/inter host networking-firewall issue.. very strange). Now I decided to run your instance, which is very well established. So thank you again. I believe the misp modules issue is on my machine but I can’t crack it when MISP is in Docker.

[ventz]

@Patriot1b4 You have 2 options:

1.) You have to expose the modules' port at runtime -- it's not by default. Simply by running "docker run ... -p 6666:6666" will do it

2.) Alternatively, you can get a shell onto the contianer "docker exec -it $containerid /bin/bash" and run curl (127.0.0.1:6666) from within that.

[cyamal1b4]

@ventz, Thank you for your prompt reply. Greatly appreciated. So its not tied at all to a local service or anything on my host? It simply needs to be exposed and it is running within that container as well? Just so I am understanding. Right now the local on my host misp modules service is not active/fails to start. The ip is the container IP correct? to put into the GUI?

I loaded in the dockerhostip, the typical 172...1 I get this now …stream_socket_client(): SSL: Success stream_socket_client(): Failed to enable crypto stream_socket_client(): unable to connect to ssl://172.17.0.1:6666 (Unknown error)

[ventz]

@Patriot1b4 Correct, the modules are started as a task, part of the supervisor:

...

[program:misp-modules]
command=/bin/bash -c "/usr/local/bin/misp-modules -l '0.0.0.0' -s"
user = www-data
startsecs = 0

...

As far as the port, yes, it just needs to be "binded" from the container, to the host running the docker container (-p 6666:6666).

If the modules are not starting for you, something else is going on -- either the port is taken (if trying to bind), or another container is already running, etc.

The IP you would them would be the IP of the docker host. (I think you are hitting the container IP instead from the - 172.17.0., and the .1 would be the gateway either way).

[cyamal1b4]

@ventz thanks for the help. Yah I exposed the port as 6666:6666 like you’d mentioned. Also, very strange, but there were missing dependencies that kept it from running from when I installed/built the container. So got a shell into the container and installed some extra python libraries and re ran the modules service at the docker host IP (the .2). It’s up now. Again thanks so much, your teams platform is solid. If any help is needed in dev I’d be happy to assist where needed in the future for any major MISP changes. Take care! I’m sure I’ll be back if something comes up.

[ventz]

@Patriot1b4 Great - thanks.

What did you have to install in terms of python? It's probably something we should include in the image if it's generic enough.

[ventz]

@Patriot1b4 Looked at the MISP modules list. It looks like the recommendations has changed from: 1.) Pining specific branch to 'latest' and 2.) Package list has changed to:

python3-dev python3-pip libpq5 libjpeg-dev tesseract-ocr libpoppler-cpp-dev imagemagick virtualenv libopencv-dev zbar-tools libzbar0 libzbar-dev libfuzzy-dev build-essential -y

Incoporating both. This also contains some of the packages @angarod had mentioned here: https://github.com/harvard-itsecurity/docker-misp/pull/40

[cyamal1b4]

@ventz Yes some of those libraries were missing as well as pyfaup. Had to pull that down separately. Bash shell into the container then ran these commands: cd /usr/local/src sudo -H git clone git://github.com/stricaud/faup.git faup cd faup sudo -H mkdir -p build cd build sudo -H cmake .. && sudo -H make (had to pull down make after error) sudo make install sudo ldconfig cd /usr/local/src/faup/src/lib/bindings chown -R www-data:www-data python cd python sudo -H -u www-data /var/www/MISP/bin/python3 setup.py install

I then started with the .2 IP and it has worked since. Although the modules are weird in the GUI, they say failed but completed and still give me results. Very strange logging.