search

OpenCTI-Platform / opencti

Open Cyber Threat Intelligence Platform
https://opencti.io
Other
5.2k stars 821 forks source link

MISP integration is not working #106

Closed oz17il closed 5 years ago

oz17il commented 5 years ago

Hi, The MISP integration is not working after the initial settings. (misp url, api key, tags. etc.) Can't find any log event that indicates about the problem.

Reproducible Steps

  1. I validate that the misp.py is run without any issues directly from the CLI.
  2. Network connection from OpenCTI to MISP in port 443 is alright.

Environment

  1. OS Ubuntu 18.04
  2. OpenCTI version: Version 1.0.1
  3. Manual installation.
AntoninHL commented 5 years ago

same issue on my side with same verifications.

SamuelHassine commented 5 years ago

@oz17il, @weSh: could you please look at the log file scheduler.log in the integration docker (and the /opt/opencti/integration folder? You should have also a /opt/opencti/integration/connectors/misp/misp.log file.

AntoninHL commented 5 years ago

no log files for me... Nothing you describe exists on my install

oz17il commented 5 years ago

no log files for me... Nothing you describe exists on my install

also on my side ..

SamuelHassine commented 5 years ago

@oz17il, @weSh:

$ docker exec -ti openctidocker_integration_1 /bin/bash
$ cd /opt/opencti/integration
$ cat scheduler.log
$ cd connectors/misp
$ cat misp.log

Please provide us the content of these logs.

AntoninHL commented 5 years ago

For me:

scheduler.log:

[2019-07-02 08:36:41] Running misp
[2019-07-02 08:37:41] Configuring connectors
[2019-07-02 08:37:42] Connector misp configured
[2019-07-02 08:37:42] Connector mitre configured
[2019-07-02 08:37:42] Connector openctidata configured
[2019-07-02 08:37:42] Running misp

misp.log doesn't exists.

SamuelHassine commented 5 years ago

@weSh: your log is correct and no error. Do you tag an event in your MISP instance with the tag you specified in the configuration of your connector?

AntoninHL commented 5 years ago

@SamuelHassine yep. In my MISP, I have: image

so in my configuration I put: OSINT

SamuelHassine commented 5 years ago

You have to put type:OSINT in your configuration.

oz17il commented 5 years ago

In my case I implemented OpenCti without docker. Logs not found: $ cat scheduler.log $ cat misp.log

AntoninHL commented 5 years ago

On my side it's ok now. I think the issue with persistence was linked to this... I'm not sure, but since the configuration works, MISP also works...

oz17il commented 5 years ago

On my side it's ok now. I think the issue with persistence was linked to this... I'm not sure, but since the configuration works, MISP also works...

Can you explain what do you mean when you say persistence?

SamuelHassine commented 5 years ago

@oz17il: Is scheduler.py launched from the integration directory? You should have a scheduler.log in the same directory. Could you please try with the new release: https://github.com/OpenCTI-Platform/opencti/releases/download/1.0.1/opencti-release.tar.gz.

@oz17il: the persistence is just for ElasticSearch and Grakn running with Docker (for volumes).

oz17il commented 5 years ago

@oz17il: Is scheduler.py launched from the integration directory? You should have a scheduler.log in the same directory. Could you please try with the new release: https://github.com/OpenCTI-Platform/opencti/releases/download/1.0.1/opencti-release.tar.gz.

@oz17il: the persistence is just for ElasticSearch and Grakn running with Docker (for volumes).

The scheduler.log not found.

I try to run the connectors_scheduler.py and I got this error: root@:/opt/opencti/integration# python3 connectors_scheduler.py [Errno 2] No such file or directory: '/opt/opencti/integration/config.yml'

SamuelHassine commented 5 years ago

@oz17il: According to the documentation you have to create the config.yml file.

If you create this file, the connectors scheduler should work.

SamuelHassine commented 5 years ago

@oz17il : did you manage to make the scheduler working?

oz17il commented 5 years ago

@oz17il : did you manage to make the scheduler working?

Hi, where can I find the values for the config.yml? api_url: 'http://REPLACE_API_URL' api_key: 'REPLACE_API_KEY'

The log now looks ok but still no event from MISP: root@ip-xxxxx:/opt/opencti/integration# tail -f scheduler.log [2019-07-04 13:54:58] Updating connector config of misp... [2019-07-04 13:54:59] Running misp [2019-07-04 13:54:59] Connector mitre configured [2019-07-04 13:54:59] Connector openctidata configured [2019-07-04 13:54:59] Running misp [2019-07-04 13:55:59] Configuring connectors [2019-07-04 13:56:00] Connector misp configured [2019-07-04 13:56:00] Connector mitre configured [2019-07-04 13:56:00] Connector openctidata configured [2019-07-04 13:56:00] Running misp

SamuelHassine commented 5 years ago

@oz17il: We just fully refactor the MISP connector and the way it communicates with our API. We will release this in a few hours.

syloktools commented 5 years ago

Question, when I try I get the following error:

[2019-07-05 18:51:40] Configuring connectors
[2019-07-05 18:51:40] You are not allowed to do this.
Deventual commented 5 years ago

Hi,

Any updates on this issue? Our setup could import only 2 indicators from our misp instance.

SamuelHassine commented 5 years ago

Hi everyone,

We are working hard to fully refactor the connectors/schedulers/worker architecture for the next major release. Please see #121. For the moment, we just advise you to check the scheduler and the worker log files.