[IVRE] Connector unable to Connect to OpenCTI Platform

izm1chael commented 2 years ago

Prerequisites

[X] I read the Deployment and Setup section of the OpenCTI documentation as well as the Troubleshooting page and didn't find anything relevant to my problem.
[X] I went through old GitHub issues and couldn't find anything relevant
[X] I googled the issue and didn't find anything relevant

Description

Environment

OS (where OpenCTI server runs): Ubuntu 20.04
OpenCTI version: 5.3.6
OpenCTI client: Frontend
Other environment details: Running on "Bare Metal", with pm2 to mange the connectors

Reproducible Steps

Steps to create the smallest reproducible scenario:

run the IVRE Connector

Additional information

OpenCTI appears to be detecting the IVRE connector and is able to queue jobs for it, but there appears to be an issue in the connection between the connector and the platform.

Config is as follows:

opencti:
  url: 'http://localhost:4000'
  token: 'OpenCTI-UUID'

connector:
  id: 'IVRE-UUID'
  type: 'INTERNAL_ENRICHMENT'
  name: 'IVRE'
  auto: true
  scope: 'autonomous-system,domain-name,ipv4-addr,ipv6-addr,mac-addr,x509-certificate'
  confidence_level: 100 # From 0 (Unknown) to 100 (Fully trusted)
  log_level: 'debug'

ivre:
  db_url: mongodb://10.2.0.4/ivre
  db_url_data: maxmind:///usr/local/share/ivre/geoip/
  db_url_passive: mongodb://10.2.0.4/ivre
  db_url_scans: mongodb://10.2.0.4/ivre
  max_tlp: 'TLP:RED'
  use_data: true
  use_passive: true
  use_passive_as: true
  use_passive_domain: true
  use_scans: true
  use_scans_as: true
  use_scans_domain: true

Output form the terminal

76|Enrichment-IVRE  | ERROR:pika.adapters.utils.io_services_utils:_AsyncBaseTransport._produce() failed, aborting connection: error=ConnectionResetError(104, 'Connection reset by peer'); sock=<socket.socket fd=8, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.2.0.5', 48836)>; Caller's stack:
76|Enrichment-IVRE  | Traceback (most recent call last):
76|Enrichment-IVRE  |   File "/home/azureuser/.local/lib/python3.8/site-packages/pika/adapters/utils/io_services_utils.py", line 1103, in _on_socket_writable
76|Enrichment-IVRE  |     self._produce()
76|Enrichment-IVRE  |   File "/home/azureuser/.local/lib/python3.8/site-packages/pika/adapters/utils/io_services_utils.py", line 819, in _produce
76|Enrichment-IVRE  |     num_bytes_sent = self._sigint_safe_send(self._sock,
76|Enrichment-IVRE  |   File "/home/azureuser/.local/lib/python3.8/site-packages/pika/adapters/utils/io_services_utils.py", line 79, in retry_sigint_wrap
76|Enrichment-IVRE  |     return func(*args, **kwargs)
76|Enrichment-IVRE  |   File "/home/azureuser/.local/lib/python3.8/site-packages/pika/adapters/utils/io_services_utils.py", line 861, in _sigint_safe_send
76|Enrichment-IVRE  |     return sock.send(data)
76|Enrichment-IVRE  | ConnectionResetError: [Errno 104] Connection reset by peer
76|Enrichment-IVRE  | Traceback (most recent call last):
76|Enrichment-IVRE  |   File "/home/azureuser/.local/lib/python3.8/site-packages/pika/adapters/utils/io_services_utils.py", line 1103, in _on_socket_writable
76|Enrichment-IVRE  |     self._produce()
76|Enrichment-IVRE  |   File "/home/azureuser/.local/lib/python3.8/site-packages/pika/adapters/utils/io_services_utils.py", line 819, in _produce
76|Enrichment-IVRE  |     num_bytes_sent = self._sigint_safe_send(self._sock,
76|Enrichment-IVRE  |   File "/home/azureuser/.local/lib/python3.8/site-packages/pika/adapters/utils/io_services_utils.py", line 79, in retry_sigint_wrap
76|Enrichment-IVRE  |     return func(*args, **kwargs)
76|Enrichment-IVRE  |   File "/home/azureuser/.local/lib/python3.8/site-packages/pika/adapters/utils/io_services_utils.py", line 861, in _sigint_safe_send
76|Enrichment-IVRE  |     return sock.send(data)
76|Enrichment-IVRE  | ConnectionResetError: [Errno 104] Connection reset by peer
76|Enrichment-IVRE  | ERROR:pika.adapters.base_connection:connection_lost: StreamLostError: ("Stream connection lost: ConnectionResetError(104, 'Connection reset by peer')",)
76|Enrichment-IVRE  | ERROR:pika.adapters.blocking_connection:Unexpected connection close detected: StreamLostError: ("Stream connection lost: ConnectionResetError(104, 'Connection reset by peer')",)
76|Enrichment-IVRE  | ERROR:root:Stream connection lost: ConnectionResetError(104, 'Connection reset by peer')
76|Enrichment-IVRE  | ERROR:pika.adapters.utils.io_services_utils:_AsyncBaseTransport._produce() failed, aborting connection: error=ConnectionResetError(104, 'Connection reset by peer'); sock=<socket.socket fd=8, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.2.0.5', 49042)>; Caller's stack:
76|Enrichment-IVRE  | Traceback (most recent call last):
76|Enrichment-IVRE  |   File "/home/azureuser/.local/lib/python3.8/site-packages/pika/adapters/utils/io_services_utils.py", line 1103, in _on_socket_writable
76|Enrichment-IVRE  |     self._produce()
76|Enrichment-IVRE  |   File "/home/azureuser/.local/lib/python3.8/site-packages/pika/adapters/utils/io_services_utils.py", line 819, in _produce
76|Enrichment-IVRE  |     num_bytes_sent = self._sigint_safe_send(self._sock,
76|Enrichment-IVRE  |   File "/home/azureuser/.local/lib/python3.8/site-packages/pika/adapters/utils/io_services_utils.py", line 79, in retry_sigint_wrap
76|Enrichment-IVRE  |     return func(*args, **kwargs)
76|Enrichment-IVRE  |   File "/home/azureuser/.local/lib/python3.8/site-packages/pika/adapters/utils/io_services_utils.py", line 861, in _sigint_safe_send
76|Enrichment-IVRE  |     return sock.send(data)
76|Enrichment-IVRE  | ConnectionResetError: [Errno 104] Connection reset by peer
76|Enrichment-IVRE  | Traceback (most recent call last):
76|Enrichment-IVRE  |   File "/home/azureuser/.local/lib/python3.8/site-packages/pika/adapters/utils/io_services_utils.py", line 1103, in _on_socket_writable
76|Enrichment-IVRE  |     self._produce()
76|Enrichment-IVRE  |   File "/home/azureuser/.local/lib/python3.8/site-packages/pika/adapters/utils/io_services_utils.py", line 819, in _produce
76|Enrichment-IVRE  |     num_bytes_sent = self._sigint_safe_send(self._sock,
76|Enrichment-IVRE  |   File "/home/azureuser/.local/lib/python3.8/site-packages/pika/adapters/utils/io_services_utils.py", line 79, in retry_sigint_wrap
76|Enrichment-IVRE  |     return func(*args, **kwargs)
76|Enrichment-IVRE  |   File "/home/azureuser/.local/lib/python3.8/site-packages/pika/adapters/utils/io_services_utils.py", line 861, in _sigint_safe_send
76|Enrichment-IVRE  |     return sock.send(data)
76|Enrichment-IVRE  | ConnectionResetError: [Errno 104] Connection reset by peer
76|Enrichment-IVRE  | ERROR:pika.adapters.base_connection:connection_lost: StreamLostError: ("Stream connection lost: ConnectionResetError(104, 'Connection reset by peer')",)
76|Enrichment-IVRE  | ERROR:pika.adapters.blocking_connection:Unexpected connection close detected: StreamLostError: ("Stream connection lost: ConnectionResetError(104, 'Connection reset by peer')",)
76|Enrichment-IVRE  | ERROR:root:Stream connection lost: ConnectionResetError(104, 'Connection reset by peer')
76|Enrichment-IVRE  | ERROR:pika.adapters.utils.io_services_utils:_AsyncBaseTransport._produce() failed, aborting connection: error=ConnectionResetError(104, 'Connection reset by peer'); sock=<socket.socket fd=8, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.2.0.5', 49132)>; Caller's stack:
76|Enrichment-IVRE  | Traceback (most recent call last):
76|Enrichment-IVRE  |   File "/home/azureuser/.local/lib/python3.8/site-packages/pika/adapters/utils/io_services_utils.py", line 1103, in _on_socket_writable
76|Enrichment-IVRE  |     self._produce()
76|Enrichment-IVRE  |   File "/home/azureuser/.local/lib/python3.8/site-packages/pika/adapters/utils/io_services_utils.py", line 819, in _produce
76|Enrichment-IVRE  |     num_bytes_sent = self._sigint_safe_send(self._sock,
76|Enrichment-IVRE  |   File "/home/azureuser/.local/lib/python3.8/site-packages/pika/adapters/utils/io_services_utils.py", line 79, in retry_sigint_wrap
76|Enrichment-IVRE  |     return func(*args, **kwargs)
76|Enrichment-IVRE  |   File "/home/azureuser/.local/lib/python3.8/site-packages/pika/adapters/utils/io_services_utils.py", line 861, in _sigint_safe_send
76|Enrichment-IVRE  |     return sock.send(data)
76|Enrichment-IVRE  | ConnectionResetError: [Errno 104] Connection reset by peer
76|Enrichment-IVRE  | Traceback (most recent call last):
76|Enrichment-IVRE  |   File "/home/azureuser/.local/lib/python3.8/site-packages/pika/adapters/utils/io_services_utils.py", line 1103, in _on_socket_writable
76|Enrichment-IVRE  |     self._produce()
76|Enrichment-IVRE  |   File "/home/azureuser/.local/lib/python3.8/site-packages/pika/adapters/utils/io_services_utils.py", line 819, in _produce
76|Enrichment-IVRE  |     num_bytes_sent = self._sigint_safe_send(self._sock,
76|Enrichment-IVRE  |   File "/home/azureuser/.local/lib/python3.8/site-packages/pika/adapters/utils/io_services_utils.py", line 79, in retry_sigint_wrap
76|Enrichment-IVRE  |     return func(*args, **kwargs)
76|Enrichment-IVRE  |   File "/home/azureuser/.local/lib/python3.8/site-packages/pika/adapters/utils/io_services_utils.py", line 861, in _sigint_safe_send
76|Enrichment-IVRE  |     return sock.send(data)
76|Enrichment-IVRE  | ConnectionResetError: [Errno 104] Connection reset by peer
76|Enrichment-IVRE  | ERROR:pika.adapters.base_connection:connection_lost: StreamLostError: ("Stream connection lost: ConnectionResetError(104, 'Connection reset by peer')",)
76|Enrichment-IVRE  | ERROR:pika.adapters.blocking_connection:Unexpected connection close detected: StreamLostError: ("Stream connection lost: ConnectionResetError(104, 'Connection reset by peer')",)
76|Enrichment-IVRE  | ERROR:root:Stream connection lost: ConnectionResetError(104, 'Connection reset by peer')

SamuelHassine commented 2 years ago

Hello @izm1chael,

Please check that the connector has access to the RabbitMQ.

Kind regards, Samuel

izm1chael commented 2 years ago

Hey @SamuelHassine,

Were would this option be?

As far as I was tracking, the RabbitMQ connection is handled through the production config?

I might have missed somthing

SamuelHassine commented 2 years ago

Hello @izm1chael,

You're right, the RabbitMQ connection parameters are passed to the connector through the API, so the configuration is on the platform side. The issue here is not related to the configuration but the fact that your connector seems to have connectivity issue to the RabbitMQ server / cluster.

Are you still experiencing this issue?

Kind regards, Samuel

Bella-chris commented 1 year ago

@SamuelHassine I have a similar issue and I don't know how can I fix this connection refused for the IVRE connector:

p-l- commented 2 months ago

Hi,

I was recently pointed to that issue, which I suppose is no longer current. For the record anyway, it seems from the screenshot that the connector cannot connect to the MongoDB instance used by IVRE. Most probably, you did not provide the connector with the IVRE instance's URL in its configuration, and the connector is trying to use the default (MongoDB on localhost).

OpenCTI-Platform / connectors