Closed mohanex closed 9 months ago
@helene-nguyen and @Megafredo can you help me for qualifying this ?
Hello,
To give you information about reproduction of the bug: 1 - clone my repo containing the connector (https://github.com/mohanex/OpenCTI_Connector) into a virtual machine containing docker that hosts an OpenCTI instance. 2 - Change Tokens and OpenCTI URl if necessary 3 - Change PROXY setting if necessary 4 - Run the connector in local using python3 or for example VScode Python Debugger.
If you still don't have enough information, tell me about things you want to know.
Thanks, I will check it and give you an update @mohanex
Hi @mohanex, I'm on it, I reproduce the bug with the following steps:
The RabbitMQ works fine and I also get the error message:
"Error sending bundle: [Errno 11001] getaddrinfo failed"
Means we need to resolve the host. I'll give you an update as soon as possible :)
Hello,
Thank you for the reproduction. If you refer to my first screenshot, I also get this "getaddrinfo failed" then followed with RabbitmQ errors like the "AMQP connection workflow failed"... Do you get the same ? And when you say RabbitMQ works fine did it get to connect with the connector ?
Hello, Yes, same error :
RabbitMQ works fine in the container, I can access the GUI:
Hi,
Yep I have exactly the same issue, RabbitMQ is also working fine for me with the other imported connectors from docker hub images. I will try to import my connector into docker and see if it works there. I'll keep you updated
Hello,
After some investigations and tests, I've found a way for you to run your OpenCTI in a container and the connector aside.
Running the connector locally and connecting it with an OpenCTI instance in a Docker container will give you this error because you'll need all the context (RabbitMQ, worker) too.
A workaround for this is to run your connector in a container and add a network, let say opencti-network
, which you can add to the end of your docker-compose.yml used to deploy the OpenCTI instance:
networks:
opencti-network:
driver: bridge
And you can add it for each services. Here's the docker-compose.yml used for the test:
version: '3'
services:
redis:
image: redis:7.2.4
volumes:
- redisdata1:/data
networks:
- opencti-network
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:8.12.0
volumes:
- esdata1:/usr/share/elasticsearch/data
environment:
# Comment-out the line below for a cluster of multiple nodes
- discovery.type=single-node
# Uncomment the line below below for a cluster of multiple nodes
# - cluster.name=docker-cluster
- xpack.ml.enabled=false
- xpack.security.enabled=false
- "ES_JAVA_OPTS=-Xms${ELASTIC_MEMORY_SIZE} -Xmx${ELASTIC_MEMORY_SIZE}"
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536
hard: 65536
ports:
- 9201:9200
- 9301:9300
networks:
- opencti-network
minio:
image: minio/minio:RELEASE.2024-01-16T16-07-38Z
volumes:
- s3data1:/data
ports:
- "9003:9000"
environment:
MINIO_ROOT_USER: ${MINIO_ROOT_USER}
MINIO_ROOT_PASSWORD: ${MINIO_ROOT_PASSWORD}
command: server /data
networks:
- opencti-network
rabbitmq:
image: rabbitmq:3.12-management
environment:
- RABBITMQ_DEFAULT_USER=${RABBITMQ_DEFAULT_USER}
- RABBITMQ_DEFAULT_PASS=${RABBITMQ_DEFAULT_PASS}
- RABBITMQ_NODENAME=rabbit01@localhost
volumes:
- amqpdata1:/var/lib/rabbitmq
ports:
- 5673:5672
- 15673:15672
networks:
- opencti-network
opencti:
image: opencti/platform:5.12.20
environment:
- NODE_OPTIONS=--max-old-space-size=8096
- APP__PORT=8081
- APP__BASE_URL=${OPENCTI_BASE_URL}
- APP__ADMIN__EMAIL=${OPENCTI_ADMIN_EMAIL}
- APP__ADMIN__PASSWORD=${OPENCTI_ADMIN_PASSWORD}
- APP__ADMIN__TOKEN=${OPENCTI_ADMIN_TOKEN}
- APP__APP_LOGS__LOGS_LEVEL=error
- REDIS__HOSTNAME=redis
- REDIS__PORT=6379
- ELASTICSEARCH__URL=http://elasticsearch:9200
- MINIO__ENDPOINT=minio
- MINIO__PORT=9000
- MINIO__USE_SSL=false
- MINIO__ACCESS_KEY=${MINIO_ROOT_USER}
- MINIO__SECRET_KEY=${MINIO_ROOT_PASSWORD}
- RABBITMQ__HOSTNAME=rabbitmq
- RABBITMQ__PORT=5672
- RABBITMQ__PORT_MANAGEMENT=15672
- RABBITMQ__MANAGEMENT_SSL=false
- RABBITMQ__USERNAME=${RABBITMQ_DEFAULT_USER}
- RABBITMQ__PASSWORD=${RABBITMQ_DEFAULT_PASS}
- SMTP__HOSTNAME=${SMTP_HOSTNAME}
- SMTP__PORT=25
- PROVIDERS__LOCAL__STRATEGY=LocalStrategy
ports:
- "8081:8081"
depends_on:
- redis
- elasticsearch
- minio
- rabbitmq
networks:
- opencti-network
worker:
image: opencti/worker:5.12.20
environment:
- OPENCTI_URL=http://opencti:8081
- OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
- WORKER_LOG_LEVEL=info
depends_on:
- opencti
deploy:
mode: replicated
replicas: 2
networks:
- opencti-network
networks:
opencti-network:
driver: bridge
volumes:
esdata1:
s3data1:
redisdata1:
amqpdata1:
Once containers are running, and OpenCTI works:
You can check networks with:
docker network ls
That way, you can find the name of the network where all containers are connected.
To connect your connector to this network, you have to add:
networks:
docker_opencti-network:
external: true
And add it to your service. Here your docker-compose.yml file:
version: '3'
services:
connector-blacklistip:
build: .
environment:
# Connector's definition parameters:
- CONNECTOR_TYPE=EXTERNAL_IMPORT
- CONNECTOR_NAME=${CONNECTOR_NAME}
- CONNECTOR_SCOPE=Incident
# Connector's generic execution parameters:
- OPENCTI_URL=http://opencti:8081
- OPENCTI_TOKEN=${OPENCTI_TOKEN}
- CONNECTOR_ID=${CONNECTOR_ID}
- CONNECTOR_CONFIDENCE_LEVEL=${CONNECTOR_CONFIDENCE_LEVEL} # From 0 (Unknown) to 100 (Fully trusted).
- CONNECTOR_LOG_LEVEL=${CONNECTOR_LOG_LEVEL}
- CONNECTOR_RUN_EVERY=${CONNECTOR_RUN_EVERY}
# Connector's custom execution parameters:
- EXTRA_PARAMETER=${EXTRA_PARAMETER}
# Add proxy parametrs below if you need them
# - HTTP_PROXY=ChangeMe
# - HTTPS_PROXY=ChangeMe
# - NO_PROXY=ChangeMe\
networks:
- docker_opencti-network
networks:
docker_opencti-network:
external: true
And when running your container, no more errors:
That way, you can test your code and your connector :)
I close the issue but we can re-open it if needed!
Hope that helps!
PS: About the name of the connector, could you rename it in DenyListIp as it is a more descriptive name?
Hello,
Thank you for the detailed response. I was about to post a comment to explain the same approach as you did. However I still don't see the feeds and data in my OpenCTI Dashboard, did you get to see them with your configuration ?
I don't see the feed at the moment, I think there may be a piece of code that needs to be changed for OpenCTI to ingest the data.
Here are some documentation that can help you:
Thank you very much, I appreciate your help. I will be looking for the feed problem and report it here whenever I manage to find the solution.
Environment
Description
I'm new to OpenCTI and to better understand how it works, I developed a very basic external-import connector. The problem appears when I debug the connector locally. see the image below : Data fetching and transformation to STIX2 bundle works perfectly, also on my OpenCTI platform I can see that my connector got successfully some information but didn't manage to send them to the plateforme. see the image below : Keep in mind that the connector is ran in the same machine where the Docker hosting OpenCTI is running.
Steps to produce the Bug