Muliple OpenCTI platforms running at once

PeeBee66 commented 4 weeks ago

Description

I'm encountering a recurring error after updating OpenCTI from version 6.2.7 to 6.2.13. This issue appears when running multiple OpenCTI platforms. The error occurs regardless of whether I'm using HTTP or HTTPS, and persists even after clearing Redis and RabbitMQ queues.

This has not effected the running of Opencti but it is filling up my logs

Environment

OS (where OpenCTI server runs): Ubuntu 20.04 (dev). RHEL 8.9 (prod)
OpenCTI version: 6.2.13 (upgraded from 6.2.7)
Docker containers
Other environment details:
- Multiple OpenCTI platforms deployed for multi-network access.
- LDAP integration from two separate networks. (testing in dev has no LDAP and same issues)
- No difference in using HTTP and HTTPS
- Reproducible Steps

Steps to create the smallest reproducible scenario:

Upgrade OpenCTI from version 6.2.7 to 6.2.13. (Unable to confirm if this error was present previously)
Deploy a additional OpenCTI platforms with the same backend on a separate server
Observe the following error: {"message":"Engine unhandled rejection","name":"GraphQLError","stack":"GraphQLError: Engine unhandled rejection..."}.

Expected Output

Expected the system to function normally across multiple OpenCTI platform frontends without encountering any errors. (like it has been for the last 6 months)

Actual Output

The following error occurs: {"message":"Engine unhandled rejection","name":"GraphQLError","stack":"GraphQLError: Engine unhandled rejection..."}. This error is consistent across both HTTP and HTTPS, and only appears when two or more OpenCTI instances are active.

[{"message":"Engine unhandled rejection","name":"GraphQLError","stack":"GraphQLError: Engine unhandled rejection\n at error (/opt/opencti/build/src/config/errors.js:7:10)\n at UnknownError (/opt/opencti/build/src/config/errors.js:81:47)\n at Object._logWithError (/opt/opencti/build/src/config/conf.js:235:17)\n at Object.error (/opt/opencti/build/src/config/conf.js:244:48)\n at process.<anonymous> (/opt/opencti/build/src/boot.js:60:10)\n at process.emit (node:events:519:28)\n at process.N1r.process.emit (/opt/opencti/build/node_modules/source-map-support/source-map-support.js:516:21)\n at emitUnhandledRejection (node:internal/process/promises:250:13)\n at throwUnhandledRejectionsMode (node:internal/process/promises:385:19)\n at processPromiseRejections (node:internal/process/promises:470:17)\n at processTicksAndRejections (node:internal/process/task_queues:96:32)"}]
level=

Additional information

The issue seems to be related to running multiple OpenCTI instances with similar configurations. Clearing Redis and RabbitMQ queues did not resolve the issue. The setup includes multiple OpenCTI servers, an ELK stack, Redis, RabbitMQ, and MinIO. Here is the config for my additional OpenCTI Platform frontend (basicly a copy paste from the core OpenCTI Platform

opencti:
  image: opencti/platform:${OPENCTI_VERSION}
  environment:
    - NODE_OPTIONS=--max-old-space-size=8096
    - APP__PORT=4001
    - APP__BASE_URL=${OPENCTI_BASE_URL}
    - APP__ADMIN__EMAIL=${OPENCTI_ADMIN_EMAIL}
    - APP__ADMIN__PASSWORD=${OPENCTI_ADMIN_PASSWORD}
    - APP__ADMIN__TOKEN=${OPENCTI_ADMIN_TOKEN}
    - APP__APP_LOGS__LOGS_LEVEL=error
    - APP__SESSION_TIMEOUT=28800000
    - REDIS__HOSTNAMEx.x.x.x
    - REDIS__PORT=6379
    - REDIS__TRIMMING=1000000
    - ELASTICSEARCH__URL=["http://x.x.x.x:9200/","http://x.x.x.x:9200/","http://x.x.x.x:9200/"]
    - MINIO__ENDPOINT=x.x.x.x
    - MINIO__PORT=9000
    - MINIO__USE_SSL=false
    - MINIO__ACCESS_KEY=${MINIO_ROOT_USER}
    - MINIO__SECRET_KEY=${MINIO_ROOT_PASSWORD}
    - RABBITMQ__HOSTNAME=x.x.x.x
    - RABBITMQ__PORT=5672
    - RABBITMQ__PORT_MANAGEMENT=15672
    - RABBITMQ__MANAGEMENT_SSL=false
    - RABBITMQ__USERNAME=${RABBITMQ_DEFAULT_USER}
    - RABBITMQ__PASSWORD=${RABBITMQ_DEFAULT_PASS}
    - SMTP__HOSTNAME=${SMTP_HOSTNAME}
    - SMTP__PORT=25
    - PROVIDERS__LOCAL__STRATEGY=LocalStrategy
  ports:
    - "4001:4001"
  restart: always

Screenshots (optional)

nino-filigran commented 3 weeks ago

Hey @PeeBee66 is it an error that pops up periodically? Is it when he interacts with the platform? I've shared your bug initially but it's a bit hard to reproduce.

nino-filigran commented 3 weeks ago

Could you please try to upgrade .14? The .13 is malfunctioning

PeeBee66 commented 2 weeks ago

Updated to 6.2.18 - Seems to have resolved the issue

OpenCTI-Platform / opencti