OpenCTI-Platform / opencti

Open Cyber Threat Intelligence Platform
https://opencti.io
Other
5.17k stars 819 forks source link

Upgrade Failure to 6.1.2 | Engine Unhandled Rejection #7038

Open The-Stuke opened 1 month ago

The-Stuke commented 1 month ago

Description

I upgraded from 6.0.7 to 6.1.2. After updating OpenCTI never launched. Tried to isolate troubleshooting this with just Elasticsearch, Redis, RabbitMQ, Minio, and main OpenCTI app running. The OpenCTI logs look to point to a few things "System dependencies check failed", "Search engine seems down", and "Engine unhandled rejection". I am unable to revert back to any older version of OpenCTI when attempting to.

I am able to wget http://elasticsearch:9200/_cluster/health and see the health status so it looks like OpenCTI should be able to connect to elasticsearch still and from Elasticsearch logs it looks to be running still.

{"cluster_name":"docker-cluster","status":"yellow","timed_out":false,"number_of_nodes":1,"number_of_data_nodes":1,"active_primary_shards":20,"active_shards":20,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":14,"delayed_unassigned_shards":0,"number_of_pending_tasks":0,"number_of_in_flight_fetch":0,"task_max_waiting_in_queue_millis":0,"active_shards_percent_as_number":58.82352941176471

Environment

  1. OS (where OpenCTI server runs): Docker
  2. OpenCTI version: 6.1.2
  3. OpenCTI client: NA
  4. Other environment details: NA

Reproducible Steps

Steps to create the smallest reproducible scenario:

  1. Run 6.0.7
  2. Upgrade to 6.1.2

Expected Output

OpenCTI to boot successfully.

Actual Output

opencti-opencti-1_logs.txt opencti-elasticsearch-1_logs.txt

Additional information

Docker compose of the main services.

version: '3'
services:
  redis:
    image: redis:7.2.4
    restart: always
    volumes:
      - redisdata:/data
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.13.4
    volumes:
      - esdata:/usr/share/elasticsearch/data
    environment:
      # Comment out the line below for single-node
      - discovery.type=single-node
      # Uncomment line below below for a cluster of multiple nodes
      # - cluster.name=docker-cluster
      - xpack.ml.enabled=false
      - xpack.security.enabled=false
      - thread_pool.search.queue_size=10000  # add queue_size (default 1000)
      - thread_pool.search.size=30  # add size (default 13)
      - logger.org.elasticsearch.discovery="ERROR"
      - "ES_JAVA_OPTS=-Xms${ELASTIC_MEMORY_SIZE} -Xmx${ELASTIC_MEMORY_SIZE}"
    restart: always
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
  minio:
    image: minio/minio:RELEASE.2024-01-16T16-07-38Z
    volumes:
      - s3data:/data
    ports:
      - "9000:9000"
    environment:
      MINIO_ROOT_USER: ${MINIO_ROOT_USER}
      MINIO_ROOT_PASSWORD: ${MINIO_ROOT_PASSWORD}    
    command: server /data
    restart: always
  rabbitmq:
    image: rabbitmq:3.13-management
    environment:
      - RABBITMQ_DEFAULT_USER=${RABBITMQ_DEFAULT_USER}
      - RABBITMQ_DEFAULT_PASS=${RABBITMQ_DEFAULT_PASS}
      - RABBITMQ_NODENAME=rabbit01@localhost
    volumes:
      - amqpdata:/var/lib/rabbitmq
    restart: always
  opencti:
    image: opencti/platform:6.1.2
    environment:
      - NODE_OPTIONS=--max-old-space-size=8096
      - APP__PORT=8080
      - APP__BASE_URL=${OPENCTI_BASE_URL}
      - APP__ADMIN__EMAIL=${OPENCTI_ADMIN_EMAIL}
      - APP__ADMIN__PASSWORD=${OPENCTI_ADMIN_PASSWORD}
      - APP__ADMIN__TOKEN=${OPENCTI_ADMIN_TOKEN}
      - APP__APP_LOGS__LOGS_LEVEL=error
      - APP__SESSION_TIMEOUT=28800000
      - APP__SESSION_IDLE_TIMEOUT=3600000
      - REDIS__HOSTNAME=redis
      - REDIS__PORT=6379
      - REDIS__TRIMMING=500000
      - ELASTICSEARCH__URL=http://elasticsearch:9200
      - MINIO__ENDPOINT=minio
      - MINIO__PORT=9000
      - MINIO__USE_SSL=false
      - MINIO__ACCESS_KEY=${MINIO_ROOT_USER}
      - MINIO__SECRET_KEY=${MINIO_ROOT_PASSWORD}
      - RABBITMQ__HOSTNAME=rabbitmq
      - RABBITMQ__PORT=5672
      - RABBITMQ__PORT_MANAGEMENT=15672
      - RABBITMQ__MANAGEMENT_SSL=false
      - RABBITMQ__USERNAME=${RABBITMQ_DEFAULT_USER}
      - RABBITMQ__PASSWORD=${RABBITMQ_DEFAULT_PASS}
      - SMTP__HOSTNAME=${SMTP_HOSTNAME}
      - SMTP__PORT=25
      - PROVIDERS__LOCAL__STRATEGY=LocalStrategy
    ports:
      - "443:8080"
    depends_on:
      - redis
      - elasticsearch
      - minio
      - rabbitmq
    restart: always

Screenshots (optional)

The-Stuke commented 1 month ago

Spent more time troubleshooting this from my end. Decided to try spinning up a brand new instance of 6.1.3 since it was just released and with roughly the same configuration in my compose file I got the same issue on a complete new installation. After adjusting some variable I found out I am able to get it to work when I am not using the built in SSL. With the following variables and having the external ports setup to 443.

  - APP__HTTPS_CERT__KEY=key_file.key
  - APP__HTTPS_CERT__CRT=cert_file.crt
  - APP__HTTPS_CERT__REJECT_UNAUTHORIZED=false

Tested this with the broken instance as well by removing the above variables and changing my reference port from 443 back to 80. In the past few versions of OpenCTI does anything break the SSL functionality and prevents the application from booting?

Below is the working docker-compose.yaml showing the commented out fields.

version: '3'
services:
  redis:
    image: redis:7.2.4
    restart: always
    volumes:
      - redisdata:/data
    logging:
      driver: "local"
      options:
        max-size: "10m"   # Limit log size to 10 MB
        max-file: "1"     # Retain up to 1 log files
        compress: "false"
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.13.4
    volumes:
      - esdata:/usr/share/elasticsearch/data
    environment:
      # Comment out the line below for single-node
      - discovery.type=single-node
      # Uncomment line below below for a cluster of multiple nodes
      # - cluster.name=docker-cluster
      - xpack.ml.enabled=false
      - xpack.security.enabled=false
      - thread_pool.search.queue_size=10000  # add queue_size (default 1000)
      - thread_pool.search.size=30  # add size (default 13)
      - logger.org.elasticsearch.discovery="ERROR"
      - "ES_JAVA_OPTS=-Xms${ELASTIC_MEMORY_SIZE} -Xmx${ELASTIC_MEMORY_SIZE}"
    restart: always
    logging:
      driver: "local"
      options:
        max-size: "10m"   # Limit log size to 10 MB
        max-file: "1"     # Retain up to 1 log files
        compress: "false"
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
  minio:
    image: minio/minio:RELEASE.2024-01-16T16-07-38Z
    volumes:
      - s3data:/data
    ports:
      - "9000:9000"
    environment:
      MINIO_ROOT_USER: ${MINIO_ROOT_USER}
      MINIO_ROOT_PASSWORD: ${MINIO_ROOT_PASSWORD}    
    command: server /data
    restart: always
    logging:
      driver: "local"
      options:
        max-size: "10m"   # Limit log size to 10 MB
        max-file: "1"     # Retain up to 1 log files
        compress: "false"
  rabbitmq:
    image: rabbitmq:3.13-management
    environment:
      - RABBITMQ_DEFAULT_USER=${RABBITMQ_DEFAULT_USER}
      - RABBITMQ_DEFAULT_PASS=${RABBITMQ_DEFAULT_PASS}
      - RABBITMQ_NODENAME=rabbit01@localhost
    volumes:
      - amqpdata:/var/lib/rabbitmq
    restart: always
    logging:
      driver: "local"
      options:
        max-size: "10m"   # Limit log size to 10 MB
        max-file: "1"     # Retain up to 1 log files
        compress: "false"
  opencti:
    image: opencti/platform:6.1.3
    environment:
      - NODE_OPTIONS=--max-old-space-size=8096
      - APP__PORT=8080
      - APP__BASE_URL=${OPENCTI_BASE_URL}
      - APP__ADMIN__EMAIL=${OPENCTI_ADMIN_EMAIL}
      - APP__ADMIN__PASSWORD=${OPENCTI_ADMIN_PASSWORD}
      - APP__ADMIN__TOKEN=${OPENCTI_ADMIN_TOKEN}
      - APP__APP_LOGS__LOGS_LEVEL=error
      #- APP__HTTPS_CERT__KEY=key_file.key
      #- APP__HTTPS_CERT__CRT=cert_file.crt
      #- APP__HTTPS_CERT__REJECT_UNAUTHORIZED=false
      - APP__SESSION_TIMEOUT=28800000
      - APP__SESSION_IDLE_TIMEOUT=3600000
      - REDIS__HOSTNAME=redis
      - REDIS__PORT=6379
      - REDIS__TRIMMING=500000
      - ELASTICSEARCH__URL=http://elasticsearch:9200
      - MINIO__ENDPOINT=minio
      - MINIO__PORT=9000
      - MINIO__USE_SSL=false
      - MINIO__ACCESS_KEY=${MINIO_ROOT_USER}
      - MINIO__SECRET_KEY=${MINIO_ROOT_PASSWORD}
      - RABBITMQ__HOSTNAME=rabbitmq
      - RABBITMQ__PORT=5672
      - RABBITMQ__PORT_MANAGEMENT=15672
      - RABBITMQ__MANAGEMENT_SSL=false
      - RABBITMQ__USERNAME=${RABBITMQ_DEFAULT_USER}
      - RABBITMQ__PASSWORD=${RABBITMQ_DEFAULT_PASS}
      - SMTP__HOSTNAME=${SMTP_HOSTNAME}
      - SMTP__PORT=25
      - PROVIDERS__SAML__CONFIG__CERT=${CONFIG_CERT}
      - PROVIDERS__SAML__CONFIG__LOGOUT_REMOTE=false
      - PROVIDERS__LOCAL__STRATEGY=LocalStrategy
    ports:
      - "80:8080"
    volumes:
      - /opt/opencti/ssl:/etc/ssl/certs
    depends_on:
      - redis
      - elasticsearch
      - minio
      - rabbitmq
    restart: always
    logging:
      driver: "local"
      options:
        max-size: "10m"   # Limit log size to 10 MB
        max-file: "1"     # Retain up to 1 log files
        compress: "false"
  worker:
    image: opencti/worker:6.1.3
    mem_limit: 2048m
    environment:
      - OPENCTI_URL=http://opencti:8080
      - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
      - WORKER_LOG_LEVEL=info
    volumes:
      - /opt/opencti/ssl:/etc/ssl/certs:ro
    deploy:
      mode: replicated
      replicas: 3
    restart: always
    logging:
      driver: "local"
      options:
        max-size: "10m"   # Limit log size to 10 MB
        max-file: "1"     # Retain up to 1 log files
        compress: "false"
    depends_on:
      - opencti
  connector-export-file-stix:
    image: opencti/connector-export-file-stix:6.1.3
    environment:
      - OPENCTI_URL=http://opencti:8080
      - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
      - CONNECTOR_ID=${CONNECTOR_EXPORT_FILE_STIX_ID} # Valid UUIDv4
      - CONNECTOR_TYPE=INTERNAL_EXPORT_FILE
      - CONNECTOR_NAME=ExportFileStix2
      - CONNECTOR_SCOPE=application/json
      - CONNECTOR_CONFIDENCE_LEVEL=15 # From 0 (Unknown) to 100 (Fully trusted)
      - CONNECTOR_LOG_LEVEL=info
    restart: always
    logging:
      driver: "local"
      options:
        max-size: "10m"   # Limit log size to 10 MB
        max-file: "1"     # Retain up to 1 log files
        compress: "false"
    depends_on:
      - opencti
  connector-export-file-csv:
    image: opencti/connector-export-file-csv:6.1.3
    environment:
      - OPENCTI_URL=http://opencti:8080
      - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
      - CONNECTOR_ID=${CONNECTOR_EXPORT_FILE_CSV_ID} # Valid UUIDv4
      - CONNECTOR_TYPE=INTERNAL_EXPORT_FILE
      - CONNECTOR_NAME=ExportFileCsv
      - CONNECTOR_SCOPE=text/csv
      - CONNECTOR_CONFIDENCE_LEVEL=15 # From 0 (Unknown) to 100 (Fully trusted)
      - CONNECTOR_LOG_LEVEL=info
    restart: always
    logging:
      driver: "local"
      options:
        max-size: "10m"   # Limit log size to 10 MB
        max-file: "1"     # Retain up to 1 log files
        compress: "false"
    depends_on:
      - opencti
  connector-export-file-txt:
    image: opencti/connector-export-file-txt:6.1.3
    environment:
      - OPENCTI_URL=http://opencti:8080
      - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
      - CONNECTOR_ID=${CONNECTOR_EXPORT_FILE_TXT_ID} # Valid UUIDv4
      - CONNECTOR_TYPE=INTERNAL_EXPORT_FILE
      - CONNECTOR_NAME=ExportFileTxt
      - CONNECTOR_SCOPE=text/plain
      - CONNECTOR_CONFIDENCE_LEVEL=15 # From 0 (Unknown) to 100 (Fully trusted)
      - CONNECTOR_LOG_LEVEL=info
    restart: always
    logging:
      driver: "local"
      options:
        max-size: "10m"   # Limit log size to 10 MB
        max-file: "1"     # Retain up to 1 log files
        compress: "false"
    depends_on:
      - opencti
  connector-import-file-stix:
    image: opencti/connector-import-file-stix:6.1.3
    environment:
      - OPENCTI_URL=http://opencti:8080
      - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
      - CONNECTOR_ID=${CONNECTOR_IMPORT_FILE_STIX_ID} # Valid UUIDv4
      - CONNECTOR_TYPE=INTERNAL_IMPORT_FILE
      - CONNECTOR_NAME=ImportFileStix
      - CONNECTOR_VALIDATE_BEFORE_IMPORT=true # Validate any bundle before import
      - CONNECTOR_SCOPE=application/json,text/xml
      - CONNECTOR_AUTO=true # Enable/disable auto-import of file
      - CONNECTOR_CONFIDENCE_LEVEL=15 # From 0 (Unknown) to 100 (Fully trusted)
      - CONNECTOR_LOG_LEVEL=info
    restart: always
    logging:
      driver: "local"
      options:
        max-size: "10m"   # Limit log size to 10 MB
        max-file: "1"     # Retain up to 1 log files
        compress: "false"
    depends_on:
      - opencti
volumes:
  esdata:
  s3data:
  redisdata:
  amqpdata:
richard-julien commented 1 month ago

Can you setup the platform logs in INFO and grab the logs?

The-Stuke commented 1 month ago

With the SSL enabled below are the platform info logs.

2024-05-21T08:46:10.006351434-07:00 {"category":"APP","cause":{"_error":{},"_showLocations":false,"_showPath":false,"data":{"cause":{"meta":{"body":null,"headers":null,"meta":{"aborted":false,"attempts":3,"connection":{"_openRequests":0,"deadCount":4,"headers":{},"id":"http://elasticsearch:9200/","resurrectTimeout":1716306849995,"roles":{"data":true,"ingest":true},"status":"dead","url":"http://elasticsearch:9200/"},"context":null,"name":"opensearch-js","request":{"id":1,"options":{},"params":{"body":null,"headers":{"user-agent":"opensearch-js/2.7.0 (linux 4.18.0-513.9.1.el8_9.x86_64-x64; Node.js v20.13.1)"},"method":"GET","path":"/","querystring":"","timeout":30000}}},"statusCode":null},"name":"ConnectionError"},"genre":"TECHNICAL","http_status":500},"internalData":{},"name":"CONFIGURATION_ERROR","time_thrown":"2024-05-21T15:46:09.996Z"},"level":"error","message":"[OPENCTI] System dependencies check failed","source":"backend","timestamp":"2024-05-21T15:46:09.996Z","version":"6.1.3"}
2024-05-21T08:46:10.006395871-07:00 {"category":"APP","level":"error","message":"Search engine seems down","source":"backend","timestamp":"2024-05-21T15:46:09.998Z","version":"6.1.3"}
2024-05-21T08:46:23.489480216-07:00 {"category":"APP","cause":{"_error":{},"_showLocations":false,"_showPath":false,"data":{"cause":{"meta":{"body":null,"headers":null,"meta":{"aborted":false,"attempts":3,"connection":{"_openRequests":0,"deadCount":4,"headers":{},"id":"http://elasticsearch:9200/","resurrectTimeout":1716306863483,"roles":{"data":true,"ingest":true},"status":"dead","url":"http://elasticsearch:9200/"},"context":null,"name":"opensearch-js","request":{"id":1,"options":{},"params":{"body":null,"headers":{"user-agent":"opensearch-js/2.7.0 (linux 4.18.0-513.9.1.el8_9.x86_64-x64; Node.js v20.13.1)"},"method":"GET","path":"/","querystring":"","timeout":30000}}},"statusCode":null},"name":"ConnectionError"},"genre":"TECHNICAL","http_status":500},"internalData":{},"name":"CONFIGURATION_ERROR","time_thrown":"2024-05-21T15:46:23.484Z"},"level":"error","message":"[OPENCTI] System dependencies check failed","source":"backend","timestamp":"2024-05-21T15:46:23.484Z","version":"6.1.3"}
2024-05-21T08:46:23.489517612-07:00 {"category":"APP","level":"error","message":"Search engine seems down","source":"backend","timestamp":"2024-05-21T15:46:23.486Z","version":"6.1.3"}
2024-05-21T08:46:47.671620494-07:00 {"category":"APP","level":"error","message":"Engine unhandled rejection","source":"backend","timestamp":"2024-05-21T15:46:47.668Z","version":"6.1.3"}
richard-julien commented 1 month ago

I need logs from platform start. Can you attached the full INFO logs? Thanks

The-Stuke commented 1 month ago

Sorry about that I think my logs were limited last time. This should be the full INFO level logs.

2024-05-21T14:13:03.510852245-05:00 {"category":"APP","environment":"production","level":"info","message":"[OPENCTI] Starting platform","source":"backend","timestamp":"2024-05-21T19:13:03.503Z","version":"6.1.3"}
2024-05-21T14:13:03.516473795-05:00 {"category":"APP","level":"info","message":"[OPENCTI] Checking dependencies statuses","source":"backend","timestamp":"2024-05-21T19:13:03.510Z","version":"6.1.3"}
2024-05-21T14:13:03.543571371-05:00 {"category":"APP","level":"info","message":"[SEARCH] Engine client not specified, trying to discover it with opensearch client","source":"backend","timestamp":"2024-05-21T19:13:03.535Z","version":"6.1.3"}
2024-05-21T14:13:03.648924808-05:00 {"category":"APP","level":"info","message":"[SEARCH] Engine detected to elk","source":"backend","timestamp":"2024-05-21T19:13:03.639Z","version":"6.1.3"}
2024-05-21T14:13:03.726665322-05:00 {"category":"APP","level":"info","message":"[SEARCH] elk (8.13.4) client selected / runtime sorting enabled / attachment processor enabled","source":"backend","timestamp":"2024-05-21T19:13:03.723Z","version":"6.1.3"}
2024-05-21T14:13:03.727085402-05:00 {"category":"APP","level":"info","message":"[CHECK] Search engine is alive","source":"backend","timestamp":"2024-05-21T19:13:03.726Z","version":"6.1.3"}
2024-05-21T14:13:03.857481212-05:00 {"category":"APP","level":"info","message":"[CHECK] File engine is alive","source":"backend","timestamp":"2024-05-21T19:13:03.853Z","version":"6.1.3"}
2024-05-21T14:13:03.951164044-05:00 {"category":"APP","level":"info","message":"[CHECK] RabbitMQ engine is alive","source":"backend","timestamp":"2024-05-21T19:13:03.949Z","version":"6.1.3"}
2024-05-21T14:13:03.977471885-05:00 {"category":"APP","level":"info","message":"[REDIS] Redis 'base' client ready","source":"backend","timestamp":"2024-05-21T19:13:03.976Z","version":"6.1.3"}
2024-05-21T14:13:03.978923080-05:00 {"category":"APP","level":"info","message":"[REDIS] Clients initialized in single mode","source":"backend","timestamp":"2024-05-21T19:13:03.977Z","version":"6.1.3"}
2024-05-21T14:13:03.979603864-05:00 {"category":"APP","level":"info","message":"[CHECK] Redis engine is alive","source":"backend","timestamp":"2024-05-21T19:13:03.978Z","version":"6.1.3"}
2024-05-21T14:13:03.989275883-05:00 {"category":"APP","level":"warn","message":"SMTP seems down, email notification will may not work","source":"backend","timestamp":"2024-05-21T19:13:03.988Z","version":"6.1.3"}
2024-05-21T14:13:04.082560232-05:00 {"category":"APP","level":"info","message":"[CHECK] Python3 is available","source":"backend","timestamp":"2024-05-21T19:13:04.074Z","version":"6.1.3"}
2024-05-21T14:13:04.087209349-05:00 {"category":"APP","level":"info","message":"[REDIS] Redis 'subscriber' client ready","source":"backend","timestamp":"2024-05-21T19:13:04.086Z","version":"6.1.3"}
2024-05-21T14:13:04.090684998-05:00 {"category":"APP","level":"info","message":"[OPENCTI-MODULE] Cache manager pub sub listener initialized","source":"backend","timestamp":"2024-05-21T19:13:04.089Z","version":"6.1.3"}
2024-05-21T14:13:04.102249836-05:00 {"category":"APP","level":"info","message":"[REDIS] Redis 'lock' client ready","source":"backend","timestamp":"2024-05-21T19:13:04.100Z","version":"6.1.3"}
2024-05-21T14:13:04.105661201-05:00 {"category":"APP","level":"info","message":"[INIT] Starting platform initialization","source":"backend","timestamp":"2024-05-21T19:13:04.103Z","version":"6.1.3"}
2024-05-21T14:13:04.199470921-05:00 {"category":"APP","level":"info","message":"[INIT] Existing platform detected, initialization...","source":"backend","timestamp":"2024-05-21T19:13:04.194Z","version":"6.1.3"}
2024-05-21T14:13:10.601168984-05:00 {"category":"APP","level":"info","message":"[INIT] admin user initialized","source":"backend","timestamp":"2024-05-21T19:13:10.599Z","version":"6.1.3"}
2024-05-21T14:13:10.678433552-05:00 {"category":"APP","level":"info","message":"[MIGRATION] Read 0 migrations from the database","source":"backend","timestamp":"2024-05-21T19:13:10.663Z","version":"6.1.3"}
2024-05-21T14:13:10.678463344-05:00 {"category":"APP","level":"info","message":"[MIGRATION] Platform already up to date, nothing to migrate","source":"backend","timestamp":"2024-05-21T19:13:10.672Z","version":"6.1.3"}
2024-05-21T14:13:10.678491757-05:00 {"category":"APP","level":"info","message":"[MIGRATION] Migration process completed","source":"backend","timestamp":"2024-05-21T19:13:10.673Z","version":"6.1.3"}
2024-05-21T14:13:10.751621550-05:00 {"category":"APP","level":"info","message":"[MIGRATION] Platform version updated to 6.1.3","source":"backend","timestamp":"2024-05-21T19:13:10.747Z","version":"6.1.3"}
2024-05-21T14:13:10.871554027-05:00 {"category":"APP","level":"info","message":"[INIT] Platform initialization done","source":"backend","timestamp":"2024-05-21T19:13:10.867Z","version":"6.1.3"}
2024-05-21T14:13:14.806120798-05:00 {"category":"APP","level":"error","message":"Engine unhandled rejection","source":"backend","timestamp":"2024-05-21T19:13:14.802Z","version":"6.1.3"}

As a side note I was able to just remove SSL entirely and put OpenCTI behind a nginx server which should fit my needs.

Kedae commented 1 month ago

Seems like an error with the httpServer.start, quite strange

MaxwellDPS commented 3 weeks ago

Seeing this error still on 6.1.6, seems to appear in tandem with "UNKNOWN_ERROR: Http call interceptor fail"

No further logs surrounding this event (from startup)

{
    "category": "APP",
    "errors": [
        {
            "attributes": {
                "genre": "TECHNICAL",
                "http_status": 500,
                "promise": {},
                "reason": {
                    "code": "ERR_HTTP_HEADERS_SENT"
                }
            },
            "message": "Engine unhandled rejection",
            "name": "UNKNOWN_ERROR",
            "stack": "UNKNOWN_ERROR: Engine unhandled rejection\n    at error (/opt/opencti/build/src/config/errors.js:8:10)\n    at UnknownError (/opt/opencti/build/src/config/errors.js:82:47)\n    at process.<anonymous> (/opt/opencti/build/src/boot.js:60:16)\n    at process.emit (node:events:519:28)\n    at process.emit (/opt/opencti/build/node_modules/source-map-support/source-map-support.js:516:21)\n    at emit (node:internal/process/promises:150:20)\n    at processPromiseRejections (node:internal/process/promises:284:27)\n    at processTicksAndRejections (node:internal/process/task_queues:96:32)"
        }
    ],
    "level": "error",
    "message": "Engine unhandled rejection",
    "source": "backend",
    "timestamp": "2024-06-17T20:21:27.867Z",
    "version": "6.1.6"
}

also seeing lots of these

{"category":"APP","level":"info","message":"[STREAM] Shutdown stream processor for Activity manager","source":"backend","timestamp":"2024-06-17T20:48:00.718Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Stream processor current promise terminated","source":"backend","timestamp":"2024-06-17T20:48:00.718Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Shutdown stream processor for Activity manager","source":"backend","timestamp":"2024-06-17T20:48:10.864Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Stream processor current promise terminated","source":"backend","timestamp":"2024-06-17T20:48:10.864Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Shutdown stream processor for Activity manager","source":"backend","timestamp":"2024-06-17T20:48:19.002Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Stream processor current promise terminated","source":"backend","timestamp":"2024-06-17T20:48:19.002Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Shutdown stream processor for Activity manager","source":"backend","timestamp":"2024-06-17T20:48:20.985Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Stream processor current promise terminated","source":"backend","timestamp":"2024-06-17T20:48:20.985Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Shutdown stream processor for Activity manager","source":"backend","timestamp":"2024-06-17T20:48:29.079Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Stream processor current promise terminated","source":"backend","timestamp":"2024-06-17T20:48:29.079Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Shutdown stream processor for Activity manager","source":"backend","timestamp":"2024-06-17T20:48:31.000Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Stream processor current promise terminated","source":"backend","timestamp":"2024-06-17T20:48:31.000Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Shutdown stream processor for Activity manager","source":"backend","timestamp":"2024-06-17T20:48:39.083Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Stream processor current promise terminated","source":"backend","timestamp":"2024-06-17T20:48:39.083Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Shutdown stream processor for Activity manager","source":"backend","timestamp":"2024-06-17T20:48:41.198Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Stream processor current promise terminated","source":"backend","timestamp":"2024-06-17T20:48:41.198Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Shutdown stream processor for Activity manager","source":"backend","timestamp":"2024-06-17T20:48:49.138Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Stream processor current promise terminated","source":"backend","timestamp":"2024-06-17T20:48:49.138Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Shutdown stream processor for Activity manager","source":"backend","timestamp":"2024-06-17T20:48:51.240Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Stream processor current promise terminated","source":"backend","timestamp":"2024-06-17T20:48:51.240Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Shutdown stream processor for Activity manager","source":"backend","timestamp":"2024-06-17T20:48:59.142Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Stream processor current promise terminated","source":"backend","timestamp":"2024-06-17T20:48:59.142Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Shutdown stream processor for Activity manager","source":"backend","timestamp":"2024-06-17T20:49:01.242Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Stream processor current promise terminated","source":"backend","timestamp":"2024-06-17T20:49:01.242Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Shutdown stream processor for Activity manager","source":"backend","timestamp":"2024-06-17T20:49:09.143Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Stream processor current promise terminated","source":"backend","timestamp":"2024-06-17T20:49:09.143Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Shutdown stream processor for Activity manager","source":"backend","timestamp":"2024-06-17T20:49:11.245Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Stream processor current promise terminated","source":"backend","timestamp":"2024-06-17T20:49:11.245Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Shutdown stream processor for Activity manager","source":"backend","timestamp":"2024-06-17T20:49:19.147Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Stream processor current promise terminated","source":"backend","timestamp":"2024-06-17T20:49:19.147Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Shutdown stream processor for Activity manager","source":"backend","timestamp":"2024-06-17T20:49:21.258Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Stream processor current promise terminated","source":"backend","timestamp":"2024-06-17T20:49:21.258Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Shutdown stream processor for Activity manager","source":"backend","timestamp":"2024-06-17T20:49:29.228Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Stream processor current promise terminated","source":"backend","timestamp":"2024-06-17T20:49:29.228Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Shutdown stream processor for Activity manager","source":"backend","timestamp":"2024-06-17T20:49:31.259Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Stream processor current promise terminated","source":"backend","timestamp":"2024-06-17T20:49:31.259Z","version":"6.1.6"}
(node:7) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 close listeners added to [Socket]. Use emitter.setMaxListeners() to increase limit
(Use `node --trace-warnings ...` to show where the warning was created)
{"category":"APP","level":"info","message":"[TELEMETRY] File exporter activated","source":"backend","timestamp":"2024-06-17T20:49:34.687Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[TELEMETRY] Otlp exporter activated","source":"backend","timestamp":"2024-06-17T20:49:35.459Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[OPENCTI-MODULE] Running Telemetry manager infinite cron handler","source":"backend","timestamp":"2024-06-17T20:49:35.567Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Shutdown stream processor for Activity manager","source":"backend","timestamp":"2024-06-17T20:49:39.717Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Stream processor current promise terminated","source":"backend","timestamp":"2024-06-17T20:49:39.717Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Shutdown stream processor for Activity manager","source":"backend","timestamp":"2024-06-17T20:49:41.263Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Stream processor current promise terminated","source":"backend","timestamp":"2024-06-17T20:49:41.263Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Shutdown stream processor for Activity manager","source":"backend","timestamp":"2024-06-17T20:49:49.736Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Stream processor current promise terminated","source":"backend","timestamp":"2024-06-17T20:49:49.736Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Shutdown stream processor for Activity manager","source":"backend","timestamp":"2024-06-17T20:49:51.306Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Stream processor current promise terminated","source":"backend","timestamp":"2024-06-17T20:49:51.306Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Shutdown stream processor for Activity manager","source":"backend","timestamp":"2024-06-17T20:49:59.922Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Stream processor current promise terminated","source":"backend","timestamp":"2024-06-17T20:49:59.922Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Shutdown stream processor for Activity manager","source":"backend","timestamp":"2024-06-17T20:50:01.308Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Stream processor current promise terminated","source":"backend","timestamp":"2024-06-17T20:50:01.308Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[OPENCTI-MODULE] Running activity manager","source":"backend","timestamp":"2024-06-17T20:50:10.012Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Starting stream processor at 0-0 for Activity manager","source":"backend","timestamp":"2024-06-17T20:50:10.013Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[REDIS] Redis 'Activity manager' client ready","source":"backend","timestamp":"2024-06-17T20:50:10.029Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Shutdown stream processor for Activity manager","source":"backend","timestamp":"2024-06-17T20:50:11.746Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Stream processor current promise terminated","source":"backend","timestamp":"2024-06-17T20:50:11.746Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Shutdown stream processor for Activity manager","source":"backend","timestamp":"2024-06-17T20:50:21.790Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Stream processor current promise terminated","source":"backend","timestamp":"2024-06-17T20:50:21.790Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Shutdown stream processor for Activity manager","source":"backend","timestamp":"2024-06-17T20:50:31.818Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Stream processor current promise terminated","source":"backend","timestamp":"2024-06-17T20:50:31.818Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Shutdown stream processor for Activity manager","source":"backend","timestamp":"2024-06-17T20:50:41.821Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Stream processor current promise terminated","source":"backend","timestamp":"2024-06-17T20:50:41.821Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Shutdown stream processor for Activity manager","source":"backend","timestamp":"2024-06-17T20:50:51.824Z","version":"6.1.6"}
{"category":"APP","level":"info","message":"[STREAM] Stream processor current promise terminated","source":"backend","timestamp":"2024-06-17T20:50:51.824Z","version":"6.1.6"}