OpenCTI-Platform / opencti

Open Cyber Threat Intelligence Platform
https://opencti.io
Other
6.32k stars 932 forks source link

ElasticSearch seems down #1412

Closed Sue1990 closed 3 years ago

Sue1990 commented 3 years ago

I followed this URL (https://www.notion.so/Using-Docker-03d5c0592b9d4547800cc9f4ff7be2b8#e4398cfb3aea431d94843013baca85cb ) to install OpenCTI , but it seem to not work . when I connect to http://localhost:8080, it shows "Unable to connect".

then I type command ( sudo docker logs --details docker_opencti_1 ) found error log . like below down:

{"error":{"name":"ConfigurationError","_error":{},"_showLocations":false,"_showPath":false,"time_thrown":"2021-06-14T13:33:37.172Z","data":{"reason":"ElasticSearch seems down","http_status":500,"category":"technical"},"internalData":{}},"category":"APP","version":"4.5.4","level":"error","message":"[OPENCTI] Platform initialization fail","timestamp":"2021-06-14T13:33:37.174Z"} {"error":{"name":"ConfigurationError","_error":{},"_showLocations":false,"_showPath":false,"time_thrown":"2021-06-14T13:36:23.321Z","data":{"reason":"ElasticSearch seems down","http_status":500,"category":"technical"},"internalData":{}},"category":"APP","version":"4.5.4","level":"error","message":"[OPENCTI] Platform initialization fail","timestamp":"2021-06-14T13:36:23.321Z"}

I lost which step ? or miss install other Dependencies?

docker service ls

截圖 2021-06-14 下午9 55 37

docker container ls

截圖 2021-06-14 下午10 00 28

docker-compose.yml configuration

version: '3' services: redis: image: redis:6.2.3 restart: always volumes:

  • redisdata:/data elasticsearch: image: docker.elastic.co/elasticsearch/elasticsearch:7.12.1 volumes:
  • esdata:/usr/share/elasticsearch/data environment:
  • discovery.type=single-node
  • xpack.ml.enabled=false restart: always ulimits: memlock: soft: -1 hard: -1 nofile: soft: 65536 hard: 65536 minio: image: minio/minio:RELEASE.2021-04-22T15-44-28Z volumes:
  • s3data:/data ports:
  • "9000:9000" environment: MINIO_ACCESS_KEY: minio MINIO_SECRET_KEY: minio command: server /data healthcheck: test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"] interval: 30s timeout: 20s retries: 3
    restart: always rabbitmq: image: rabbitmq:3.8-management environment:
  • RABBITMQ_DEFAULT_USER=admin
  • RABBITMQ_DEFAULT_PASS=admin volumes:
  • amqpdata:/var/lib/rabbitmq restart: always opencti: image: opencti/platform:4.5.4 environment:
  • NODE_OPTIONS=--max-old-space-size=8096
  • APP__PORT=8080
  • APPADMINEMAIL=xxxx@xxx.xxx.xxx
  • APPADMINPASSWORD=testest
  • APPADMINTOKEN=87f74106-d8d7-4262-aaeb-25851c61bc0b
  • APP__APP_LOGS__LOGS_LEVEL=error
  • REDIS__HOSTNAME=redis
  • REDIS__PORT=6379
  • ELASTICSEARCH__URL=http://elasticsearch:9200
  • MINIO__ENDPOINT=minio
  • MINIO__PORT=9000
  • MINIO__USE_SSL=false
  • MINIO__ACCESS_KEY=minio
  • MINIO__SECRET_KEY=minio
  • RABBITMQ__HOSTNAME=rabbitmq
  • RABBITMQ__PORT=5672
  • RABBITMQ__PORT_MANAGEMENT=15672
  • RABBITMQ__MANAGEMENT_SSL=false
  • RABBITMQ__USERNAME=admin
  • RABBITMQ__PASSWORD=admin
  • PROVIDERSLOCALSTRATEGY=LocalStrategy ports:
  • "8080:8080"
    depends_on:
  • redis
  • elasticsearch
  • minio
  • rabbitmq restart: always worker: image: opencti/worker:4.5.4 environment:
  • OPENCTI_URL=http://opencti:8080
  • OPENCTI_TOKEN=87f74106-d8d7-4262-aaeb-25851c61bc0b
  • WORKER_LOG_LEVEL=info depends_on:
  • opencti deploy: mode: replicated replicas: 3 restart: always connector-history: image: opencti/connector-history:4.5.4 environment:
  • OPENCTI_URL=http://opencti:8080
  • OPENCTI_TOKEN=87f74106-d8d7-4262-aaeb-25851c61bc0b
  • CONNECTOR_ID=87f74106-d8d7-4262-aaeb-25851c61bc0b # Valid UUIDv4
  • CONNECTOR_TYPE=STREAM
  • CONNECTOR_NAME=History
  • CONNECTOR_SCOPE=history
  • CONNECTOR_CONFIDENCE_LEVEL=15 # From 0 (Unknown) to 100 (Fully trusted)
  • CONNECTOR_LOG_LEVEL=info restart: always connector-export-file-stix: image: opencti/connector-export-file-stix:4.5.4 environment:
  • OPENCTI_URL=http://opencti:8080
  • OPENCTI_TOKEN=87f74106-d8d7-4262-aaeb-25851c61bc0b
  • CONNECTOR_ID=87f74106-d8d7-4262-aaeb-25851c61bc0b # Valid UUIDv4
  • CONNECTOR_TYPE=INTERNAL_EXPORT_FILE
  • CONNECTOR_NAME=ExportFileStix2
  • CONNECTOR_SCOPE=application/json
  • CONNECTOR_CONFIDENCE_LEVEL=15 # From 0 (Unknown) to 100 (Fully trusted)
  • CONNECTOR_LOG_LEVEL=info restart: always connector-export-file-csv: image: opencti/connector-export-file-csv:4.5.4 environment:
  • OPENCTI_URL=http://opencti:8080
  • OPENCTI_TOKEN=87f74106-d8d7-4262-aaeb-25851c61bc0b
  • CONNECTOR_ID=87f74106-d8d7-4262-aaeb-25851c61bc0b # Valid UUIDv4
  • CONNECTOR_TYPE=INTERNAL_EXPORT_FILE
  • CONNECTOR_NAME=ExportFileCsv
  • CONNECTOR_SCOPE=text/csv
  • CONNECTOR_CONFIDENCE_LEVEL=15 # From 0 (Unknown) to 100 (Fully trusted)
  • CONNECTOR_LOG_LEVEL=info restart: always connector-import-file-stix: image: opencti/connector-import-file-stix:4.5.4 environment:
  • OPENCTI_URL=http://opencti:8080
  • OPENCTI_TOKEN=87f74106-d8d7-4262-aaeb-25851c61bc0b
  • CONNECTOR_ID=87f74106-d8d7-4262-aaeb-25851c61bc0b # Valid UUIDv4
  • CONNECTOR_TYPE=INTERNAL_IMPORT_FILE
  • CONNECTOR_NAME=ImportFileStix
  • CONNECTOR_SCOPE=application/json,text/xml
  • CONNECTOR_AUTO=false # Enable/disable auto-import of file
  • CONNECTOR_CONFIDENCE_LEVEL=15 # From 0 (Unknown) to 100 (Fully trusted)
  • CONNECTOR_LOG_LEVEL=info restart: always connector-import-report: image: opencti/connector-import-report:4.5.4 environment:
  • OPENCTI_URL=http://opencti:8080
  • OPENCTI_TOKEN=87f74106-d8d7-4262-aaeb-25851c61bc0b
  • CONNECTOR_ID=87f74106-d8d7-4262-aaeb-25851c61bc0b # Valid UUIDv4
  • CONNECTOR_TYPE=INTERNAL_IMPORT_FILE
  • CONNECTOR_NAME=ImportReport
  • CONNECTOR_SCOPE=application/pdf,text/plain
  • CONNECTOR_AUTO=false # Enable/disable auto-import of file
  • CONNECTOR_ONLY_CONTEXTUAL=true # Only extract data related to an entity (a report, a threat actor, etc.)
  • CONNECTOR_CONFIDENCE_LEVEL=15 # From 0 (Unknown) to 100 (Fully trusted)
  • CONNECTOR_LOG_LEVEL=info
  • IMPORT_REPORT_CREATE_INDICATOR=false restart: always volumes: esdata: s3data: redisdata: amqpdata:

another question :

  1. OPENCTI_TOKEN and CONNECTOR_ID are using the same UUIDv4?
  2. should I install " Grakn " ?

thanks a lot

markkkkkkkkkkkkkkkkkkk commented 3 years ago

Hi @Sue1990 - are you still experiencing this problem?

Sue1990 commented 3 years ago

@markkkkkkkkkkkkkkkkkkk Yes, I haven't resolved this problem

markkkkkkkkkkkkkkkkkkk commented 3 years ago

@Sue1990

Try adding the deploy placement constraints under the depends on portion of your YML file.

depends_on:
  - redis
  - elasticsearch
  - minio
  - rabbitmq
restart: always
deploy:
  placement:
    constraints:
      - "node.role==manager"
markkkkkkkkkkkkkkkkkkk commented 3 years ago

If you're runing your deployment in Swarm mode, its possible that the opencti-platform container is being spawned on a worker and if your Docker networking isn't configured for that then the opencti-platform won't be able to communicate to Elasticsearch. Adding the constraint will ensure that the manager is never spawned on a worker, only the manager itself. Hope this works!!

Sue1990 commented 3 years ago

@markkkkkkkkkkkkkkkkkkk Thanks your help , I will try. If I can't resolved problem, I will open this issue again.

wh524611095 commented 2 years ago

@Sue1990 Hi,Have you solved this problem smoothly?I still haven't solved it according to this solution

StitchDave commented 1 year ago

I have the same issue, and although I thought the above fix might work, it didnt.