Cant get mine running

[robbiemueller]

Ive gone through the guide twice now with a fresh VM each time to be sure I didn't screw any steps up.

I keep getting these logs on the opencti container and it wont start

{"category":"APP","error":{"context":{"category":"technical","error":{"_error":{},"_showLocations":false,"_showPath":false,"data":{"category":"technical","http_status":500,"reason":"[INIT] Fail initialize schema, index already exists, previous initialization fail because you kill the platform before the end of the initialization. Please remove your elastic/opensearch data and restart."},"internalData":{},"name":"ConfigurationError","time_thrown":"2023-08-05T16:45:06.530Z"},"http_status":500,"reason":"[OPENCTI] Platform initialization fail"},"message":"An unknown error has occurred","name":"UnknownError","stack":"UnknownError: An unknown error has occurred\n at error (/opt/opencti/build/src/config/errors.js:8:10)\n at UnknownError (/opt/opencti/build/src/config/errors.js:68:47)\n at platformInit (/opt/opencti/build/src/initialization.js:393:13)\n at processTicksAndRejections (node:internal/process/task_queues:95:5)\n at platformStart (/opt/opencti/build/src/boot.js:183:5)"},"level":"error","message":"[OPENCTI] Platform start fail","timestamp":"2023-08-05T16:45:06.531Z","version":"5.9.6"}

It does this even on a fresh install so not sure why its saying I need to remove the elastic/opensearch data and restart.

Ive tried shutting down the elasticsearch service, deleting the volume, and then re-deploying the stack and I get the same thing

[robbiemueller]

Here is my docker compose

services:
  redis:
    image: redis:7.0.12
    restart: always
    volumes:
      - redisdata:/data
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.8.2
    volumes:
      - esdata:/usr/share/elasticsearch/data
    environment:
      # Comment out the line below for single-node
      - discovery.type=single-node
      # Uncomment line below below for a cluster of multiple nodes
      # - cluster.name=docker-cluster
      - xpack.ml.enabled=false
      - xpack.security.enabled=false
      - "ES_JAVA_OPTS=-Xms${ELASTIC_MEMORY_SIZE} -Xmx${ELASTIC_MEMORY_SIZE}"
    restart: always
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
  minio:
    image: minio/minio:RELEASE.2023-05-18T00-05-36Z
    volumes:
      - s3data:/data
    ports:
      - "9000:9000"
    environment:
      MINIO_ROOT_USER: ${MINIO_ROOT_USER}
      MINIO_ROOT_PASSWORD: ${MINIO_ROOT_PASSWORD}    
    command: server /data
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3
    restart: always
  rabbitmq:
    image: rabbitmq:3.12-management
    environment:
      - RABBITMQ_DEFAULT_USER=${RABBITMQ_DEFAULT_USER}
      - RABBITMQ_DEFAULT_PASS=${RABBITMQ_DEFAULT_PASS}
    volumes:
      - amqpdata:/var/lib/rabbitmq
    restart: always
  opencti:
    image: opencti/platform:5.9.6
    environment:
      - NODE_OPTIONS=--max-old-space-size=8096
      - APP__PORT=8080
      - APP__BASE_URL=${OPENCTI_BASE_URL}
      - APP__ADMIN__EMAIL=${OPENCTI_ADMIN_EMAIL}
      - APP__ADMIN__PASSWORD=${OPENCTI_ADMIN_PASSWORD}
      - APP__ADMIN__TOKEN=${OPENCTI_ADMIN_TOKEN}
      - APP__APP_LOGS__LOGS_LEVEL=error
      - REDIS__HOSTNAME=redis
      - REDIS__PORT=6379
      - ELASTICSEARCH__URL=http://elasticsearch:9200
      - MINIO__ENDPOINT=minio
      - MINIO__PORT=9000
      - MINIO__USE_SSL=false
      - MINIO__ACCESS_KEY=${MINIO_ROOT_USER}
      - MINIO__SECRET_KEY=${MINIO_ROOT_PASSWORD}
      - RABBITMQ__HOSTNAME=rabbitmq
      - RABBITMQ__PORT=5672
      - RABBITMQ__PORT_MANAGEMENT=15672
      - RABBITMQ__MANAGEMENT_SSL=false
      - RABBITMQ__USERNAME=${RABBITMQ_DEFAULT_USER}
      - RABBITMQ__PASSWORD=${RABBITMQ_DEFAULT_PASS}
      - SMTP__HOSTNAME=${SMTP_HOSTNAME}
      - SMTP__PORT=25
      - PROVIDERS__LOCAL__STRATEGY=LocalStrategy
    ports:
      - "8080:8080"
    depends_on:
      - redis
      - elasticsearch
      - minio
      - rabbitmq
    restart: always
  worker:
    image: opencti/worker:5.9.6
    environment:
      - OPENCTI_URL=http://opencti:8080
      - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
      - WORKER_LOG_LEVEL=info
    depends_on:
      - opencti
    deploy:
      mode: replicated
      replicas: 3
    restart: always
  connector-export-file-stix:
    image: opencti/connector-export-file-stix:5.9.6
    environment:
      - OPENCTI_URL=http://opencti:8080
      - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
      - CONNECTOR_ID=${CONNECTOR_EXPORT_FILE_STIX_ID} # Valid UUIDv4
      - CONNECTOR_TYPE=INTERNAL_EXPORT_FILE
      - CONNECTOR_NAME=ExportFileStix2
      - CONNECTOR_SCOPE=application/json
      - CONNECTOR_CONFIDENCE_LEVEL=15 # From 0 (Unknown) to 100 (Fully trusted)
      - CONNECTOR_LOG_LEVEL=info
    restart: always
    depends_on:
      - opencti
  connector-export-file-csv:
    image: opencti/connector-export-file-csv:5.9.6
    environment:
      - OPENCTI_URL=http://opencti:8080
      - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
      - CONNECTOR_ID=${CONNECTOR_EXPORT_FILE_CSV_ID} # Valid UUIDv4
      - CONNECTOR_TYPE=INTERNAL_EXPORT_FILE
      - CONNECTOR_NAME=ExportFileCsv
      - CONNECTOR_SCOPE=text/csv
      - CONNECTOR_CONFIDENCE_LEVEL=15 # From 0 (Unknown) to 100 (Fully trusted)
      - CONNECTOR_LOG_LEVEL=info
    restart: always
    depends_on:
      - opencti
  connector-export-file-txt:
    image: opencti/connector-export-file-txt:5.9.6
    environment:
      - OPENCTI_URL=http://opencti:8080
      - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
      - CONNECTOR_ID=${CONNECTOR_EXPORT_FILE_TXT_ID} # Valid UUIDv4
      - CONNECTOR_TYPE=INTERNAL_EXPORT_FILE
      - CONNECTOR_NAME=ExportFileTxt
      - CONNECTOR_SCOPE=text/plain
      - CONNECTOR_CONFIDENCE_LEVEL=15 # From 0 (Unknown) to 100 (Fully trusted)
      - CONNECTOR_LOG_LEVEL=info
    restart: always
    depends_on:
      - opencti
  connector-import-file-stix:
    image: opencti/connector-import-file-stix:5.9.6
    environment:
      - OPENCTI_URL=http://opencti:8080
      - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
      - CONNECTOR_ID=${CONNECTOR_IMPORT_FILE_STIX_ID} # Valid UUIDv4
      - CONNECTOR_TYPE=INTERNAL_IMPORT_FILE
      - CONNECTOR_NAME=ImportFileStix
      - CONNECTOR_VALIDATE_BEFORE_IMPORT=true # Validate any bundle before import
      - CONNECTOR_SCOPE=application/json,text/xml
      - CONNECTOR_AUTO=true # Enable/disable auto-import of file
      - CONNECTOR_CONFIDENCE_LEVEL=15 # From 0 (Unknown) to 100 (Fully trusted)
      - CONNECTOR_LOG_LEVEL=info
    restart: always
    depends_on:
      - opencti
  connector-import-document:
    image: opencti/connector-import-document:5.9.6
    environment:
      - OPENCTI_URL=http://opencti:8080
      - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
      - CONNECTOR_ID=${CONNECTOR_IMPORT_DOCUMENT_ID} # Valid UUIDv4
      - CONNECTOR_TYPE=INTERNAL_IMPORT_FILE
      - CONNECTOR_NAME=ImportDocument
      - CONNECTOR_VALIDATE_BEFORE_IMPORT=true # Validate any bundle before import
      - CONNECTOR_SCOPE=application/pdf,text/plain,text/html
      - CONNECTOR_AUTO=true # Enable/disable auto-import of file
      - CONNECTOR_ONLY_CONTEXTUAL=false # Only extract data related to an entity (a report, a threat actor, etc.)
      - CONNECTOR_CONFIDENCE_LEVEL=15 # From 0 (Unknown) to 100 (Fully trusted)
      - CONNECTOR_LOG_LEVEL=info
      - IMPORT_DOCUMENT_CREATE_INDICATOR=true
    restart: always
    depends_on:
      - opencti
  connector-alienvault:
    image: opencti/connector-alienvault:5.9.6
    environment:
      - OPENCTI_URL=http://opencti:8080
      - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
      - CONNECTOR_ID=4d3c1dbf-b521-44b5-8d4d-c011554993b7
      - CONNECTOR_TYPE=EXTERNAL_IMPORT
      - CONNECTOR_NAME=AlienVault
      - CONNECTOR_SCOPE=alienvault
      - CONNECTOR_CONFIDENCE_LEVEL=15 # From 0 (Unknown) to 100 (Fully trusted)
      - CONNECTOR_UPDATE_EXISTING_DATA=false
      - CONNECTOR_LOG_LEVEL=info
      - ALIENVAULT_BASE_URL=https://otx.alienvault.com
      - ALIENVAULT_API_KEY=REDACTED
      - ALIENVAULT_TLP=White
      - ALIENVAULT_CREATE_OBSERVABLES=true
      - ALIENVAULT_CREATE_INDICATORS=true
      - ALIENVAULT_PULSE_START_TIMESTAMP=2020-05-01T00:00:00                  # BEWARE! Could be a lot of pulses!
      - ALIENVAULT_REPORT_TYPE=threat-report
      - ALIENVAULT_REPORT_STATUS=New
      - ALIENVAULT_GUESS_MALWARE=false                                        # Use tags to guess malware.
      - ALIENVAULT_GUESS_CVE=false                                            # Use tags to guess CVE.
      - ALIENVAULT_EXCLUDED_PULSE_INDICATOR_TYPES=FileHash-MD5,FileHash-SHA1  # Excluded Pulse indicator types.
      - ALIENVAULT_ENABLE_RELATIONSHIPS=true                                  # Enable/Disable relationship creation between SDOs.
      - ALIENVAULT_ENABLE_ATTACK_PATTERNS_INDICATES=true                      # Enable/Disable "indicates" relationships between indicators and attack patterns
      - ALIENVAULT_INTERVAL_SEC=1800
    restart: always
    depends_on:
      - opencti
  connector-cybercrimetracker:
    image: opencti/connector-cybercrime-tracker:5.9.6
    environment:
      - OPENCTI_URL=http://opencti:8080
      - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
      - CONNECTOR_ID=11aee9c6-83e8-4cc3-8e06-c47e2707b31e
      - CONNECTOR_TYPE=EXTERNAL_IMPORT
      - CONNECTOR_NAME=Cybercrime-Tracker
      - CONNECTOR_SCOPE=cybercrime-tracker
      - CONNECTOR_CONFIDENCE_LEVEL=15 # From 0 (Unknown) to 100 (Fully trusted)
      - CONNECTOR_UPDATE_EXISTING_DATA=false
      - CONNECTOR_LOG_LEVEL=info
      - CYBERCRIME_TRACKER_FEED_URL=http://cybercrime-tracker.net/rss.xml
      - CYBERCRIME_TRACKER_TLP=WHITE
      - CYBERCRIME_TRACKER_INTERVAL=86400
      - CYBERCRIME_TRACKER_CREATE_INDICATORS=true
      - CYBERCRIME_TRACKER_CREATE_OBSERVABLES=true
    restart: always
    depends_on:
      - opencti
  abuseipdb-ipblacklist:
    image: opencti/connector-abuseipdb-ipblacklist:5.9.6
    environment:
      - OPENCTI_URL=http://opencti:8080
      - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
      - CONNECTOR_ID=ef2efb3c-1200-47e7-a40e-952b20b2ac5b # Valid UUIDv4
      - CONNECTOR_TYPE=EXTERNAL_IMPORT
      - "CONNECTOR_NAME=AbuseIPDB IP Blacklist"
      - CONNECTOR_SCOPE=abuseipdb
      - CONNECTOR_CONFIDENCE_LEVEL=100 # From 0 (Unknown) to 100 (Fully trusted)
      - CONNECTOR_LOG_LEVEL=info
      - ABUSEIPDB_URL=https://api.abuseipdb.com/api/v2/blacklist
      - ABUSEIPDB_API_KEY=REDACTED
      - ABUSEIPDB_SCORE=100
      - ABUSEIPDB_LIMIT=10000
      - ABUSEIPDB_INTERVAL=2 #Day
    restart: always
    depends_on:
      - opencti
  connector-mitre:
    image: opencti/connector-mitre:5.9.6
    environment:
      - OPENCTI_URL=http://opencti:8080
      - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
      - CONNECTOR_ID=f61ed1b2-2ade-4987-b902-3ac2042f05ec
      - CONNECTOR_TYPE=EXTERNAL_IMPORT
      - "CONNECTOR_NAME=MITRE Datasets"
      - CONNECTOR_SCOPE=marking-definition,identity,attack-pattern,course-of-action,intrusion-set,campaign,malware,tool,report,external-reference-as-report
      - CONNECTOR_CONFIDENCE_LEVEL=75 # From 0 (Unknown) to 100 (Fully trusted)
      - CONNECTOR_UPDATE_EXISTING_DATA=false
      - CONNECTOR_RUN_AND_TERMINATE=false
      - CONNECTOR_LOG_LEVEL=info
      - MITRE_ENTERPRISE_FILE_URL=https://raw.githubusercontent.com/mitre-attack/attack-stix-data/master/enterprise-attack/enterprise-attack.json
      - MITRE_MOBILE_ATTACK_FILE_URL=https://raw.githubusercontent.com/mitre-attack/attack-stix-data/master/mobile-attack/mobile-attack.json
      - MITRE_ICS_ATTACK_FILE_URL=https://raw.githubusercontent.com/mitre-attack/attack-stix-data/master/ics-attack/ics-attack.json
      - MITRE_CAPEC_FILE_URL=https://raw.githubusercontent.com/mitre/cti/master/capec/2.1/stix-capec.json
      - MITRE_INTERVAL=3 # In days, must be strictly greater than 1
    restart: always
    depends_on:
      - opencti
  connector-urlhaus:
    image: opencti/connector-urlhaus:5.9.6
    environment:
      - OPENCTI_URL=http://opencti:8080
      - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
      - CONNECTOR_ID=7ff6b8fd-7a4c-4cdc-9f1d-c5a8ff65a724
      - CONNECTOR_TYPE=EXTERNAL_IMPORT
      - "CONNECTOR_NAME=Abuse.ch URLhaus"
      - CONNECTOR_SCOPE=urlhaus
      - CONNECTOR_CONFIDENCE_LEVEL=40 # From 0 (Unknown) to 100 (Fully trusted)
      - CONNECTOR_UPDATE_EXISTING_DATA=false
      - CONNECTOR_LOG_LEVEL=info
      - URLHAUS_CSV_URL=https://urlhaus.abuse.ch/downloads/csv_recent/
      - URLHAUS_IMPORT_OFFLINE=true
      - URLHAUS_CREATE_INDICATORS=true
      - URLHAUS_INTERVAL=2 # In days, must be strictly greater than 1
    restart: always
    depends_on:
      - opencti
  connector-urlscan:
    image: opencti/connector-urlscan:5.9.6
    environment:
      - OPENCTI_URL=http://opencti:8080
      - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
      - CONNECTOR_ID=2b94f5a3-280c-4dbc-82dd-0a60d5c8707c
      - CONNECTOR_TYPE=EXTERNAL_IMPORT
      - CONNECTOR_NAME=Urlscan
      - CONNECTOR_SCOPE=urlscan
      - CONNECTOR_LOG_LEVEL=info
      - CONNECTOR_CONFIDENCE_LEVEL=40 # From 0 (Unknown) to 100 (Fully trusted)
      - CONNECTOR_UPDATE_EXISTING_DATA=false
      - CONNECTOR_CREATE_INDICATORS=true
      - CONNECTOR_TLP=white
      - CONNECTOR_LABELS=Phishing,Phishfeed
      - CONNECTOR_INTERVAL=86400 # seconds, 1d
      - URLSCAN_URL=https://urlscan.io/api/v1/pro/phishfeed?format=json&q=date:>now-1d
      - URLSCAN_API_KEY=REDACTED
    restart: always
    depends_on:
      - opencti

  connector-vxvault:
    image: opencti/connector-vxvault:5.9.6
    environment:
      - OPENCTI_URL=http://opencti:8080
      - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
      - CONNECTOR_ID=7b721741-08a4-45e8-8582-b22411dfdb1d
      - CONNECTOR_TYPE=EXTERNAL_IMPORT
      - "CONNECTOR_NAME=VX Vault URL list"
      - CONNECTOR_SCOPE=vxvault
      - CONNECTOR_CONFIDENCE_LEVEL=40 # From 0 (Unknown) to 100 (Fully trusted)
      - CONNECTOR_UPDATE_EXISTING_DATA=false
      - CONNECTOR_LOG_LEVEL=info
      - VXVAULT_URL=http://vxvault.net/URL_List.php
      - VXVAULT_CREATE_INDICATORS=true
      - VXVAULT_INTERVAL=3 # In days, must be strictly greater than 1
    restart: always
    depends_on:
      - opencti
  connector-shodan:
    image: opencti/connector-shodan:5.9.6
    environment:
      - OPENCTI_URL=http://opencti:8080
      - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
      - CONNECTOR_ID=c97f6bca-dd49-490b-a8ac-2b7446c396e1
      - CONNECTOR_TYPE=INTERNAL_ENRICHMENT
      - CONNECTOR_NAME=Shodan
      - CONNECTOR_SCOPE=IPv4-Addr
      - CONNECTOR_AUTO=true
      - CONNECTOR_CONFIDENCE_LEVEL=75 # From 0 (Unknown) to 100 (Fully trusted)
      - CONNECTOR_LOG_LEVEL=info
      - SHODAN_TOKEN=REDACTED
      - SHODAN_MAX_TLP=TLP:AMBER
      - SHODAN_CREATE_INDICATORS=true
    restart: always
    depends_on:
      - opencti 
  connector-intezer-sandbox:
    image: opencti/connector-intezer-sandbox:5.9.6
    environment:
      - OPENCTI_URL=http://opencti:8080
      - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
      - CONNECTOR_ID=58ae8833-08a8-432b-8b26-face00297cbe
      - CONNECTOR_TYPE=INTERNAL_ENRICHMENT
      - "CONNECTOR_NAME=IntezerSandbox"
      - CONNECTOR_SCOPE=Artifact
      - CONNECTOR_AUTO=true # Enable/disable auto-enrichment of observables
      - CONNECTOR_CONFIDENCE_LEVEL=50 # From 0 (Unknown) to 100 (Fully trusted)
      - CONNECTOR_LOG_LEVEL=info
      - INTEZER_SANDBOX_API_KEY=REDACTED
      - INTEZER_SANDBOX_FAMILY_COLOR=#0059f7 # Label color for family
      - INTEZER_SANDBOX_MALICIOUS_COLOR=#d90e18 # Label color for malicious verdict
      - INTEZER_SANDBOX_TRUSTED_COLOR=#d90e18 # And so on...
      - INTEZER_SANDBOX_UNKNOWN_COLOR=#ffff00
      - INTEZER_SANDBOX_SUSPICIOUS_COLOR=#f79e00
      - INTEZER_SANDBOX_MAX_TLP=TLP:AMBER
    restart: always
    depends_on:
      - opencti 
  connector-ipinfo:
    image: opencti/connector-ipinfo:5.9.6
    environment:
      - OPENCTI_URL=http://opencti:8080
      - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
      - CONNECTOR_ID=98a30a00-a5f7-4e78-8674-158d09602a22
      - CONNECTOR_TYPE=INTERNAL_ENRICHMENT
      - CONNECTOR_NAME=IpInfo
      - CONNECTOR_SCOPE=IPv4-Addr
      - CONNECTOR_AUTO=true
      - CONNECTOR_CONFIDENCE_LEVEL=75 # From 0 (Unknown) to 100 (Fully trusted)
      - CONNECTOR_LOG_LEVEL=info
      - IPINFO_TOKEN=REDACTED
      - IPINFO_MAX_TLP=TLP:AMBER
    restart: always
    depends_on:
      - opencti
  connector-opencti:
    image: opencti/connector-opencti:5.9.6
    environment:
      - OPENCTI_URL=http://opencti:8080
      - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
      - CONNECTOR_ID=b4103b49-6ab4-4754-b208-c38fdcec596c
      - CONNECTOR_TYPE=EXTERNAL_IMPORT
      - CONNECTOR_NAME=OpenCTI
      - CONNECTOR_SCOPE=marking-definition,identity,location
      - CONNECTOR_CONFIDENCE_LEVEL=90 # From 0 (Unknown) to 100 (Fully trusted)
      - CONNECTOR_UPDATE_EXISTING_DATA=true
      - CONNECTOR_RUN_AND_TERMINATE=false
      - CONNECTOR_LOG_LEVEL=info
      - CONFIG_SECTORS_FILE_URL=https://raw.githubusercontent.com/OpenCTI-Platform/datasets/master/data/sectors.json
      - CONFIG_GEOGRAPHY_FILE_URL=https://raw.githubusercontent.com/OpenCTI-Platform/datasets/master/data/geography.json
      - CONFIG_INTERVAL=2 # In days, must be strictly greater than 1
    restart: always
    depends_on:
      - opencti
  connector-aptcampaign:
    image: opencti/connector-cyber-campaign-collection:5.9.6
    environment:
      - OPENCTI_URL=http://opencti:8080
      - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
      - CONNECTOR_ID=aecb0da5-d065-4404-9dec-23272bb835e9
      - CONNECTOR_TYPE=EXTERNAL_IMPORT
      - "CONNECTOR_NAME=APT & Cybercriminals Campaign Collection"
      - CONNECTOR_SCOPE=report
      - CONNECTOR_CONFIDENCE_LEVEL=0 # From 0 (Unknown) to 100 (Fully trusted)
      - CONNECTOR_UPDATE_EXISTING_DATA=false
      - CONNECTOR_RUN_AND_TERMINATE=false
      - CONNECTOR_LOG_LEVEL=info
      - CYBER_MONITOR_GITHUB_TOKEN=ENTERGITHUBTOKENHERE # If not provided, rate limit will be very low
      - CYBER_MONITOR_FROM_YEAR=2018
      - CYBER_MONITOR_INTERVAL=2 # In days, must be strictly greater than 1
    restart: always
    depends_on:
      - opencti
  connector-malware-bazaar-recent-additions:
    image: opencti/connector-malwarebazaar-recent-additions:5.9.6
    environment:
      - OPENCTI_URL=http://opencti:8080
      - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
      - CONNECTOR_ID=a928d96c-65d1-4571-b8de-036e99a2a34f
      - CONNECTOR_TYPE=EXTERNAL_IMPORT
      - "CONNECTOR_NAME=MalwareBazaar Recent Additions"
      - CONNECTOR_CONFIDENCE_LEVEL=50 # From 0 (Unknown) to 100 (Fully trusted)
      - CONNECTOR_UPDATE_EXISTING_DATA=false
      - CONNECTOR_LOG_LEVEL=info
      - MALWAREBAZAAR_RECENT_ADDITIONS_API_URL=https://mb-api.abuse.ch/api/v1/
      - MALWAREBAZAAR_RECENT_ADDITIONS_COOLDOWN_SECONDS=300 # Time to wait in seconds between subsequent requests
      - MALWAREBAZAAR_RECENT_ADDITIONS_INCLUDE_TAGS=exe,dll,docm,docx,doc,xls,xlsx,xlsm,js # (Optional) Only download files if any tag matches. (Comma separated)
      - MALWAREBAZAAR_RECENT_ADDITIONS_INCLUDE_REPORTERS= # (Optional) Only download files uploaded by these reporters. (Comma separated)
      - MALWAREBAZAAR_RECENT_ADDITIONS_LABELS=malware-bazar # (Optional) Labels to apply to uploaded Artifacts. (Comma separated)
      - MALWAREBAZAAR_RECENT_ADDITIONS_LABELS_COLOR=#54483b # Color to use for labels
    restart: always
    depends_on:
      - opencti
  connector-ivre:
    image: opencti/connector-ivre:5.9.6
    environment:
      - OPENCTI_URL=http://opencti:8080
      - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
      - CONNECTOR_ID=d9f361ba-75d0-4066-aade-e3d3878a6b6c
      - CONNECTOR_TYPE=INTERNAL_ENRICHMENT
      - CONNECTOR_NAME=ivre
      - CONNECTOR_SCOPE=autonomous-system,domain-name,ipv4-addr,ipv6-addr,mac-addr,x509-certificate
      - CONNECTOR_AUTO=true
      - CONNECTOR_CONFIDENCE_LEVEL=100 # From 0 (Unknown) to 100 (Fully trusted)
      - CONNECTOR_LOG_LEVEL=info
      - IVRE_MAX_TLP=TLP:AMBER
    restart: always
    depends_on:
      - opencti
  connector-misp-feed:
    image: opencti/connector-misp-feed:5.9.6
    environment:
      - OPENCTI_URL=http://opencti:8080
      - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
      - CONNECTOR_ID=3275a246-8833-4305-87e2-1ce54ccb5cc6
      - CONNECTOR_TYPE=EXTERNAL_IMPORT
      - "CONNECTOR_NAME=MISP Feed (CIRCL)"
      - CONNECTOR_SCOPE=misp-feed-circl
      - CONNECTOR_CONFIDENCE_LEVEL=20 # From 0 (Unknown) to 100 (Fully trusted)
      - CONNECTOR_UPDATE_EXISTING_DATA=false
      - CONNECTOR_RUN_AND_TERMINATE=false
      - CONNECTOR_LOG_LEVEL=info
      - MISP_FEED_URL=https://www.circl.lu/doc/misp/feed-osint
      - MISP_FEED_SSL_VERIFY=true # Required
      - MISP_FEED_IMPORT_FROM_DATE=2021-01-01 # Required, import all event from this date
      - MISP_FEED_CREATE_REPORTS=true # Required, create report for MISP event
      - MISP_FEED_REPORT_TYPE=misp-event
      - MISP_FEED_CREATE_INDICATORS=true # Required, create indicators from attributes
      - MISP_FEED_CREATE_OBSERVABLES=true # Required, create observables from attributes
      - MISP_FEED_CREATE_OBJECT_OBSERVABLES=true # Required, create text observables for MISP objects
      - MISP_FEED_IMPORT_TO_IDS_NO_SCORE=40 # Optional, use as a score for the indicator/observable if the attribute to_ids is no
      - MISP_FEED_IMPORT_UNSUPPORTED_OBSERVABLES_AS_TEXT=false #  Optional, import unsupported observable as x_opencti_text
      - MISP_FEED_IMPORT_WITH_ATTACHMENTS=true # Optional, try to import a PDF file from the attachment attribute
      - MISP_FEED_INTERVAL=5 # Required, in minutes
    restart: always
    depends_on:
      - opencti
  connector-tweetfeed:
    image: opencti/connector-tweetfeed:5.9.6
    environment:
      - OPENCTI_URL=http://opencti:8080
      - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
      - CONNECTOR_ID=01ade3f8-a7ff-49e5-ae15-82d8be8c112c
      - CONNECTOR_TYPE=EXTERNAL_IMPORT
      - CONNECTOR_NAME=Tweetfeed
      - CONNECTOR_SCOPE=tweetfeed
      - CONNECTOR_CONFIDENCE_LEVEL=15
      - CONNECTOR_LOG_LEVEL=info
      - TWEETFEED_CONFIDENCE_LEVEL=15  # From 0 (Unknown) to 100 (Fully trusted)
      - TWEETFEED_CREATE_INDICATORS=true
      - TWEETFEED_CREATE_OBSERVABLES=true
      - TWEETFEED_INTERVAL=1
      - TWEETFEED_UPDATE_EXISTING_DATA=true
      - "TWEETFEED_ORG_DESCRIPTION=Tweetfeed, a connector to import IOC from Twitter."
      - TWEETFEED_ORG_NAME=Tweetfeed
    restart: always
    depends_on:
      - opencti
  connector-google-dns:
    image: opencti/connector-google-dns:5.9.6
    environment:
      - OPENCTI_URL=http://opencti:8080
      - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
      - CONNECTOR_ID=2366061c-874e-404a-bffa-ad3db19b81d7
      - CONNECTOR_TYPE=INTERNAL_ENRICHMENT
      - CONNECTOR_NAME=Google DNS
      - CONNECTOR_SCOPE=Domain-Name,Hostname # MIME type or Stix Object
      - CONNECTOR_AUTO=false
      - CONNECTOR_CONFIDENCE_LEVEL=100 # From 0 (Unknown) to 100 (Fully trusted)
      - CONNECTOR_LOG_LEVEL=info
    restart: always
    depends_on:
      - opencti

volumes:
  esdata:
  s3data:
  redisdata:
  amqpdata:```

and here are my env variables

```OPENCTI_ADMIN_EMAIL=REDACTED
OPENCTI_ADMIN_PASSWORD=REDACTED
OPENCTI_ADMIN_TOKEN=9a22733d-0b06-4078-bc07-c71beb129986
MINIO_ROOT_USER=opencti
MINIO_ROOT_PASSWORD=REDACTED
RABBITMQ_DEFAULT_USER=opencti
RABBITMQ_DEFAULT_PASS=REDACTED
CONNECTOR_EXPORT_FILE_STIX_ID=12284fb8-480a-45d0-ae33-1064b2e82715
CONNECTOR_EXPORT_FILE_CSV_ID=a44c8231-669a-4c96-a970-6b9ec23af062
CONNECTOR_EXPORT_FILE_TXT_ID=b9bb909b-2346-4632-ac3f-2e1cb069168e
CONNECTOR_IMPORT_FILE_STIX_ID=14508809-e5a6-410e-9606-6263e14c353e
CONNECTOR_IMPORT_DOCUMENT_ID=21823e0c-7305-4630-a627-899ba3967e4f
ELASTIC_MEMORY_SIZE=4G
OPENCTI_BASE_URL=http://localhost:8080```

[2004gixxer600]

I took a quick look, and your Docker-compose file looks correct, I didn't spot any obvious errors. I did however notice 3 characters after the last line that shouldn't be there. I have to assume this is a copy and paste glitch from your text editor if I had to guess, perhaps hidden characters are getting inserted. I don't think that is it though, although double check that. Yours has ``` after ampqdata:

I have personally never seen this error after many successful installs with different OpenCTI versions. Although apparently in the forum post link below, some people are having this issue.

See this forum post, and pay attention to the last post for a workaround to the particular error your getting.

https://github.com/OpenCTI-Platform/opencti/issues/3487

Try this:

stop docker service service

docker stop

and then run command

rm -Rf /var/lib/docker/*

After purge all, restart docker service docker start docker-compose up -d again, wait about 3m and done

[robbiemueller]

So those last 3 characters are just my formatting mistake on trying to get the code box to work in the thread here, those arent actually in my file.

Just tried doing that and no dice. Still getting the same thing.

I had found some other suggestions online to try starting the stack with the bare minimum connectors so I commented out almost all of them to try again and still get the same error. I'm really not sure whats happening here.

The first container that opencti tries to create gets this error, which sounds like elastic is just not up yet fully.

{"category":"APP","error":{"context":{"category":"technical","error":"getaddrinfo ENOTFOUND elasticsearch","http_status":500,"reason":"[SEARCH] Search engine seems down"},"message":"A configuration error has occurred","name":"ConfigurationError","stack":"ConfigurationError: A configuration error has occurred\n at error (/opt/opencti/build/src/config/errors.js:8:10)\n at ConfigurationError (/opt/opencti/build/src/config/errors.js:61:53)\n at /opt/opencti/build/src/database/engine.js:185:15\n at processTicksAndRejections (node:internal/process/task_queues:95:5)\n at searchEngineInit (/opt/opencti/build/src/database/engine.js:175:3)\n at checkSystemDependencies (/opt/opencti/build/src/initialization.js:138:3)\n at platformStart (/opt/opencti/build/src/boot.js:179:5)"},"level":"error","message":"[OPENCTI] Platform start fail","timestamp":"2023-08-05T19:45:40.508Z","version":"5.9.6"}

The second container that opencti attempts to start gets this error {"category":"APP","error":{"context":{"category":"technical","error":{"meta":{"body":{"error":{"index":"opencti_history-000001","index_uuid":"djjKgClUQa6WfClQJ8sCPA","reason":"index [opencti_history-000001/djjKgClUQa6WfClQJ8sCPA] already exists","root_cause":[{"index":"opencti_history-000001","index_uuid":"djjKgClUQa6WfClQJ8sCPA","reason":"index [opencti_history-000001/djjKgClUQa6WfClQJ8sCPA] already exists","type":"resource_already_exists_exception"}],"type":"resource_already_exists_exception"},"status":400},"headers":{"content-length":"429","content-type":"application/vnd.elasticsearch+json;compatible-with=8","x-elastic-product":"Elasticsearch"},"meta":{"aborted":false,"attempts":1,"connection":{"headers":{},"id":"http://elasticsearch:9200/","status":"alive","url":"http://elasticsearch:9200/"},"context":null,"name":"elasticsearch-js","request":{"id":7,"options":{},"params":{"body":"{\"aliases\":{\"opencti_history\":{}}}","headers":{"accept":"application/vnd.elasticsearch+json; compatible-with=8","content-length":"34","content-type":"application/vnd.elasticsearch+json; compatible-with=8","user-agent":"elasticsearch-js/8.8.1 Node.js 20.4.0; Transport 8.3.2; (linux 5.15.0-78-generic x64)","x-elastic-client-meta":"es=8.8.1,js=20.4.0,t=8.3.2,hc=20.4.0"},"method":"PUT","path":"/opencti_history-000001","querystring":""}}},"statusCode":400,"warnings":null},"name":"ResponseError"},"http_status":500,"reason":"[OPENCTI] Platform initialization fail"},"message":"An unknown error has occurred","name":"UnknownError","stack":"UnknownError: An unknown error has occurred\n at error (/opt/opencti/build/src/config/errors.js:8:10)\n at UnknownError (/opt/opencti/build/src/config/errors.js:68:47)\n at platformInit (/opt/opencti/build/src/initialization.js:393:13)\n at processTicksAndRejections (node:internal/process/task_queues:95:5)\n at platformStart (/opt/opencti/build/src/boot.js:183:5)"},"level":"error","message":"[OPENCTI] Platform start fail","timestamp":"2023-08-05T19:48:54.580Z","version":"5.9.6"}

Which is saying that opencti_history-000001 already exists for some reason. Probably something to do with the first one trying to connect to elastic and failing, but getting far enough to create that history file.

The 3rd and subsequent containers that are created will all give that first error I posted in the OP. Each time ive tried this it follows this pattern.

Also attached here is the logs of the elastic container. I ran these logs through chatgpt and it highlighted the max memory warning it was giving. I adjusted my vm.max_map_count=262144 on my host VM and re-deployed the stack.

_opencti_elasticsearch.1.nl3u5a9ft45cvxglld7i3wosa_logs.txt

That warning is now gone in elastic, but opencti is still going through the same error cycle and not deploying. So fixing that warning didnt seem to help.

Im at a loss here I've seen a lot of similar issues and have tried many of the fixes to no avail. Going to give it a break for now but will try digging into it again later

[2004gixxer600]

What is your environment? Where are you running this VM on in terms of hosting? Strange, sorry your having difficulty. Whats the resources you assigned to the server etc?

You also may want to try adjusting the docker-compose OpenCTI Version numbers to a prior release, instead of the newest. So for example, change 5.9.6 to something like 5.8.6 or even older and see if it runs? Then if you get it running and stabilized then change the version numbers to the newest and it might upgrade just fine.

[robbiemueller]

im just doing this for a project for myself for fun, so not a huge deal but im gonna keep tinkering with it to see if i can get it.

my host is a windows 11 machine, pretty beefy rig. amd7950x3d CPU and 64gb of ram.

the VM is running in hyperv, 20gb of memory, 16 virtual processors, 200gb storage allocated. Ubuntu 22.04 LTS in minimal mode like the guide

[2004gixxer600]

yeah those should be plenty of resources to start out. If you haven't already tried after blowing everything away, plug in older OpenCTI versions in the docker-compose and start up that stack, could be a bug in the newer release they have not gotten around to fully identifying or fixing.

[robbiemueller]

I actually finally got it up and running this morning. I couldnt get it to work at all in portainer. I recreated the VM and started fresh, but this time used docker-compose and not portainer. The first attempt failed, but I was able to see some logs about http timeouts. ChatGPT suggested that I add the environment variable COMPOSE_HTTP_TIMEOUT=200 to increase the timeout time

So I did that and tried once more, and this time it worked. Not sure if me adding that to the stack in portainer would have fixed deployment there or not, but at least I have it up and running now.

Thanks for the help and putting this guide together! Even though I had a lot of troubleshooting it was fun to finally figure out how to get mine going.

[2004gixxer600]

Your welcome! Glad you got it sorted!

[christiantroldmand]

I actually finally got it up and running this morning. I couldnt get it to work at all in portainer. I recreated the VM and started fresh, but this time used docker-compose and not portainer. The first attempt failed, but I was able to see some logs about http timeouts. ChatGPT suggested that I add the environment variable COMPOSE_HTTP_TIMEOUT=200 to increase the timeout time

So I did that and tried once more, and this time it worked. Not sure if me adding that to the stack in portainer would have fixed deployment there or not, but at least I have it up and running now.

Thanks for the help and putting this guide together! Even though I had a lot of troubleshooting it was fun to finally figure out how to get mine going.

Hi i have the same issue as you (identical error message), but I am not able to fix it by adding the timeout. Did you do anything else besides what mentioned here?

[2004gixxer600]

Nope. The timeout a few people have mentioned over the past year since I wrote the article, most people do not have any issues. I do however update the Docker files and version numbers. It's possible I made an error, I will recheck it, as I do it fairly often,. Note however that even when done correctly and things are healthy, the initial startup time to initialize everything takes a good 15-20 min in alot of cases. I wonder if the people having these issues are trying to not assign enough resources to the VM. Without enough memory and or swap file, you can easily get containers that run out of memory. Have you checked the docker logs and such?

[2004gixxer600]

ok, I just noticed, there is one new line under the Rabbit section in the new docker compose that wasn't there in any of the past versions, not sure if that's whats throwing a wrench in the gears. I went and updated my example docker compose to reflect that, so give it another try. The line added was "- RABBITMQ_NODENAME=rabbit01@localhost" it was not in my latest updated docker-compose file. That was hard to spot. :) The way these containers rely on each other and with the depends on variables. If one thing isn't right, that can prevent the whole thing from starting successfully.

rabbitmq: image: rabbitmq:3.12-management environment:

• RABBITMQ_DEFAULT_USER=${RABBITMQ_DEFAULT_USER}
• RABBITMQ_DEFAULT_PASS=${RABBITMQ_DEFAULT_PASS}
• RABBITMQ_NODENAME=rabbit01@localhost

[christiantroldmand]

Hi! Thank you so much for your aid her, adding that fixed the issue and openCTI launched in around 10 minutes time.

Thanks!!!!

From: 2004gixxer600 @.> Sent: Sunday, October 8, 2023 2:39 PM To: 2004gixxer600/openctihowto @.> Cc: Christian Klint @.>; Comment @.> Subject: Re: [2004gixxer600/openctihowto] Cant get mine running (Issue #2)

ok, I just noticed, there is one new line under the Rabbit section in the new docker compose that was there in any of the past versions, not sure if that's whats throwing a wrench in the gears. I went and updated my example docker compose to reflect that, so give it another try. The line added was "- @.***" it was not in my latest updated docker-compose file. That was hard to spot. :) The way these containers rely on each other and with the depends on variables. If one thing isn't right, that can prevent the whole thing from starting successfully.

rabbitmq: image: rabbitmq:3.12-management environment:

• RABBITMQ_DEFAULT_USER=${RABBITMQ_DEFAULT_USER}

• RABBITMQ_DEFAULT_PASS=${RABBITMQ_DEFAULT_PASS}

• @.***

• Reply to this email directly, view it on GitHubhttps://github.com/2004gixxer600/openctihowto/issues/2#issuecomment-1752018009, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AD2NNBXXUH6RM4KVMWE2TBTX6KNGHAVCNFSM6AAAAAA3FKFLA6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJSGAYTQMBQHE. You are receiving this because you commented.Message ID: @.**@.>>

[2004gixxer600]

Thanks for making me take another look, I wouldn't of been aware of it otherwise, and it would likely fail for many others!

Cheers!