How do I use OpenSearch instead of ElasticSearch?

[misohouse]

I plan to open and operate the OpenCTI platform externally.

However, due to ElasticSearch licensing, I would like to change (migrate) to OpenSearch. (The ElasticSearch version currently in use is 8.8.1)

I tried to change docker-compose, but the sample provided on OpenCTI github only specifies information related to ElasticSearch.

I searched the OpenCTI github issues and it seems to support OpenSearch.

I would appreciate it if you could tell me a related link or method.

[SouadHadjiat]

Hello @misohouse,

Here is a sample of docker-compose for OpenSearch in single node, you will need to adapt it to your install :

opensearch:
    image: opensearchproject/opensearch:latest # Specifying the latest available image - modify if you want a specific version
    container_name: opensearch-node1
    environment:
      - discovery.type=single-node
      - plugins.security.disabled=true # Only in development mode
      - bootstrap.memory_lock=true # Disable JVM heap memory swapping
      - "OPENSEARCH_JAVA_OPTS=-Xms2G -Xmx2G" # Set min and max JVM heap sizes to at least 50% of system RAM
    ulimits:
      memlock:
        soft: -1 # Set memlock to unlimited (no soft or hard limit)
        hard: -1
      nofile:
        soft: 65536 # Maximum number of open files for the opensearch user - set to at least 65536
        hard: 65536
    ports:
      - 9200:9200 # REST API
      - 9600:9600 # Performance Analyzer

Here is a complete documentation with a sample of docker-compose in multi-node: https://opensearch.org/docs/latest/install-and-configure/install-opensearch/docker/#sample-docker-composeyml

You will then need to change your opencti configuration to use OpenSearch (specify url and engine_selector):

"elasticsearch": {
  "index_prefix": "opencti",
  "url": "http://localhost:9200"
  "index_creation_pattern": "-000001",
  "search_ignore_throttled": false,
  "max_pagination_result": 5000,
  "max_concurrency": 4,
  "engine_selector": "opensearch"
}

[misohouse]

@SouadHadjiat

Thank you for the detailed answer!

Currently, the server I built is built with elasticsearch and opencti separated into VMs on different servers.

I'm trying to fix it myself... Despite your detailed answer, I'm not good at handling Docker, so I don't know how to fix it.

Based on what you wrote, would you give me advice on what part of my docker-compose I should fix?

Below is the elasticsearch and opencti part of docker-compose.

Thank you so much again.

If you need any other information, I will reply.

ElasticSearch

version: '3'
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.8.1
    ports:
      - 9200:9200
      - 9300:9300
    volumes:
      - esdata:/usr/share/elasticsearch/data
    environment:
      # Comment out the line below for single-node
      - discovery.type=single-node
      - ingest.geoip.downloader.enabled=false
      # Uncomment line below below for a cluster of multiple nodes
      # - cluster.name=docker-cluster
      - xpack.ml.enabled=false
      - xpack.security.enabled=false
      - "ES_JAVA_OPTS=-Xms${ELASTIC_MEMORY_SIZE} -Xmx${ELASTIC_MEMORY_SIZE}"
    restart: always
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
volumes:
  esdata:

OpenCTI

  opencti:
    image: opencti/platform:5.10.2
    environment:
      - NODE_OPTIONS=--max-old-space-size=8096
      - APP__PORT=8080
      - APP__BASE_URL=${OPENCTI_BASE_URL}
      - APP__ADMIN__EMAIL=${OPENCTI_ADMIN_EMAIL}
      - APP__ADMIN__PASSWORD=${OPENCTI_ADMIN_PASSWORD}
      - APP__ADMIN__TOKEN=${OPENCTI_ADMIN_TOKEN}
      - APP__APP_LOGS__LOGS_LEVEL=error
      - APP__SESSION_TIMEOUT=10800000 # 1hour = 3600000
      - ELASTICSEARCH__URL=http://192.168.200.153:9200
      - REDIS__HOSTNAME=redis
      - REDIS__PORT=6379
      - MINIO__ENDPOINT=minio
      - MINIO__PORT=9000
      - MINIO__USE_SSL=false
      - MINIO__ACCESS_KEY=${MINIO_ROOT_USER}
      - MINIO__SECRET_KEY=${MINIO_ROOT_PASSWORD}
      - RABBITMQ__HOSTNAME=rabbitmq
      - RABBITMQ__PORT=5672
      - RABBITMQ__PORT_MANAGEMENT=15672
      - RABBITMQ__MANAGEMENT_SSL=false
      - RABBITMQ__USERNAME=${RABBITMQ_DEFAULT_USER}
      - RABBITMQ__PASSWORD=${RABBITMQ_DEFAULT_PASS}
      - SMTP__HOSTNAME=${SMTP_HOSTNAME}
      - SMTP__PORT=25
      - PROVIDERS__LOCAL__STRATEGY=LocalStrategy
    ports:
      - "8080:8080"
    depends_on:
      - redis
      - minio
      - rabbitmq
    restart: always

[SouadHadjiat]

@misohouse So, here is what you can try in docker-compose for OpenSearch (instead of ElasticSearch) :

services:
  opensearch:
    image: opensearchproject/opensearch:latest
    ports:
      - 9200:9200
      - 9300:9300
    volumes:
      - opensearchdata:/usr/share/opensearch/data
    environment:
      # Comment out the line below for single-node
      - discovery.type=single-node
      - plugins.security.disabled=true
      - bootstrap.memory_lock=true # Disable JVM heap memory swapping
      # Uncomment line below below for a cluster of multiple nodes
      # - cluster.name=docker-cluster
      - "OPENSEARCH_JAVA_OPTS=-Xms${ELASTIC_MEMORY_SIZE} -Xmx${ELASTIC_MEMORY_SIZE}"
    restart: always
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
volumes:
  opensearchdata:

In volumes, I put a new volume for opensearch (opensearchdata), which means your database will be empty, I'm not sure what you will want to do with your elasticsearch data.

And for opencti, there is nothing to change I think, as long as ELASTICSEARCH__URL has the right url to your opensearch database.

[misohouse]

@SouadHadjiat

Thank you for reply!

There is data that I have accumulated for about a year in Elasticsearch DB.

So I need to migrate that data to use opensearch.

If I modify docker-compose as you suggested, will I be able to do what I want?

[SouadHadjiat]

Hello @misohouse,

I have never done a data migration from Elasticsearch to Opensearch, so I can only send you to OpenSearch documentation on this subject : https://opensearch.org/docs/latest/upgrade-to/index/