Closed misohouse closed 11 months ago
Hello @misohouse,
Here is a sample of docker-compose for OpenSearch in single node, you will need to adapt it to your install :
opensearch:
image: opensearchproject/opensearch:latest # Specifying the latest available image - modify if you want a specific version
container_name: opensearch-node1
environment:
- discovery.type=single-node
- plugins.security.disabled=true # Only in development mode
- bootstrap.memory_lock=true # Disable JVM heap memory swapping
- "OPENSEARCH_JAVA_OPTS=-Xms2G -Xmx2G" # Set min and max JVM heap sizes to at least 50% of system RAM
ulimits:
memlock:
soft: -1 # Set memlock to unlimited (no soft or hard limit)
hard: -1
nofile:
soft: 65536 # Maximum number of open files for the opensearch user - set to at least 65536
hard: 65536
ports:
- 9200:9200 # REST API
- 9600:9600 # Performance Analyzer
Here is a complete documentation with a sample of docker-compose in multi-node: https://opensearch.org/docs/latest/install-and-configure/install-opensearch/docker/#sample-docker-composeyml
You will then need to change your opencti configuration to use OpenSearch (specify url and engine_selector):
"elasticsearch": {
"index_prefix": "opencti",
"url": "http://localhost:9200"
"index_creation_pattern": "-000001",
"search_ignore_throttled": false,
"max_pagination_result": 5000,
"max_concurrency": 4,
"engine_selector": "opensearch"
}
@SouadHadjiat
Thank you for the detailed answer!
Currently, the server I built is built with elasticsearch and opencti separated into VMs on different servers.
I'm trying to fix it myself... Despite your detailed answer, I'm not good at handling Docker, so I don't know how to fix it.
Based on what you wrote, would you give me advice on what part of my docker-compose I should fix?
Below is the elasticsearch and opencti part of docker-compose.
Thank you so much again.
If you need any other information, I will reply.
version: '3'
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:8.8.1
ports:
- 9200:9200
- 9300:9300
volumes:
- esdata:/usr/share/elasticsearch/data
environment:
# Comment out the line below for single-node
- discovery.type=single-node
- ingest.geoip.downloader.enabled=false
# Uncomment line below below for a cluster of multiple nodes
# - cluster.name=docker-cluster
- xpack.ml.enabled=false
- xpack.security.enabled=false
- "ES_JAVA_OPTS=-Xms${ELASTIC_MEMORY_SIZE} -Xmx${ELASTIC_MEMORY_SIZE}"
restart: always
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536
hard: 65536
volumes:
esdata:
opencti:
image: opencti/platform:5.10.2
environment:
- NODE_OPTIONS=--max-old-space-size=8096
- APP__PORT=8080
- APP__BASE_URL=${OPENCTI_BASE_URL}
- APP__ADMIN__EMAIL=${OPENCTI_ADMIN_EMAIL}
- APP__ADMIN__PASSWORD=${OPENCTI_ADMIN_PASSWORD}
- APP__ADMIN__TOKEN=${OPENCTI_ADMIN_TOKEN}
- APP__APP_LOGS__LOGS_LEVEL=error
- APP__SESSION_TIMEOUT=10800000 # 1hour = 3600000
- ELASTICSEARCH__URL=http://192.168.200.153:9200
- REDIS__HOSTNAME=redis
- REDIS__PORT=6379
- MINIO__ENDPOINT=minio
- MINIO__PORT=9000
- MINIO__USE_SSL=false
- MINIO__ACCESS_KEY=${MINIO_ROOT_USER}
- MINIO__SECRET_KEY=${MINIO_ROOT_PASSWORD}
- RABBITMQ__HOSTNAME=rabbitmq
- RABBITMQ__PORT=5672
- RABBITMQ__PORT_MANAGEMENT=15672
- RABBITMQ__MANAGEMENT_SSL=false
- RABBITMQ__USERNAME=${RABBITMQ_DEFAULT_USER}
- RABBITMQ__PASSWORD=${RABBITMQ_DEFAULT_PASS}
- SMTP__HOSTNAME=${SMTP_HOSTNAME}
- SMTP__PORT=25
- PROVIDERS__LOCAL__STRATEGY=LocalStrategy
ports:
- "8080:8080"
depends_on:
- redis
- minio
- rabbitmq
restart: always
@misohouse So, here is what you can try in docker-compose for OpenSearch (instead of ElasticSearch) :
services:
opensearch:
image: opensearchproject/opensearch:latest
ports:
- 9200:9200
- 9300:9300
volumes:
- opensearchdata:/usr/share/opensearch/data
environment:
# Comment out the line below for single-node
- discovery.type=single-node
- plugins.security.disabled=true
- bootstrap.memory_lock=true # Disable JVM heap memory swapping
# Uncomment line below below for a cluster of multiple nodes
# - cluster.name=docker-cluster
- "OPENSEARCH_JAVA_OPTS=-Xms${ELASTIC_MEMORY_SIZE} -Xmx${ELASTIC_MEMORY_SIZE}"
restart: always
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536
hard: 65536
volumes:
opensearchdata:
In volumes, I put a new volume for opensearch (opensearchdata), which means your database will be empty, I'm not sure what you will want to do with your elasticsearch data.
And for opencti, there is nothing to change I think, as long as ELASTICSEARCH__URL has the right url to your opensearch database.
@SouadHadjiat
Thank you for reply!
There is data that I have accumulated for about a year in Elasticsearch DB.
So I need to migrate that data to use opensearch.
If I modify docker-compose as you suggested, will I be able to do what I want?
Hello @misohouse,
I have never done a data migration from Elasticsearch to Opensearch, so I can only send you to OpenSearch documentation on this subject : https://opensearch.org/docs/latest/upgrade-to/index/
I plan to open and operate the OpenCTI platform externally.
However, due to ElasticSearch licensing, I would like to change (migrate) to OpenSearch. (The ElasticSearch version currently in use is 8.8.1)
I tried to change docker-compose, but the sample provided on OpenCTI github only specifies information related to ElasticSearch.
I searched the OpenCTI github issues and it seems to support OpenSearch.
I would appreciate it if you could tell me a related link or method.