search

OpenCTI-Platform / opencti

Open Cyber Threat Intelligence Platform
https://opencti.io
Other
6.13k stars 908 forks source link

Scaled Platform from 2 to 5 Nodes in AWS - Platform Fails to Initialize with Error - The client noticed that the server is not Elasticsearch #8332

Closed f1b3ropt1k closed 1 day ago

f1b3ropt1k commented 5 days ago

Description

Deployment is in AWS using Terraform. There are two platform clusters - Public and Private. Platform was configured for 2 nodes. I scaled to 5 nodes and deployed the task in AWS. The deployment never completes and the tasks continue to start and fail with these error messages:

"user-agent":"elasticsearch-js/8.15.0 (linux 5.10.223-212.873.amzn2.aarch64-arm64; Node.js 20.17.0; Transport 8.7.0)","x-elastic-client-meta":"es=8.15.0,js=20.17.0,t=8.7.0,hc=20.17.0"},"method":"HEAD","path":"/opencti_internal_objects","querystring":""}}},"statusCode":200,"warnings":null},"name":"ProductNotSupportedError","options":{"redaction":{"additionalKeys":[],"type":"replace"}}},"level":"error","message":"[OPENCTI] Platform default initialization failed","source":"backend","timestamp":"2024-09-12T12:31:04.674Z","version":"6.2.18"}

/opt/opencti/build/node_modules/@elastic/transport/src/Transport.ts:701:14)\n at RWt.putPipeline (/opt/opencti/build/node_modules/@elastic/elasticsearch/src/api/api/ingest.ts:323:12)\n at elConfigureAttachmentProcessor (/opt/opencti/build/src/database/engine.js:867:5)\n at searchEngineInit (/opt/opencti/build/src/database/engine.js:332:32)\n at checkSystemDependencies (/opt/opencti/build/src/initialization.js:38:3)\n at platformStart (/opt/opencti/build/src/boot.js:15:7)"}],"level":"error","message":"Engine attachment processor configuration fail","source":"backend","timestamp":"2024-09-12T12:31:04.020Z","version":"6.2.18"}

(/opt/opencti/build/src/database/engine.js:551:22)\n at initializeSchema (/opt/opencti/build/src/database/engine.js:934:33)\n at platformInit (/opt/opencti/build/src/initialization.js:104:7)\n at platformStart (/opt/opencti/build/src/boot.js:29:7)"}],"level":"error","message":"The client noticed that the server is not Elasticsearch and we do not support this unknown product.","source":"backend","timestamp":"2024-09-12T12:30:55.479Z","version":"6.2.18"}

Environment

  1. OS (where OpenCTI server runs): { e.g. Mac OS 10, Windows 10, Ubuntu 16.4, etc. }

Amazon AWS ECS Ubuntu

  1. OpenCTI version: { e.g. OpenCTI 1.0.2 }

6.2.18

  1. OpenCTI client: { e.g. frontend or python }

Frontend

  1. Other environment details:

Amazon OpenSearch 2.13 Redis 6.2 2 ECS Clusters:

Reproducible Steps

Steps to create the smallest reproducible scenario:

Expected Output

I expected the number of platform nodes to increase from 2 to 5 - scaling horizontal as defined in the documentation to 3 nodes per worker.

Actual Output

AWS deployment does not complete and the tasks fail - AWS indicates the tasks keep failing and the logs indicate the errors above.

Additional information

Prior to scaling out the nodes I updated OpenSearch from 2.11 to 2.13 in AWS

Full length content of the three logs that appear to be most relevant:

{"category":"APP","cause":{"meta":{"headers":{"access-control-allow-origin":"*","connection":"close","content-length":"99498","content-type":"application/json; charset=UTF-8","date":"Thu, 12 Sep 2024 12:31:04 GMT"},"meta":{"aborted":false,"attempts":0,"connection":{"_openRequests":0,"deadCount":0,"headers":{"authorization":"[redacted]","user-agent":"elasticsearch-js/8.15.0 (linux 5.10.223-212.873.amzn2.aarch64-arm64; Node.js 20.17.0; Transport 8.7.0)","x-elastic-client-meta":"es=8.15.0,js=20.17.0,t=8.7.0,hc=20.17.0"},"id":"https://vpc-opencti-opensearch-omqy5r5uictqmrlx6jtjoshvw4.us-west-2.es.amazonaws.com/","maxEventListeners":100,"pool":{"_events":{},"_eventsCount":0},"resurrectTimeout":0,"timeout":30000,"tls":{"ca":null,"rejectUnauthorized":true},"url":"https://vpc-opencti-opensearch-omqy5r5uictqmrlx6jtjoshvw4.us-west-2.es.amazonaws.com/","weight":1000},"context":null,"name":"elasticsearch-js","request":{"id":3,"options":{},"params":{"headers":{"accept":"application/vnd.elasticsearch+json; compatible-with=8,text/plain","authorization":"[redacted]","user-agent":"elasticsearch-js/8.15.0 (linux 5.10.223-212.873.amzn2.aarch64-arm64; Node.js 20.17.0; Transport 8.7.0)","x-elastic-client-meta":"es=8.15.0,js=20.17.0,t=8.7.0,hc=20.17.0"},"method":"HEAD","path":"/opencti_internal_objects","querystring":""}}},"statusCode":200,"warnings":null},"name":"ProductNotSupportedError","options":{"redaction":{"additionalKeys":[],"type":"replace"}}},"level":"error","message":"[OPENCTI] Platform default initialization failed","source":"backend","timestamp":"2024-09-12T12:31:04.674Z","version":"6.2.18"}

{"category":"APP","errors":[{"attributes":{"genre":"TECHNICAL","http_status":500},"message":"Engine attachment processor configuration fail","name":"CONFIGURATION_ERROR","stack":"GraphQLError: Engine attachment processor configuration fail\n at error (/opt/opencti/build/src/config/errors.js:7:10)\n at ConfigurationError (/opt/opencti/build/src/config/errors.js:75:53)\n at /opt/opencti/build/src/database/engine.js:879:20\n at processTicksAndRejections (node:internal/process/task_queues:95:5)\n at elConfigureAttachmentProcessor (/opt/opencti/build/src/database/engine.js:867:5)\n at searchEngineInit (/opt/opencti/build/src/database/engine.js:332:32)\n at checkSystemDependencies (/opt/opencti/build/src/initialization.js:38:3)\n at platformStart (/opt/opencti/build/src/boot.js:15:7)"},{"message":"{\"error\":\"Content-Type header [application/vnd.elasticsearch+json; compatible-with=8] is not supported\",\"status\":406}","name":"ResponseError","stack":"ResponseError: {\"error\":\"Content-Type header [application/vnd.elasticsearch+json; compatible-with=8] is not supported\",\"status\":406}\n at a$t._request (/opt/opencti/build/node_modules/@elastic/transport/src/Transport.ts:601:17)\n at processTicksAndRejections (node:internal/process/task_queues:95:5)\n at /opt/opencti/build/node_modules/@elastic/transport/src/Transport.ts:704:22\n at a$t.request (/opt/opencti/build/node_modules/@elastic/transport/src/Transport.ts:701:14)\n at RWt.putPipeline (/opt/opencti/build/node_modules/@elastic/elasticsearch/src/api/api/ingest.ts:323:12)\n at elConfigureAttachmentProcessor (/opt/opencti/build/src/database/engine.js:867:5)\n at searchEngineInit (/opt/opencti/build/src/database/engine.js:332:32)\n at checkSystemDependencies (/opt/opencti/build/src/initialization.js:38:3)\n at platformStart (/opt/opencti/build/src/boot.js:15:7)"}],"level":"error","message":"Engine attachment processor configuration fail","source":"backend","timestamp":"2024-09-12T12:31:04.020Z","version":"6.2.18"}

{"category":"APP","errors":[{"attributes":{"genre":"TECHNICAL","http_status":500},"message":"The client noticed that the server is not Elasticsearch and we do not support this unknown product.","name":"UNKNOWN_ERROR","stack":"GraphQLError: The client noticed that the server is not Elasticsearch and we do not support this unknown product.\n at error (/opt/opencti/build/src/config/errors.js:7:10)\n at UnknownError (/opt/opencti/build/src/config/errors.js:81:47)\n at Object._logWithError (/opt/opencti/build/src/config/conf.js:238:17)\n at Object.error (/opt/opencti/build/src/config/conf.js:247:48)\n at platformStart (/opt/opencti/build/src/boot.js:42:12)\n at processTicksAndRejections (node:internal/process/task_queues:95:5)"},{"message":"The client noticed that the server is not Elasticsearch and we do not support this unknown product.","name":"ProductNotSupportedError","stack":"ProductNotSupportedError: The client noticed that the server is not Elasticsearch and we do not support this unknown product.\n at a$t._request (/opt/opencti/build/node_modules/@elastic/transport/src/Transport.ts:544:17)\n at processTicksAndRejections (node:internal/process/task_queues:95:5)\n at /opt/opencti/build/node_modules/@elastic/transport/src/Transport.ts:704:22\n at a$t.request (/opt/opencti/build/node_modules/@elastic/transport/src/Transport.ts:701:14)\n at NWt.exists (/opt/opencti/build/node_modules/@elastic/elasticsearch/src/api/api/indices.ts:664:12)\n at elIndexExists (/opt/opencti/build/src/database/engine.js:551:22)\n at initializeSchema (/opt/opencti/build/src/database/engine.js:934:33)\n at platformInit (/opt/opencti/build/src/initialization.js:104:7)\n at platformStart (/opt/opencti/build/src/boot.js:29:7)"}],"level":"error","message":"The client noticed that the server is not Elasticsearch and we do not support this unknown product.","source":"backend","timestamp":"2024-09-12T12:30:55.479Z","version":"6.2.18"}

Screenshots (optional)

Screenshot 2024-09-12 at 5 00 17 PM Screenshot 2024-09-12 at 5 00 38 PM Screenshot 2024-09-12 at 5 02 52 PM

romain-filigran commented 5 days ago

Shared with our platform team to see if they already encountered this issue.

richard-julien commented 5 days ago

Could be due to the opensearch upgrade. For some users upgrade the cluster in AWS produces a config change that prevent opencti to correctly detect the type of server. Can you check if the opencti config for elasticsearch engine_selector is configured to opensearch? This config force opencti to use a specific client instead of trying to detect it.

f1b3ropt1k commented 4 days ago

Could be due to the opensearch upgrade. For some users upgrade the cluster in AWS produces a config change that prevent opencti to correctly detect the type of server. Can you check if the opencti config for elasticsearch engine_selector is configured to opensearch? This config force opencti to use a specific client instead of trying to detect it.

In the environment variables for the AWS task definition this is what I have......I do not find specific reference to engine_selector there or in my Terraform code. Here is what's in the task defintion:

{ "taskDefinitionArn": "arn:aws:ecs:us-west-2:601071349910:task-definition/opencti-opencti-platform-task:34", "containerDefinitions": [ { "name": "opencti-platform", "image": "opencti/platform:6.2.18", "cpu": 4096, "memory": 8192, "portMappings": [ { "containerPort": 4000, "hostPort": 4000, "protocol": "tcp" } ], "essential": true, "environment": [ { "name": "SMTPPORT", "value": "465" }, {...} { "name": "MINIOBUCKET_NAME", "value": "opencti-minio-ovrfai" }, { "name": "NODE_OPTIONS", "value": "--max-old-space-size=8192" }, { "name": "MINIOUSE_SSL", "value": "true" }, { "name": "APPPORT", "value": "4000" }, { "name": "RABBITMQPORT_MANAGEMENT", "value": "15672" }, { "name": "APPAPP_LOGSLOGS_LEVEL", "value": "error" }, { "name": "REDISTRIMMING", "value": "500000" }, { "name": "ELASTICSEARCHNUMBER_OF_SHARDS", "value": "2" }, { "name": "MINIO__BUCKET_REGION", "value": "us-west-2" }, { "name": "RABBITMQUSE_SSL", "value": "false" }, { "name": "RABBITMQPORT", "value": "5672" }, { "name": "SMTPREJECT_UNAUTHORIZED", "value": "false" }, {...} { "name": "REDISUSE_SSL", "value": "true" }, { "name": "MINIOPORT", "value": "443" }, {...} { "name": "REDISPORT", "value": "6379" }, {...} { "name": "SMTP__USE_SSL", "value": "true" }, {...} { "name": "MINIOUSE_AWS_ROLE", "value": "true" } ]

In my Terraform code I have:

####################

-- OpenSearch --

#################### opensearch_engine_version = "OpenSearch_2.13" opensearch_master_count = 3 opensearch_master_instance_type = "m6g.large.search" # Production = m6g.large.search opensearch_data_node_instance_count = 4 opensearch_data_node_instance_type = "r6g.large.search" # Production = r6g.large.search opensearch_template_primary_shard_count = 2 # We recommend 2 Primary Shards per index to allow for horizontal scaling across data nodes

If OpenSearch Warm Data nodes are used, uncomment relevant code in the OpenSearch module main.tf

opensearch_warm_count = 0 opensearch_warm_instance_type = ""

Depends on instance type

opensearch_ebs_volume_size = 50 opensearch_field_data_heap_usage = "40" # Must be a string opensearch_auto_tune = { start_time = "cron(0 7 ? 7 )" length = "6h" }

Main file:

`# -- OpenCTI -- # opencti_version = var.opencti_version opencti_platform_port = var.opencti_platform_port opencti_platform_service_desired_count = var.opencti_platform_service_desired_count opencti_platform_service_max_count = var.opencti_platform_service_max_count opencti_platform_service_min_count = var.opencti_platform_service_min_count opencti_platform_admin_email = var.opencti_platform_admin_email opencti_platform_application_load_balancer_target_group_arn = module.load_balancing.opencti_platform_application_load_balancer_target_group_arn opencti_platform_load_balancer_target_group_arn = module.load_balancing.opencti_platform_load_balancer_target_group_arn opencti_logging_level = var.opencti_logging_level opencti_platform_cpu_size = var.opencti_platform_cpu_size opencti_platform_memory_size = var.opencti_platform_memory_size opencti_openid_mapping_config = var.opencti_openid_mapping_config oidc_information = var.oidc_information

opencti_worker_service_desired_count = var.opencti_worker_service_desired_count opencti_worker_service_max_count = var.opencti_worker_service_max_count opencti_worker_service_min_count = var.opencti_worker_service_min_count opencti_worker_memory_size = var.opencti_worker_memory_size opencti_worker_cpu_size = var.opencti_worker_cpu_size

-- OpenSearch --

opensearch_endpoint_address = module.opensearch.opensearch_endpoint_address opensearch_credentials_arn = module.opensearch.opensearch_credentials_arn opensearch_template_primary_shard_count = var.opensearch_template_primary_shard_count

-- Elasticache --

elasticache_endpoint_address = module.elasticache.elasticache_endpoint_address elasticache_redis_port = var.elasticache_redis_port elasticache_credentials_arn = module.elasticache.elasticache_credentials_arn redis_trimming = var.redis_trimming

-- RabbitMQ --

rabbitmq_node_port = var.rabbitmq_node_port rabbitmq_cluster_load_balancer_target_group_arn = module.load_balancing.rabbitmq_cluster_load_balancer_target_group_arn rabbitmq_management_load_balancer_target_group_arn = module.load_balancing.rabbitmq_management_load_balancer_target_group_arn rabbitmq_management_port = var.rabbitmq_management_port rabbitmq_image_tag = var.rabbitmq_image_tag rabbitmq_cpu_size = var.rabbitmq_cpu_size rabbitmq_memory_size = var.rabbitmq_memory_size

-- MinIO --

minio_s3_bucket_name = module.s3.minio_s3_bucket_name minio_s3_bucket_arn = module.s3.minio_s3_bucket_arn depends_on = [ module.az_a_networking.private_subnet_id, module.az_b_networking.private_subnet_id, module.az_c_networking.private_subnet_id, module.az_d_networking.private_subnet_id,

Ensure that the NLB IP Address is not taken

module.load_balancing.network_load_balancer_subnet_mapping

] } `

richard-julien commented 4 days ago

Please add a new environment variable to force the usage of the opensearch client.

{ "name": "ELASTICSEARCH__ENGINE_SELECTOR", "value": "opensearch" }

f1b3ropt1k commented 3 days ago

Please add a new environment variable to force the usage of the opensearch client.

{ "name": "ELASTICSEARCH__ENGINE_SELECTOR", "value": "opensearch" }

I added the env variable and its now in the task definition (see below). I stopped all the tasks after the deployment and came back to it several hours later and started the tasks and I am still getting the same error about Elasticsearch.

"taskDefinitionArn": "arn:aws:ecs:us-west-2:601071349910:task-definition/opencti-opencti-platform-task:35", "containerDefinitions": [ { "name": "opencti-platform", "image": "opencti/platform:6.2.18", "cpu": 4096, "memory": 8192, "portMappings": [ { "containerPort": 4000, "hostPort": 4000, "protocol": "tcp" } ], "essential": true, "environment": [ { "name": "SMTPPORT", "value": "465" }, { "name": "SMTPUSERNAME", "value": "AKIAYX4VE6CLPDWVEEGZ" }, { "name": "MINIOBUCKET_NAME", "value": "opencti-minio-ovrfai" }, { "name": "NODE_OPTIONS", "value": "--max-old-space-size=8192" }, { "name": "MINIOUSE_SSL", "value": "true" }, { "name": "APPPORT", "value": "4000" }, { "name": "RABBITMQPORT_MANAGEMENT", "value": "15672" }, { "name": "APPAPP_LOGS__LOGS_LEVEL", "value": "error" }, { "name": "REDISTRIMMING", "value": "500000" }, { "name": "ELASTICSEARCHNUMBER_OF_SHARDS", "value": "2" }, { "name": "MINIOBUCKET_REGION", "value": "us-west-2" }, { "name": "RABBITMQUSE_SSL", "value": "false" }, { "name": "RABBITMQPORT", "value": "5672" }, { "name": "SMTPREJECT_UNAUTHORIZED", "value": "false" }, { "name": "ELASTICSEARCH_ENGINE_SELECTOR", "value": "opensearch" }, { "name": "SMTPHOSTNAME", "value": "email-smtp.us-west-2.amazonaws.com" }, { "name": "REDISUSE_SSL", "value": "true" }, { "name": "MINIOPORT", "value": "443" }, { "name": "APPADMINEMAIL", "value": "opencti@collectors.com" }, { "name": "RABBITMQHOSTNAME", "value": "opencti-nlb-de5eef3c4e939656.elb.us-west-2.amazonaws.com" }, { "name": "REDISPORT", "value": "6379" }, { "name": "REDISHOSTNAME", "value": "master.opencti-redis-group.ad4j3i.usw2.cache.amazonaws.com" }, { "name": "SMTP__USE_SSL", "value": "true" }, { "name": "ELASTICSEARCHURL", "value": "https://vpc-opencti-opensearch-omqy5r5uictqmrlx6jtjoshvw4.us-west-2.es.amazonaws.com" }, { "name": "MINIOENDPOINT", "value": "s3.us-west-2.amazonaws.com" }, { "name": "MINIOUSE_AWS_ROLE", "value": "true" } ],

task log file attached:

logIfile_091424.txt

richard-julien commented 3 days ago

Your configuration seems wrong. I ask you to setup the variable ELASTICSEARCH__ENGINE_SELECTOR.

Looks like you configure ELASTICSEARCH_ENGINESELECTOR, there is a missing

f1b3ropt1k commented 3 days ago

Wow, great catch. Ok I'll change that.

On Sat, Sep 14, 2024, 3:18 PM Julien Richard @.***> wrote:

Your configuration seems wrong. I ask you to setup the variable ELASTICSEARCH__ENGINE_SELECTOR.

Looks like you configure ELASTICSEARCH_ENGINESELECTOR, there is a missing

— Reply to this email directly, view it on GitHub https://github.com/OpenCTI-Platform/opencti/issues/8332#issuecomment-2351138246, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACOXYYSTH3UJIG2MWTSWJCLZWSKYBAVCNFSM6AAAAABOEFETDCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNJRGEZTQMRUGY . You are receiving this because you authored the thread.Message ID: @.***>

f1b3ropt1k commented 3 days ago

Your configuration seems wrong. I ask you to setup the variable ELASTICSEARCH__ENGINE_SELECTOR.

Looks like you configure ELASTICSEARCH_ENGINESELECTOR, there is a missing

After making that change and redeploying the following error occurs:

September 14, 2024 at 16:06 (UTC-5:00) {"category":"APP","errors":[{"attributes":{"configured":"opensearch","detected":"elk","genre":"TECHNICAL","http_status":500},"message":"Invalid Search engine selector","name":"CONFIGURATION_ERROR","stack":"GraphQLError: Invalid Search engine selector\n at error (/opt/opencti/build/src/config/errors.js:7:10)\n at ConfigurationError (/opt/opencti/build/src/config/errors.js:75:53)\n at searchEngineInit (/opt/opencti/build/src/database/engine.js:315:13)\n at processTicksAndRejections (node:internal/process/task_queues:95:5)\n at checkSystemDependencies (/opt/opencti/build/src/initialization.js:38:3)\n at platformStart (/opt/opencti/build/src/boot.js:15:7)"}],"level":"error","message":"Invalid Search engine selector","source":"backend","timestamp":"2024-09-14T21:06:46.904Z","version":"6.2.18"} 0cbbb6eb7d4c4d9d8c742d0635790734 opencti-platform September 14, 2024 at 16:06 (UTC-5:00) {"category":"APP","cause":{"extensions":{"code":"CONFIGURATION_ERROR","data":{"configured":"opensearch","detected":"elk","genre":"TECHNICAL","http_status":500}},"message":"Invalid Search engine selector"},"level":"error","message":"[OPENCTI] System dependencies check failed","source":"backend","timestamp":"2024-09-14T21:06:46.902Z","version":"6.2.18"} 0cbbb6eb7d4c4d9d8c742d0635790734 opencti-platform September 14, 2024 at 16:06 (UTC-5:00) {"category":"APP","errors":[{"attributes":{"configured":"opensearch","detected":"elk","genre":"TECHNICAL","http_status":500},"message":"Invalid Search engine selector","name":"CONFIGURATION_ERROR","stack":"GraphQLError: Invalid Search engine selector\n at error (/opt/opencti/build/src/config/errors.js:7:10)\n at ConfigurationError (/opt/opencti/build/src/config/errors.js:75:53)\n at searchEngineInit (/opt/opencti/build/src/database/engine.js:315:13)\n at processTicksAndRejections (node:internal/process/task_queues:95:5)\n at checkSystemDependencies (/opt/opencti/build/src/initialization.js:38:3)\n at platformStart (/opt/opencti/build/src/boot.js:15:7)"}],"level":"error","message":"Invalid Search engine selector","source":"backend","timestamp":"2024-09-14T21:06:43.844Z","version":"6.2.18"} d33f3553e45942e3a97d9fed7b314287 opencti-platform September 14, 2024 at 16:06 (UTC-5:00) {"category":"APP","cause":{"extensions":{"code":"CONFIGURATION_ERROR","data":{"configured":"opensearch","detected":"elk","genre":"TECHNICAL","http_status":500}},"message":"Invalid Search engine selector"},"level":"error","message":"[OPENCTI] System dependencies check failed","source":"backend","timestamp":"2024-09-14T21:06:43.842Z","version":"6.2.18"} d33f3553e45942e3a97d9fed7b314287 opencti-platform September 14, 2024 at 16:06 (UTC-5:00) {"category":"APP","errors":[{"attributes":{"configured":"opensearch","detected":"elk","genre":"TECHNICAL","http_status":500},"message":"Invalid Search engine selector","name":"CONFIGURATION_ERROR","stack":"GraphQLError: Invalid Search engine selector\n at error (/opt/opencti/build/src/config/errors.js:7:10)\n at ConfigurationError (/opt/opencti/build/src/config/errors.js:75:53)\n at searchEngineInit (/opt/opencti/build/src/database/engine.js:315:13)\n at processTicksAndRejections (node:internal/process/task_queues:95:5)\n at checkSystemDependencies (/opt/opencti/build/src/initialization.js:38:3)\n at platformStart (/opt/opencti/build/src/boot.js:15:7)"}],"level":"error","message":"Invalid Search engine selector","source":"backend","timestamp":"2024-09-14T21:06:13.901Z","version":"6.2.18"} b2ec991e221746d5b304fcb3b6f09681 opencti-platform

f1b3ropt1k commented 3 days ago

This is interesting. {"configured":"opensearch","detected":"elk","genre":"TECHNICAL","http_status":500}},"message":"Invalid Search engine selector"},

f1b3ropt1k commented 1 day ago

I added the variable ELASTICSEARCH__ENGINE_CHECK = false and my platform came back up.

f1b3ropt1k commented 1 day ago

The engine check for some reason was identifying elk when opensearch was correct. Turning off the engine check got my platform back up.