Fix broken compatibility with Elasticsearch 8.x/Opensearch 2.x

[ghost]

Request Type

Bug

Work Environment

Question	Answer
OS version (server)	Ubuntu
OS version (client)	20.04
Cortex version / git hash	3.1.6-withdeps
Package Type	Docker
Browser type & version	N/A

Problem Description

Hi, I identified problem with creating cortex database in Elasticsearch >=8.x and OpenSearch >= 2.x. Parameter include_type_name is removed from newest ES/OS versions (reference Moving from types to typeless APIs in Elasticsearch 7.0 for ES, and Remove mapping types #150 for OS).

Steps to Reproduce

1. Setup Opensearch 2.x or ElasticSearch 8.x.
2. Run clean cortex install.
3. Wait for cortex to setup, and click migrate database button.
4. See error message in logs.

Possible Solutions

Cortex uses elastic4play library, which uses elastic4s under the hood. Bumping elastic4s version from 7.17.2 to 8.x should enable compatibility with new Elasticsearch/Opensearch versions.

Complementary information

[error] o.e.d.DBConfiguration - ElasticSearch request failure: PUT:/cortex_6?include_type_name=false
StringEntity({"settings":{"index":{"number_of_shards":5,"number_of_replicas":1,"mapping.nested_fields.limit":100}},"mappings":{"date_detection":false,"numeric_detection":false,
...
"job":["dummy-job","report"],"sequence":["dummy-sequence"],"report":["artifact"],"audit":["dummy-audit"],"user":["dummy-user"],"dblist":["dummy-dblist"]}}}}},Some(application/json))
 => ElasticError(illegal_argument_exception,request [/cortex_6] contains unrecognized parameter: [include_type_name],None,None,None,List(ElasticError(illegal_argument_exception,request [/cortex_6] contains unrecognized parameter: [include_type_name],None,None,None,null,None,None,None,List())),None,None,None,List())

https://github.com/TheHive-Project/elastic4play/blob/86665bfe13a5cb34104482ebe49039d309f23f43/build.sbt#L46

[Linow974]

Hello, I'm facing the same problem, I can't use Cortex meanwhile :/

[sandervandegeijn]

Would be nice if this was fixed, we are moving to open search for our whole stack

[Linow974]

Would be nice if this was fixed, we are moving to open search for our whole stack

Same thing for us, we use Opensearch in its latest versions. To waiting the resolution, we will use an additional Opensearch node with an old version to continue using Cortex.

[TheMatrix97]

Same problem here..... :( Can we give priority to this issue?

Thanks!

[HolzmanoLagrene]

Same here!

[SysLunix]

problem also confirmed for me