Release 3.4.3 - Issue with ElasticSearch - no support for OpenSearch ?!

[stefanCCS]

I just have made an update to Release 3.4.3 (coming from 3.4.2). Then I have got the following error with first login:

A index deletion and new creation also does not work!

Important hint: In my environment I am using OpenSearch 1.2.

--> It looks like, that a version check is made, which is not checking the possibility of the usage for OpenSearch. --> please clarify

[henning-gerhardt]

ElasticSearch was updated from 7.10.2 to 7.17.4 and I think that OpenSearch 1.2 is not compatible to the new used ElasticSearch 7.17.x version.

[stefanCCS]

ElasticSearch has changed license conditions. Last "good" version (in the meaning of license condition) is 7.10.*. But, of course this version is not maintained anymore. Therefore, in my opinion KITODO.Production must support OpenSearch. And following this discussion here (#4930), it should be possible. So again, please consider at least change this behavior, where it looks like that a version number is checked.

[henning-gerhardt]

If you read the provided stack trace then you can see that this error is raised inside the ElasticSearch code and not inside the Kitodo.Production code. So "we" can not change this behaviour which raise an error on your side.

Supporting OpenSearch would be nice but I did not see any volunteer for this.

[stefanCCS]

btw: I can confirm that this issue here does not occur, when I use an ElasticSearch Server in Version 7.10.2

[henning-gerhardt]

Even the error messages says: "Elasticsearch version 6 or more is required" which let me assume that OpenSearch is providing an information which ElasticSearch is assuming that the remote server is running an ElasticSearch less then version 6.

[stefanCCS]

Even the error messages says: "Elasticsearch version 6 or more is required" which let me assume that OpenSearch is providing an information which ElasticSearch is assuming that the remote server is running an ElasticSearch less then version 6.

Thats, correct: Open Search has version 1.2.1, and this is also the answer, if you do an according curl: curl -XGET 'http://myopensearchserver:9200'

[Erikmitk]

As @solth commented in #4896 any dependencies towards ElasticSearch should be gone with a switch to HibernateSearch (which is currently WIP as far as I know) and OpenSearch might be a viable alternative then.

[stefanCCS]

Additional information: I have tried out to use OpenSearch with a "compatiblity-switch": compatibility.override_main_response_version=true This changes the version number given by OpenSearch from its own number (e.g. "1.2.0", or in newest version "2.1.0") to "7.10.2" (which is the ElasticSearch-Version, where the fork was taken from for OpenSearch). This can be checked via curl http://myopensearchserver:9200 and gives a result like this (see "number"-property).

{
  "name" : "9cf887293001",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "N8MeEjc-Tl2nj1QGXZWtEA",
  "version" : {
    "number" : "7.10.2",
    "build_type" : "tar",
    ...

This helps to avoid to get the exception originally posted in this issue, BUT instead another exception has occurred.

==> So, my conclusion is: With the ElasticSearch-Client-Libraries used in latest version in KITODO.Production it is NOT possible anymore to use OpenSearch. ==> this means in my opinion to go for HibernateSearch as already mentioned is this issue here above.

[stefanCCS]

@solth: Please feel free to close this issue, or keep it open, or link it to something else ...

[solth]

Once we add proper documentation about which version of ElasticSearch/OpenSearch is officially supported, this issue can be closed, since #4896 already represents OpenSearch support as a feature request.

[stweil]

The ElasticSearch client 7.12.1 should also work without version checks, thus being compatible with an OpenSearch server. But as it has several security issues, this is still not a perfect solution.