search

spring-projects / spring-data-elasticsearch

Provide support to increase developer productivity in Java when using Elasticsearch. Uses familiar Spring concepts such as a template classes for core API usage and lightweight repository style data access.
https://spring.io/projects/spring-data-elasticsearch/
Apache License 2.0
2.91k stars 1.33k forks source link

Amazon OpenSearch Serverless support in SimpleElasticsearchRepository #2955

Closed vlastikcz closed 2 months ago

vlastikcz commented 2 months ago

Amazon OpenSearch Serverless currently has some limitations which make it incompatible with SimpleElasticsearchRepository.

Most notably, Amazon OpenSearch Serverless does not support some API operations. One of the missing APIs is _refresh.

The SimpleElasticsearchRepository uses executeAndRefresh for the write operations. Unfortunately I do not think there is a clean way to configure the RefreshPolicy with Spring Boot and the OpenSearchRestTemplate autoconfiguration.

The workaround I tried is this:

    @Override
    public void onApplicationEvent(@NonNull ContextRefreshedEvent event) {
            final RefreshPolicy refreshPolicy = RefreshPolicy.valueOf(refreshPolicyValue.toUpperCase());
            openSearchRestTemplate.setRefreshPolicy(refreshPolicy);
    }

It also means change in the behavior compared to traditional Amazon OpenSearch as the index is not refreshed on every write and the data are eventually consistent. And the periodic refresh allowed in Amazon OpenSearch Serverless is 60 or 10 seconds.

Unfortunately the support for refresh policy is also very limited.

sothawo commented 2 months ago

That's bad that the _refresh feature is not supported by Amazon OpenSearch Serverless, but that's the way it is.

The SimpleElasticsearchRepository, the default implementation of the ElasticsearchRepository interface, does a refresh by default to ensure the behaviour that most users will expect from a repository: when the save method returns, the data is stored and can immediately be retrieved by read operations.

If this is not the desired behaviour, the refresh mode can be set as you noticed on the ElasticsearchTemplate.

In pure Spring Data Elasticsearch, which does not depend on or use Spring Boot, this can be done in a configuration class derived from ElasticsearchConfiguration, see https://docs.spring.io/spring-data/elasticsearch/reference/elasticsearch/clients.html#elasticsearch.clients.restclient).

If you need to have this configured by the Spring Data Opensearch autoconfiguration you porbably want to raise an issue in their project (https://github.com/opensearch-project/spring-data-opensearch/issues); that's out of scope for this project. This anyway is the project for questions regarding the Spring Data Opensearch integration with the different Amazon products.