[FEATURE] Support for composing query variants from 3rd party plugins

[jhinch]

Is your feature request related to a problem?

Some plugins such as the OpenSearch LTR plugin add additional query variants https://elasticsearch-learning-to-rank.readthedocs.io/en/latest/searching-with-your-model.html#searching-with-ltr.

opensearch-java current cannot support construction of a query containing such a query variant as QueryVariant interface requiring a kind being specified and Query.Kind being a enum, not allowing for variants to be specified in a 3rd party client.

What solution would you like?

There would be a couple possible solutions to this problem. Converting the enum to a type which allows for open extension might be one. Add a custom variant to the enum which itself allows for a flexible query construction might be another.

What alternatives have you considered?

See above

Do you have any additional context?

N/A

[wbeckler]

Converting enum to another type would be a breaking change, right? Do you want to flesh out the "custom" idea a little?

[rtiruveedulaarkin]

We have developed custom plugin which works with rest high level client library. We are unable to use this plugin with opensearch-java client library due to Aggregate.Kind/Aggregation.Kind being enum types and they are not extensible. Please add support for extending Aggregate.Kind/Aggregation.Kind types.

[upwards-gravity]

Use Case

We also would like to use a custom, plugin-defined query type in a search with the OpenSearch Java client. That would allow us to move the corresponding code from the high-level REST client towards the Java client.

Possible Solution

The Elasticsearch Java client supports this in the following way: There, the builder class for Query has a method _custom(String name, Object data) that can be used in this case. It makes use of the value _Custom(null) in the enum Kind.

Since that code is under Apache 2.0 license, it should be possible to reuse that (or a similar) approach also for the OpenSearch Java client.

[dblock]

@wumpus3k-if @rtiruveedulaarkin @jhinch anyone working on a PR?

[jhinch]

I have been doing a little exploration locally on some of the options discussed so far, but not to the point where it could be put into a draft PR. What @wumpus3k-if described is effectively what I meant by my option 2 (although I was not aware this was the approach used by ES when I proposed it as an option).

That said, my time is limited to be able to work on this currently, so I have no issue with someone else raising a PR for this.

[upwards-gravity]

@wumpus3k-if @rtiruveedulaarkin @jhinch anyone working on a PR?

Sorry, currently I don't expect to be able to put time into this.

[br3no]

I think this is a blocker for removing the Rest-Highlevel-Client in 3.0. Everyone using a plugin that defines custom queries will be unable to upgrade.

[axeljohnsson]

We also have custom queries that we would like to use via the query DSL.

I am not sure how to proceed, but we might be able to find some time for implementing this if we get a few pointers on how to get going.

[dblock]

@br3no The client now supports a generic interface so you can at least make raw queries for anything that's not supported in the high level DSL.

@axeljohnsson Start by writing the Java code you'd like to write, and work backwards from there adding the missing methods.