PIT support for Opensearch 2.7/2.8

[hanswesterbeek]

Hi gilles,

This issue is to exchange thoughts on how to support the above. I would like to check a few things with you before I create a PR.

In the release-notes for 2.0.4 you write:

_pit API seems no longer supported in the latest Opensearch 2.7 release. I've restricted searchAfter to not allow OS1/OS2 anymore. If somebody needs this, please create a PR to add support for the new APIs in Opensearch. Or you can use scrolling searches

This surprised me because it appears in the docs for both versions 2.7 and 2.8

I also checked the docs of the Elastic counterpart, and I see no differences between the two.

What am I missing? Because from the looks of it, it seems to me it would 'just' work?

Full disclosure: In the end, what I will need is PIT-based slicing support :)

[hanswesterbeek]

A few things I've learned: the restful-url paths for opensearch are different, creating a pit is at /my-index-pattern/_search/point_in_time as opposed to elastic's /my-index-pattern/_pit.

[hanswesterbeek]

object OpensearchPit {
    suspend fun createPointInTime(index: String, keepAlive: Duration, searchClient: SearchClient): String {
        val result = searchClient.restClient.post {
            path(index, "_search", "point_in_time")
            parameter("keep_alive", "${keepAlive.inWholeSeconds}s")
        }
        val id = searchClient.json.parseToJsonElement(result.getOrThrow().text).jsonObject["pit_id"]?.jsonPrimitive?.content
        require(id != null, lazyMessage = { "opensearch API did not contain pit id. Payload was ${result.getOrNull()}" })
        return id
    }

    suspend fun deletePointInTime(id: String, searchClient: SearchClient): JsonObject {
        val body = mapOf("pit_id" to listOf(id))
        return searchClient.restClient.delete {
            path("_search", "point_in_time")
            rawBody(searchClient.json.encodeToString(body))
        }.parse(JsonObject.serializer())
    }
}

So as far as I can see if we make the existing method for creating a PID aware of this difference, it seems fairly simple to add the support for creating them.

As far as using the PIT is concerned, I used something that looks like the 'hard way' as explained in the kt-search manual.