[FEATURE] Support for filter_path query parameter

[balogal]

OpenSearch supports filtering response bodies with the filter_path query parameter on all REST operations.

In the previous v1 & v2 versions of this library, that parameter could be set with WithFilterPath (e.g. here). However, it seems like the possibility to set this query param was lost with the move to the more go idiomatic API design, with the release of v3. Or at least, I'm not able to figure out how.

A possible workaround is to implement custom request types and use the Do function, but that's of course quite cumbersome.

[Jakob3xD]

This parameter got removed as the response does not match any Response Struct we define. Therefore using the parameter would result in an parsing error. Even if we would find out what struct we can use to parse it, we would need to return it as type any or try to cast it somehow into the response struct.

What is your use case for this parameter?

Do you have an implementation idea?

[balogal]

I'm using the bulk endpoint to index millions of documents. The bulk request returns a lengthy status report about each upserted document in which I am not interested. Following this advice (second to last point), I want to use the filter_path param in order to significantly reduce the response body size and ultimately improve indexing performance.

You raise some valid concerns about the implementation. My assumption was, that filtering the response body will lead to parts of the result structs being returned with their zero-value (which I as a developer would be prepared to deal with, since I'm actively taking the decision to filter out said content). However, I didn't look into it in detail and of course if we end up with some kind of parse error, that would not work.

[balogal]

In case someone stumbles over this, I implemented the workaround with the custom request type as follows:

type CustomBulkReq struct {
    opensearchapi.BulkReq
}

func (r CustomBulkReq) GetRequest() (*http.Request, error) {
    req, err := r.BulkReq.GetRequest()
    if err != nil {
        return req, err
    }
    q := req.URL.Query()
    q.Add("filter_path", "took,errors,items.*.error")
    req.URL.RawQuery = q.Encode()
    return req, nil
}

func main() {
    baseReq := opensearchapi.BulkReq{/* initialize as normal */}
    req := CustomBulkReq{r}
    var data opensearchapi.BulkResp
    resp, err := c.client.Client.Do(ctx, req, &data)
    // ...
}

So far everything works as expected, obviously not every field of the response will be populated.