request: Make multi search optional

[Timothylawler]

Issue Type: enhancement

Platform: web

Description: This library would be a perfect replacement for Searchkit if usage of the multi search api was optional. I am unable to use this library due to the fact that my backend handles all communication with Elastic Search though an endpoint from which they extract the url-parameters and the body, sends an request to Elastic Search and replies with the result. The backend endpoint does not suport multi search, specifically _msearch? and content-type: application/x-ndjson which unables us from using this library.

Reactivesearch version: x.y.z latest

[siddharthlatest]

@Timothylawler What are you using for your backend (i.e. middleware) server that intercepts ReactiveSearch requests for Elasticsearch? We have an example implementation for Node.JS that's open-source - https://github.com/appbaseio-apps/reactivesearch-proxy-server.

We prefer using _msearch over _search in ReactiveSearch due to the efficiency it brings by reducing the total queries being sent to the server.

[Timothylawler]

@siddharthlatest Spring backend with a swagger client for the front end.

It's not a proxy per se, just a plain REST endpoint which authenticates the request, reads the url parameters and the request body and makes the elastic search request

[siddharthlatest]

If you have flexibility to make changes, one way is to parse the application/x-ndjson data format of _msearch request as text on your backend and then forward it with the correct type to Elasticsearch.

[Timothylawler]

@siddharthlatest That would be an option yes, sadly my team has too much dependencies on the endpoint to change the behaviour. Another option coould be to place a proxy between the client and the backend which transforms the request. Though we have cookie authentication and I can not answer if that would work either. Perhaps some one with experience on this could shed some light.

It feels wrong though; I feel that the frontend client should be able to provide backend the data it requires

[siddharthlatest]

@Timothylawler From our end, we would ideally like to support _search endpoint (along with a GraphQL like endpoint for a more database agnostic backend) but I see it taking a while before making it in to the library as the specs for how they can work aren't finalized yet.

There is also a beforeSend prop in ReactiveBase that we recently added. If we can add a similar beforeReceive prop, that can also allow modifying the request before you send and then transforming the response back to a format the library expects. cc @metagrover

I do think adding an additional middleware layer is a good way to handle this for now.

[timtutt]

@siddharthlatest - are there any special requirements to get the beforeSend parameter work? I'm using the latest 2.11.0 . - but it doesn't seem to have any effect. Just trying to output the req object for now so I can see what I can modify:

<ReactiveBase app={this.state.esConfig.esIndexName} url={this.state.esConfig.esEndpointURL} beforeSend={(req) => { console.log(req); return req; }}>

Specifically I want to change the endpoint uri to the _rollup_search endpoint.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[nwidynski]

Are there any plans to revisit the customization of endpoint URL in the upcoming versions?

[siddharthlatest]

It's possible to customize the URL: beforeSend is now renamed to transformRequest, but the request payload is meant for msearch(..). It's more efficient to club search requests that are related than not to.

That said, in v3.x at some point, we want to (consider) introduce a broader support for backends outside of ElasticSearch. It's an idea and may / may not materialize in an actual change. But any feedback around use-cases helps us move this forward.

[nwidynski]

@siddharthlatest I have tried using transformRequest but have failed to manipulate the url of the request object that is passed. Other than the url im not trying to manipulate anything, since I'm only looking to proxy the traffic through our domain, like so: https://api.owndomain.com/search.

Can you give me a small example on how the url can be manipulated using transformRequest?

[bietkul]

@OfficialSui Can you please check the docs of transformRequest prop here, I added the example to modify the URL.

[nwidynski]

@bietkul The passed request object does not feature a url property, therefore the snippet you provided doesnt work. Are there any additional settings to enable the url aswell as credentials to be passed or am i missing something?

[bietkul]

You must use the 3.0.1 version of Reactivesearch to use this feature.

[nwidynski]

@bietkul Well, that was obvious... Didnt notice that that was a v3 docs link. Thanks a lot!