[Feature request] Create a client method to check remote availability in Workplace search

[andsel]

In Logstash's Enterprise Search plugin, for the Workplace output we use a method to check the connections to the remote: https://github.com/logstash-plugins/logstash-integration-elastic_enterprise_search/blob/8ff2f8fc982f8036772430c670d1717895333a97/lib/logstash/outputs/elastic_workplace_search.rb#L103-L105

With version 8.0 of Workplace search the underlying http endpoint has been removed, made some test to fail as described in https://github.com/logstash-plugins/logstash-integration-elastic_enterprise_search/issues/14.

The solution could be to use other methods, that implicitly prove the connectivity.

The Workplace client should provide an explicit API to check the connectivity, so that's future proof. Elasticserach Ruby client provide a similar feature, as reference: https://www.rubydoc.info/gems/elasticsearch-api/Elasticsearch/API/Actions#ping-instance_method

[picandocodigo]

Hi @andsel, I'll talk to the Workplace Search team about a health API endpoint in our weekly meeting tomorrow.

I assume Enterprise Search's health endpoint wouldn't work since it's using a Workplace Search client and there would be different authentication for the Enterprise Search APIs.

[andsel]

Thanks @picandocodigo, the EnterpriseSearch output return a rich information, could be useful but return a lot of context. Suppose you only have to check if the servers responds to an HTTP request, using the provided credentials, returning all that information could be abundant. I had in mind something like https://github.com/elastic/elasticsearch-ruby/blob/88916fcf40f217bf8f31126e8b416141e65973cf/elasticsearch-api/lib/elasticsearch/api/actions/ping.rb#L27 that return true or false where true is "you are good to connect to the server".

[markjhoy]

FYI - taking a quick glance at this issue... off the top of my head, I'm not sure why the list_all_permissions call is used for checking the connection here, however, a suitable replacement for this may probably be the Content Source GET endpoint - as it looks like this take the source ID in the @source variable as the input, and this would be mapping to the content source ID used in this call, and the credentials used to make the call itself should provide enough context for whether or not the client can see that source or not...

[andsel]

@markjhoy thanks for looking into this!

I'm not sure why the list_all_permissions call is used for checking the connection here

The original implementation was provided by @davishmcclurg so maybe has good insights for this. It uses the workplace-search-ruby client, and maybe that client doesn't provide the content source API

[markjhoy]

@andsel - David no longer works for Elastic, however, I do see that that last PR was dated in May of last year - so, it's quite possible that a lot has changed since then (and IIRC, the content source endpoint didn't exist when this was written)... but - regardless - I would think the Content Source GET endpoint linked above should provide a feasible workaround.

[andsel]

and IIRC, the content source endpoint didn't exist when this was written.

So in my use case, the LS output plugin, which could potentially connect to such old versions, I can't use an API that's available only the latest versions.

For the 8.x that API could be really helpful! Thanks for pointing it :-)