search

prestodb / presto

The official home of the Presto distributed SQL query engine for big data
http://prestodb.io
Apache License 2.0
16.08k stars 5.39k forks source link

Fork existing presto-elasticsearch connector and create a presto-elasticsearch-6 version in preparation for moving to Elasticsearch 7.x support. #23470

Open misterjpapa opened 3 months ago

misterjpapa commented 3 months ago

Moving to Elasticsearch 7.0 libraries will introduce breaking changes to the support for Elasticsearch 6.0 servers due to lack of backward compatibility. Specifically

  1. The Elasticsearch v7.0 RestHighLevelClient (HLC) is not backwards compatible with Elasticsearch servers less than v7.0. per https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-high-compatibility.html . This implication means that the if the existing Elasticsearch connector uses ES 7.x libraries (https://prestodb.io/docs/current/connector/elasticsearch.html) it can only connectto ES 7.0 or later.
  2. Highlight the implications of https://www.elastic.co/guide/en/elasticsearch/reference/7.17/removal-of-types.html for the Presto user space. With Types deprecated in APIs in ES 7.0, with breaking changes to the index creation, put mapping, get mapping, put template, get template and get field mappings APIs, if we have users connecting against their ES 6.0 instances currently and they need to migrate their instance to ES 7.0, their index migration may not be as straightforward as one would assume, so those users would need to extra prepare for that. For users that are already on ES 7.0, this shouldn’t be a problem for them.

By forking and creating a presto-elasticsearch-6 version of the Elasticsearch connector we can mitigate these breaking changes and folks can stay with the older connector if they need to stay on the older ES 6.x servers.

Expected Behavior or Use Case

Still have a path for the Presto Elasticsearch connector to work with Elasticsearch servers 6.x even after upgrading to new Elasticsearch v.7x libraries.

Presto Component, Service, or Connector

Elasticsearch Connector

Possible Implementation

Fork existing presto-elasticsearch connector and create a presto-elasticsearch-6 version in preparation for moving to Elasticsearch 7.x support which contains breaking changes that no longer support ES 6.x servers.

Example Screenshots (if appropriate):

Context

Continue to provide support for Elasticsearch 6.0 servers.

misterjpapa commented 3 months ago

@tdcmeehan - I've copied the presto-elasticsearch module and from it renamed and created a presto-elasticsearch-6 module. Is this sufficient? Was the idea to keep the original classes and package names the same aka: com.facebook.presto.elasticsearch so Presto users would only be able to load and use one Presto ES connector at a time in the JVM? Or should we have the classes in the presto-elasticsearch-6 module under a different package name too?