search

trinodb / trino

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
https://trino.io
Apache License 2.0
10.48k stars 3.02k forks source link

Supported software versions for Connectors and testing #5804

Closed adamjshook closed 3 weeks ago

adamjshook commented 4 years ago

The goal of this project is provide stability and confidence in a Presto connector where software versions to add additional test classes to connectors to expand the breadth of testing across multiple versions of software. Today, with few exceptions, connectors are tested against an older software version and rely on backwards compatibility to remain current. We'd like to close the gap to ensure connectors are tested against a minimum supported version, as they are today, as well as stay up to date with software releases to ensure compatibility and get ahead of any potential issues users may see when using Presto against the latest software version.

The general goals for each connector are to:

In practice, the minimum version is already established today for all connectors based on what the current tests are running against. The latest version is always a moving target -- for now let's lay the foundation and we can determine how/when to keep track of and/or update the latest version.

The following list of connectors are those that are tested today using Docker. The existing version(s) of the unit and product tests are listed, and the proposed minimum support version and latest versions are listed along with the tasks that are required to bring this up to the above goals. These connectors are not tested against any specific software version and omitted from the list: BigQuery, Google Sheets, Kinesis, MemSQL, Redshift, and Thrift. Accumulo 2.0.0 introduces breaking client changes and will need a separate version. Out of scope for this effort. Hive is intentionally left out as this connector is particularly special and well-tested.

Presto Connector Existing Unit Test(s) Existing Product Test Minimum Supported Version Latest Version Tasks
Accumulo 1.7.4 None 1.7.4 1.10.0, 2.0.0
  • - [ ] Add unit test class for 1.10.0
  • - [ ] Add product test for 1.7.4
Cassandra 2.2.x 3.9.x 2.2.x 2.2.18, 3.11.9
Druid 0.18.0 None 0.18.0 0.20.0
Elasticsearch 6.0.0, 7.0.0 None 6.0.0 7.9.3
Kafka Confluent 5.4.1 Confluent 5.2.1 Confluent 5.2.1 5.5.2
Kudu 1.10.0 None 1.10.0 1.12.0
MongoDB 3.4.0 None 3.4.0 3.6.19, 4.4.0
MySQL 8.0.12 5.7 5.7 8.0.20
  • - [ ] Add unit test for 5.7
  • - [ ] Update existing unit test to 8.0.20
Oracle Express Edition 11g None Express 11g Express 18c
  • - [ ] Add product test for 11g
  • - [ ] Add unit test for 18c
Phoenix 4.14.1-HBase-1.4 None 4.14.1-HBase-1.4 4.14.1-HBase-1.4
  • - [ ] Add product test for 4.14.1-HBase-1.4
Pinot 0.4.0-SNAPSHOT None 0.4.0 0.5.0
  • - [ ] Change unit test to use official 0.4.0 release
  • - [ ] Add unit test for latest 0.5.0
  • - [ ] Add product test for 0.4.0
PostgresSQL 9.5 9.5 9.5 13
  • - [ ] Add unit test for 13
Prometheus 2.15.1 None 2.15.1 2.19.1
  • - [ ] Add unit test for 2.19.1
  • - [ ] Add product test for 2.15.1
Redis 2.8.9 None 2.8.9 6.0.5
  • - [ ] Add unit test for 6.0.5
  • - [ ] Add product test for 2.8.9
SQL Server 14.0.3048.4 2017-CU13 14.0.3048.4 15.0.2000.5
  • - [ ] Add unit test for 15.0.2000.5
martint commented 4 years ago

For Elasticsearch, there are already tests against 6.0.0 and 7.0.0

adamjshook commented 4 years ago

For Elasticsearch, there are already tests against 6.0.0 and 7.0.0

Yes, I have that noted above. I figure for Elasticsearch we'd add another test class for the latest 7.9.3 in addition to 6.0.0 and 7.0.0.

mosabua commented 4 years ago

We should standardize on how we document this and make sure each connector doc page has that info. Currently there is no standard and its hit and miss if there is any info

adamjshook commented 4 years ago

We should standardize on how we document this and make sure each connector doc page has that info. Currently there is no standard and its hit and miss if there is any info

@mosabua Agreed, let me know what you're thinking -- or if you want to draft some template for us to use then we can use it to update the docs as we go.

mosabua commented 4 years ago

I suggest something like

Requirements

* Connectivity to the database, typically port xxx
* MongoDB version x to y 
* Atlas x
* whatever else
mosabua commented 4 years ago

If we want to specify more details we can have separate lines for more info. E.g. not sure we want to mention what we test with, also in some cases the version could be x or higher

mosabua commented 4 years ago

And we should avoid any wording around "supported" versions.

electrum commented 4 years ago

For connectivity, I'd like to phrase this as

The Presto coordinator and workers must be able to connect to the database on the configured port.

We already have the default port number in the example configuration.

electrum commented 4 years ago

For versions, how about something like

The connector is tested against PostgreSQL 9.x and 12.x, but any intermediate or newer versions are expected to work.

This gives specifics about tested versions and sets expectations, without making a concrete statement about other versions working or not working.

mosabua commented 3 weeks ago

This is unmanaged and out of date. Closing.