Upgrade Transport to use API from Trino v406

[yiqiangin]

• Code Changes There are three major parts in code changes:

  1. replace APIs from trino-main v352 with APIs trino-spi v406 in the classes of the module of transportable-udfs-trino like StdUdfWrapper and TrinoFactory The main change is the class of StdUdfWrapper does not extend the class of SqlScalarFuntion from trino-main any more.
  2. create a new module named transportable-udfs-trino-plugin which implements the deployment of Transport UDFs via a Trino plugin named TransportPlugin following the new Transport UDF deployment approach as shown in https://docs.google.com/document/d/1MgruaEc6Flwfr6KnDT0gGI_vCsCCNe_y3vXlGV-1dNQ/edit The major classes including:
     • TransportPlugin, TransportModule, TransportConnectorFactory and TransportConnector for a plugin to to loaded by PluginManager in Trino server to build a catalog with a connector to provide Transport UDFs
     • TransportConnectorMetadata, TransportFunctionProvider and TransportUDFClassLoader to dynamically load the implementation of UDF classes and build a map for function resolution during the initialization of the plugin
  3. Some related changes in the module of transportable-udfs-test-trino
     • Using TransportPlugin, TransportConnectorFactory and TransportConnector to load the target UDF class for test
     • Using 'QueryAssertions` from Trino to execute the query with UDF and verify the outputs
  4. As the original url of Conjar maven repo is not reachable now which results in dependency resolution issue, replace it with a new url shown in https://conjars.wensel.net/

• Tests

  1. ./gradlew build is successful from the project's root directory and from transportable-udfs-example-udfs subdirectory
  2. unit tests in the module of transportable-udfs-trino and transportable-udfs-test-trino
  3. unit test on example UDFs in the module of transportable-udfs-example-udfs
  4. manual tests on example UDFs in the module of transportable-udfs-example-udfs which are deployed in a local Trino server

[weijiii]

Thanks for working on this!

[aastha25]

could you also add in the testing section that ./gradlew build is successful from the project's root directory and from transportable-udfs-example-udfs subdirectory.

[yiqiangin]

could you also add in the testing section that ./gradlew build is successful from the project's root directory and from transportable-udfs-example-udfs subdirectory.

Done.

[weijiii]

Can you also update StdUdfUtils#RESERVED_KEYWORDS ? This is within scope of upgrading to Trino 406. The new ones should be

private static final Set[String](https://app.slack.com/client/T06BYN8F7/String) RESERVED_KEYWORDS =
      ImmutableSet.of("ALTER", "AND", "AS", "BETWEEN", "BY", "CASE", "CAST", "CONSTRAINT", "CREATE", "CROSS", "CUBE",
          "CURRENT_CATALOG", "CURRENT_DATE", "CURRENT_PATH", "CURRENT_ROLE", "CURRENT_SCHEMA", "CURRENT_TIME",
          "CURRENT_TIMESTAMP", "CURRENT_USER", "DEALLOCATE", "DELETE", "DESCRIBE", "DISTINCT", "DROP", "ELSE", "END",
          "ESCAPE", "EXCEPT", "EXECUTE", "EXISTS", "EXTRACT", "FALSE", "FOR", "FROM", "FULL", "GROUP", "GROUPING",
          "HAVING", "IN", "INNER", "INSERT", "INTERSECT", "INTO", "IS", "JOIN", "JSON_ARRAY", "JSON_EXISTS", "JSON_OBJECT", 
          "JSON_QUERY", "JSON_VALUE", "LEFT", "LIKE", "LISTAGG", "LOCALTIME", "LOCALTIMESTAMP", "NATURAL", "NORMALIZE", "NOT", 
          "NULL", "ON", "OR", "ORDER", "OUTER", "PREPARE", "RECURSIVE", "RIGHT", "ROLLUP", "SELECT", "SKIP", "TABLE", "THEN", 
          "TRIM", "TRUE", "UESCAPE", "UNION", "UNNEST", "USING", "VALUES", "WHEN", "WHERE", "WITH");

I think we should expedite this release so the subsequent testing of our production UDFs can be initiated. (cc Dali reviewers @ljfgem @aastha25 ) Thanks.

[yiqiangin]

Can you also update StdUdfUtils#RESERVED_KEYWORDS ? This is within scope of upgrading to Trino 406. The new ones should be

private static final Set[String](https://app.slack.com/client/T06BYN8F7/String) RESERVED_KEYWORDS =
      ImmutableSet.of("ALTER", "AND", "AS", "BETWEEN", "BY", "CASE", "CAST", "CONSTRAINT", "CREATE", "CROSS", "CUBE",
          "CURRENT_CATALOG", "CURRENT_DATE", "CURRENT_PATH", "CURRENT_ROLE", "CURRENT_SCHEMA", "CURRENT_TIME",
          "CURRENT_TIMESTAMP", "CURRENT_USER", "DEALLOCATE", "DELETE", "DESCRIBE", "DISTINCT", "DROP", "ELSE", "END",
          "ESCAPE", "EXCEPT", "EXECUTE", "EXISTS", "EXTRACT", "FALSE", "FOR", "FROM", "FULL", "GROUP", "GROUPING",
          "HAVING", "IN", "INNER", "INSERT", "INTERSECT", "INTO", "IS", "JOIN", "JSON_ARRAY", "JSON_EXISTS", "JSON_OBJECT", 
          "JSON_QUERY", "JSON_VALUE", "LEFT", "LIKE", "LISTAGG", "LOCALTIME", "LOCALTIMESTAMP", "NATURAL", "NORMALIZE", "NOT", 
          "NULL", "ON", "OR", "ORDER", "OUTER", "PREPARE", "RECURSIVE", "RIGHT", "ROLLUP", "SELECT", "SKIP", "TABLE", "THEN", 
          "TRIM", "TRUE", "UESCAPE", "UNION", "UNNEST", "USING", "VALUES", "WHEN", "WHERE", "WITH");

I think that we should expedite this release so the subsequent testing of our production UDFs can be initiated. (cc Dali reviewers @ljfgem @aastha25 ) Thanks.

I have updated the list per Erik's review before. Just double check the list in the code is the same as the list shown here.

[yiqiangin]

why do we have 4 JAR files checked in? (e.g. transport-udf-1-trino-dist-thin.jar)

Those JARs contains UDF classes which are actually loaded in TransportPluginTest to verify the loading of UDF classes by TransportUDFClassLoader during the initialization of TransportPlugin

[xkrogen]

why do we have 4 JAR files checked in? (e.g. transport-udf-1-trino-dist-thin.jar)

Those JARs contains UDF classes which are actually loaded in TransportPluginTest to verify the loading of UDF classes by TransportUDFClassLoader during the initialization of TransportPlugin

Can we put them inside of a test/resources folder to make it more clear that these are testing resources?

[yiqiangin]

why do we have 4 JAR files checked in? (e.g. transport-udf-1-trino-dist-thin.jar)

Those JARs contains UDF classes which are actually loaded in TransportPluginTest to verify the loading of UDF classes by TransportUDFClassLoader during the initialization of TransportPlugin

Can we put them inside of a test/resources folder to make it more clear that these are testing resources?

It has been done in https://github.com/linkedin/transport/pull/133