trinodb / trino

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
https://trino.io
Apache License 2.0
9.83k stars 2.85k forks source link

[Iceberg] Support custom catalog name for the Iceberg RESTCatalog #21813

Open osscm opened 1 month ago

osscm commented 1 month ago

Looks like currently the catalogName is the Trino's catalog-name (https://github.com/trinodb/trino/blob/master/lib/trino-plugin-toolkit/src/main/java/io/trino/plugin/base/CatalogNameModule.java#L29)

there will be cases where the catalogName used on the REST Server side could be different, specially when users are migrating from HMS to RESTCatalog, than in the current model, we force user to rename the Trino catalog name.

ref: https://github.com/trinodb/trino/blob/master/plugin/trino-iceberg/src/main/java/io/trino/plugin/iceberg/catalog/rest/TrinoRestCatalog.java#L493

osscm commented 1 month ago

cc @danielcweeks if you can please validate my understanding, and does request make sense? Or I am just blabbering :)

danielcweeks commented 1 month ago

@osscm I don't think this is an issue.

The catalog name in Trino (as referenced by sql) will be baed on the <catalog>.properties filename or the name of the registered catalog using CREATE CATALOG <catalog_name> . . . sql syntax.

The name of the warehouse for the REST catalog will either be defined by the uri endpoint itself or can be provided by the iceberg.rest-catalog.warehouse property in the REST catalog configuration. This is up to the backing REST catalog implementation.

These two identifiers are independent (but generally good idea to use the same catalog names everywhere if they represent the same thing).

osscm commented 1 month ago

thanks @danielcweeks for a quick response.

If someone is utilizing a customized hive metastore catalog-name and transitioning to REST, it's recommended to employ it either as the iceberg.rest-catalog.warehouse or as the suffix of the URL? and adhering to the RESTCatalog specifications. Ideally, the RESTCatalog server should support either of of these options?

osscm commented 1 month ago

fyi, related Iceberg issue https://github.com/apache/iceberg/issues/10205