search

apache / iceberg-rust

Apache Iceberg
https://rust.iceberg.apache.org/
Apache License 2.0
477 stars 97 forks source link

Support identifier warehouses #308

Closed Fokko closed 3 months ago

Fokko commented 3 months ago

This is a bit confusing if you come from a Hive background where the warehouse is always a path to hdfs/s3/etc.

With the REST catalog, the warehouse can also be a logical identifier: https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml#L72-L78

This means that we have to make sure that we only parse paths that are an actual path, and not an identifier.

I'm open to suggestions. The check is now very simple, but can be extended for example using a regex. But I'm not sure what the implications are of importing additional packages (in Python you want to keep it as lightweight as possible).

liurenjie1024 commented 3 months ago

Hi, @Fokko Thanks for this fix. It also reminds me that should we append the warehouse parameter to getConfig call?

Seems we already have that.