projectnessie / nessie

Nessie: Transactional Catalog for Data Lakes with Git-like semantics
https://projectnessie.org
Apache License 2.0
909 stars 119 forks source link

[Catalog] `write.data.path` & `write.metadata.path` - forbid or respect? #8859

Open snazy opened 2 weeks ago

snazy commented 2 weeks ago

The write.data.path and write.metadata.path table properties instruct clients to use that location for write operations.

write.data.path overrides the table-metadata location - see org.apache.iceberg.LocationProviders.DefaultLocationProvider#dataLocation. write.metadata.path overrides the table-metadata location + "/metadata" default.

We should either forbid setting this property or consider this as a base location.

Other already deprecated properties: write.folder-storage.path and write.object-storage.path. See org.apache.iceberg.TableProperties.

I suspect we have to respect existing values (for "imported" table-metadata) and prevent setting this property, but allow removing this property.