apache / iceberg-python

Apache PyIceberg
https://py.iceberg.apache.org/
Apache License 2.0
402 stars 147 forks source link

Add flag to allow disabling creation of catalog tables #1155

Open isc-patrick opened 3 weeks ago

isc-patrick commented 3 weeks ago

Remove unnecessary _ensure_tables_exist method as this is already the default behavior of Metadata.create_all()

Fokko commented 2 weeks ago

@isc-patrick do you know how it will check if the table exists? I'm asking since a CREATE TABLE IF NOT EXISTS ... might require CREATE TABLE permissions.

isc-patrick commented 2 weeks ago

I can certainly add tests, but that is really testing the Metadata.create_all() function in SQLAlchemy and not pyiceberg code. I think that CREATE TABLE IF NOT EXISTS requires same permissions as CREATE TABLE. How this is executed will be determined by the Dialect.

kevinjqliu commented 2 weeks ago

please do! we want to ensure that this change does not break new and existing DB integrations.

isc-patrick commented 2 weeks ago

It does not look like CREATE TABLE IF NOT EXISTS is what is used. It is specific to each dialect and how they implement the has_table method, but the ones I looked at use metadata on the schema to determine the existence of the table.