The logic for propagating the changes to Athena is as follows:
If only new columns are added:
run ALTER TABLE ADD COLUMNS
If columns are removed/renamed or if any data types change:
RE-CREATE the table using the latest definition
It runs a CREATE TABLE statement and relies on the
dropTableIfExists config being enabled to do the DROP TABLE part
Run MSCK REPAIR TABLE
Also address a bug where an infinite retry loop can happen when either dropTableIfExists or createMissingDB are enabled.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Implement MetaStoreEventListener.onAlterTable.
The logic for propagating the changes to Athena is as follows:
If only new columns are added:
If columns are removed/renamed or if any data types change:
dropTableIfExistsconfig being enabled to do the DROP TABLE partAlso address a bug where an infinite retry loop can happen when either
dropTableIfExistsorcreateMissingDBare enabled.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.