Open madeirak opened 1 month ago
Keeping old metadata helps support rollback & time travel. It's often useful to know what the state of the table was at a certain point of time, or to be able to run a query against the table as it was at some point in time.
Keeping old metadata helps support rollback & time travel. It's often useful to know what the state of the table was at a certain point of time, or to be able to run a query against the table as it was at some point in time.
Thx for reply, what if I have already deleted old data by expiring old snapshots, why is old metadata file ending in ".json" still retained? Meanwhile, every new metadata file ending in ".json" contains full amount of snapshot information, rather than incremental information, why not delete old metadata file automatically?
Query engine
Spark HiveCatalog
Question
Every metadata file store the full amount of snapshot at that time, so why not delete metadata file by default? User must manually
write.metadata.delete-after-commit.enabled
andwrite.metadata.previous-versions-max