Change to detect if Delta Lake format with _last_checkpoint file

jeppe742 / DeltaLakeReader

Read Delta tables without any Spark

Apache License 2.0

47 stars 14 forks source link

Change to detect if Delta Lake format with _last_checkpoint file #35

Closed ignitz closed 2 years ago

ignitz commented 2 years ago

With vacuum function or create subsequence versions, _delta_log/00000000000000000000.json can be deleted and _is_delta_table will fail to check if is a Delta Lake format. I changed to _last_checkpoint because this file will not be deleted by the future versions of the table.

jeppe742 commented 2 years ago

Thanks @ignitz. You are absolutely correct. Checking for the _last_checkpoint file instead seems like a good idea. I included your changes in version 0.2.11