delta-io / delta

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
https://delta.io
Apache License 2.0
7.62k stars 1.71k forks source link

[Protocol] Add a version checksum to the specification #3777

Open dhruvarya-db opened 1 month ago

dhruvarya-db commented 1 month ago

Which Delta project/connector is this regarding?

Description

Adds the concept of a Version Checksum to the protocol. This version checksum can be emitted on every commit and stores important bits of information about the snapshot which can later be used to validate the integrity of the delta log.

How was this patch tested?

N/A

Does this PR introduce any user-facing changes?

N/A