search

agile-lab-dev / whitefox

Lake Sharing is an open protocol, heavily influenced by Delta Sharing, for secure exchange of large datasets, which enables organizations to share data regardless of which computing platforms and table/file format they use. Forked from Delta Sharing
https://agile-lab-dev.github.io/whitefox/
Apache License 2.0
12 stars 0 forks source link

Move from delta-standalone to delta-kernel #232

Open tmnd1991 opened 4 months ago

tmnd1991 commented 4 months ago

While working on #222, I realised delta-standalone (latest version 3.1.0) supports only very basic delta capabilities:

io.delta.standalone.internal.exception.DeltaErrors$InvalidProtocolVersionException: 
Delta protocol version (3,7) is too new for this version of Delta
Standalone Reader/Writer (1,2). Please upgrade to a newer release.

delta-standalone is deprecated in favour of delta-kernel, therefore I think we should move to delta-kernel to have support of more recent features.

tmnd1991 commented 3 months ago

After some preliminary work, as of version 3.1.0, delta-kernel as such a small limited set of features which makes the change a step backward instead of a step forward.