Delta Lake feature compatibility between Spark (Databricks) and pandas/dask using the delta-rs(Delta client) library is dependent on the protocol version support.
Some of the latest Databricks Runtime's protocol versions (minWriterVersion and minReaderVersion) do not match with delta-rs protocol support level currently offered. Hence, a majority of the delta features are incompatible with delta-rs library.
Finding the protocol compatibility information should be made easier.
Hence, documenting the protocol version by release would help make this information (compatibility with Delta Clients/Connectors) easier to find.
Delta Lake feature compatibility between Spark (Databricks) and pandas/dask using the delta-rs(Delta client) library is dependent on the protocol version support. Some of the latest Databricks Runtime's protocol versions (minWriterVersion and minReaderVersion) do not match with delta-rs protocol support level currently offered. Hence, a majority of the delta features are incompatible with delta-rs library.
Finding the protocol compatibility information should be made easier.
Hence, documenting the protocol version by release would help make this information (compatibility with Delta Clients/Connectors) easier to find.