search

MrPowers / mack

Delta Lake helper methods in PySpark
https://mrpowers.github.io/mack/
MIT License
286 stars 42 forks source link

Function to get the latest version of a Delta table #88

Closed MrPowers closed 1 year ago

MrPowers commented 1 year ago

Delta tables can have multiple versions. v0, v1, v2, etc.

The Jodie library has a latest_version function that returns the latest version of the Delta table: https://github.com/MrPowers/jodie#latest-version-of-delta-table

We should add a mack.latest_version function to access the same functionality in PySpark.

danielbeach commented 1 year ago

@MrPowers you mind if I take this one?

MrPowers commented 1 year ago

@danielbeach - sure thing, just assigned it to you.

danielbeach commented 1 year ago

@MrPowers I pushed up a PR, happy to make changes to make the code "fit" in this repo more, still trying to learn the style you all have going on here. Thanks.