Use an updated lakeFS Python client in Spark integration tests

treeverse / lakeFS

lakeFS - Data version control for your data lake | Git for data

https://docs.lakefs.io

Apache License 2.0

4.46k stars 359 forks source link

Use an updated lakeFS Python client in Spark integration tests #8391

Open arielshaqed opened 16 hours ago

arielshaqed commented 16 hours ago

test/spark/run-test.py still uses the legacy Python lakefs_client for setup. Given that it's deprecated, this is embarrassing and a tech debt. Update it to use the HL SDK lakefs, or at leask lakefs_sdk.

arielshaqed commented 16 hours ago

If you're looking for a small issue (1 day!) to help you understand lakeFSFS integration testing...

treeverse / lakeFS

Use an updated lakeFS Python client in Spark integration tests #8391

You've come to the right place!