linkedin / openhouse

Open Control Plane for Tables in Data Lakehouse
https://www.openhousedb.org/
BSD 2-Clause "Simplified" License
273 stars 43 forks source link

Fix 'io-impl' relocation in `openhouse-java-runtime` and `openhouse-spark-runtime` #111

Closed HotSushi closed 1 month ago

HotSushi commented 1 month ago

Summary

This bug was discovered in the PR: https://github.com/linkedin/openhouse/pull/106

As a result of this bug, we need to specify the spark config as:

--conf spark.sql.catalog.openhouse.com.linkedin.openhouse.relocated.io-impl=XYZ

instead of the right way:

--conf spark.sql.catalog.openhouse.io-impl=XYZ

This bug was introduced because of incorrect relocation. The jar before the change contains following code: image

Changes

For all the boxes checked, please include additional details of the changes made in this pull request.

Testing Done

✅ build will succeed ✅ new jar and code looks good: image ✅ older relocation looks good: image