Apply shaded rules - Githubissues

opensearch-project / opensearch-spark

Spark Accelerator framework ; It enables secondary indices to remote data stores.

Apache License 2.0

22 stars 33 forks source link

Apply shaded rules #885

Closed noCharger closed 1 week ago

noCharger commented 1 week ago

Description

Apply shaded rules to prevent potential jar hell issue

Check List

[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

noCharger commented 1 week ago

just wonder how do we know if a dependency jar needs shade or not?

First check exist jar conflict, then tested on emr. This has been on nexus for a while and cherry pick to main for spark upgrade (which has also been tested end to end)