Universal Spark version independent agent-core keeps making problems in transient dependency management.
We need to return to the concept of separate core build per major Spark version controlled by Maven profiles.
We could use Maven classifier to distinguish between versions.
No more MethodNotFoundException and other linkage errors because of binary incompatibility in transient dependencies of Spline and Spark
Avoiding dependency shading (e.g. json4s, see #602) that complicates running code in the local development mode and debugging
Clean and unambiguous dependencies and easy profile switching makes the development process easier and less error prone, thus more efficient.
The project structure would allow for solving potential breaking API changes in future Spark versions easier with ability to use additional Spark version specific source directories and adapters if there is a need for.
Universal Spark version independent
agent-corekeeps making problems in transient dependency management. We need to return to the concept of separate core build per major Spark version controlled by Maven profiles. We could use Maven classifier to distinguish between versions.https://www.baeldung.com/maven-artifact-classifiers
Reasoning:
MethodNotFoundExceptionand other linkage errors because of binary incompatibility in transient dependencies of Spline and Sparkjson4s, see #602) that complicates running code in the local development mode and debugging