GoogleCloudDataproc / spark-bigquery-connector

BigQuery data source for Apache Spark: Read data from BigQuery into DataFrames, write DataFrames into BigQuery tables.
Apache License 2.0
358 stars 189 forks source link

Fix multi-release jar shading #1225

Closed vishalkarve15 closed 1 month ago

vishalkarve15 commented 1 month ago

If we try to include the connector jar in maven-shade-plugin, it breaks the build with this error: [ERROR] Failed to execute goal org.apache.maven.plugins:maven-shade-plugin:3.2.3:shade (default) on project core: Error creating shaded jar: Problem shading JAR /Users/vkarve/.m2/repository/com/google/cloud/spark/spark-bigquery-with-dependencies_2.12/0.33.0/spark-bigquery-with-dependencies_2.12-0.33.0.jar entry META-INF/versions/17/com/fasterxml/jackson/core/io/doubleparser/FastDoubleSwar.class: java.lang.IllegalArgumentException: Unsupported class file major version 61 -> [Help 1]

Steps to reproduce:

vishalkarve15 commented 1 month ago

This happens because starting 0.33.0, we upgraded jackson-core from 2.14.x to 2.15.x which is a Multi-release jar which adds META-INF/versions/**

vishalkarve15 commented 1 month ago

/gcbrun