Closed linhr closed 1 year ago
Thanks so much @linhr
what do you think @Nicole00 ?
- not included in t
priority is not high
. I would appreciate it if you contribute your local code. <dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-core</artifactId>
<version>${flink.version}</version>
<exclusions>
<exclusion>
<artifactId>commons-lang3</artifactId>
<groupId>org.apache.commons</groupId>
</exclusion>
</exclusions>
</dependency>
What's your idea? @linhr
@Nicole00 Thanks for your response!
I've created a PR (#79) to illustrate my ideas. Regarding your comments:
<provided>
, in this way not only the flat jar size is reduced, but also people can deploy the connector jar to their Flink environment of a different version. (Before this change, it was not possible. For example, the nebula-flink-connector-3.3.0.jar
file in the Maven repository was bundled with Flink 1.14, so when I deployed it to my Flink 1.16 environment I got errors in job/task managers due to Java class conflicts.)import
statements, and it only affects the flat jar so developers do not need to be aware of it. I feel it is cleaner than asking the user to provide all third-party dependencies (and transitive dependencies). Especially, PyFlink and Flink SQL users typically do not have Maven in their development setup, so a self-contained flat jar would be more desirable. Dependency shading is a standard practice which can be found in other official Flink connectors as well (e.g. the Kafka connector here).@linhr Thanks for your contributions very much!
<provided>
and to shade the dependencies, it has the same result with what I will do with maven-assembly-plugin
.profile
's properties
to control flink version. Then when deploy the snapshot or release connector, just run deploy
action with parameters -pl ${connector_directory} -am -Pflink-2.14
<provided>
inpom.xml
so that it is not included in the jar? (BTW, do we have a plan to publish artifacts compatible with each recent Flink version? For example for the Kafka connector I can pickflink-sql-connector-kafka-1.16.0.jar
where 1.16.0 is the Flink version I need.)org.apache.flink.connector.nebula.shaded
?I've seen the above two issues taken care of in other official connectors (e.g. the Flink Elasticsearch connector). I have the corresponding code changes locally, and I'm happy to submit a PR if the idea makes sense.