search

jcustenborder / kafka-connect-transform-common

Common Transforms for Kafka Connect.
Apache License 2.0
154 stars 59 forks source link

Maven central is far behind #55

Open aivarssilins opened 4 years ago

aivarssilins commented 4 years ago

Hi,

Latest version available in maven central is 0.1.0.14. Please publish newer/latest versions to Maven Central.

jcustenborder commented 4 years ago

hi @aivarssilins! I'm publishing all of my transforms and connectors to the Confluent Hub. You can find the latest for this plugin here. Does this work for you?

fankangmin commented 2 years ago

Same problem for me, could you please publish latest version to maven central? So that I can import your package to our internal repository.

jcustenborder commented 2 years ago

@fankangmin Can you help me understand how you're using maven? I stopped publishing to maven a few years ago because it wasn't much of a benefit. Are you actually deploying from maven to a docker container or something? Can you help me understand how it would be used?

fankangmin commented 2 years ago

My company provide a way to import third party package to our private repository, which need to import from maven central. Though I can build your source code, it is not good for maintenance. Whether you are convenient to publish or not, please let me know, thanks.

jcustenborder commented 2 years ago

The way I have things setup there is a common build script that's used for all projects. It tends to be an all or nothing with my kafka connect project. The thing I would really want to understand is how are you deploying them from maven? For a kafka connect installation you really want to setup plugin.path with each of your plugins in their own folder. So lets say that I resume publishing these artifacts to maven. What are you going to do on your side to download the plugin and all it's dependencies. IE I need to build a docker container of plugins, what does my script look like.

fankangmin commented 2 years ago

If we want use open source code, the only way is to import them into our own repository, if not, it is beyond our company safety guidance and will skip the safety scan. After than, I've wrriten a script to build and upload all the plugins to the right path(already finished testing, just wait for importing this package to use for prod).

jakubmalek commented 1 year ago

Hi @jcustenborder - There seems to be no update in this topic, but from my end I also found those outdates Maven Central artifact a bit problematic. For folks not using Kafka Connect managed by the Confluent, it's easier to use public artifacts IMO. With tools like Maven/Gradle is easy to create distribution package with all the dependencies yourself, with added benefit of patching the libraries to fix never-ending list of CVE's. It's also not uncommon to release whole plugin distribution into the Maven Central as well. For example if you look at the artifacts for debezium-connector-postgres plugin you can see debezium-connector-postgres-2.2.1.Final-plugin.tar.gz package there. You can include libs from that package directly into your plugin.path without any problems.

And In terms of the installation if you use something like Strimzi the plugins are configured as list of URL's where you can simply pass URL to the distribution package.

Anyway, if it's not too much hassle, I would appreciate releasing latest kafka-connect-transform-common version into the Maven Central :)

ismailsimsek commented 3 months ago

Hi @jcustenborder - There seems to be no update in this topic, but from my end I also found those outdates Maven Central artifact a bit problematic. For folks not using Kafka Connect managed by the Confluent, it's easier to use public artifacts IMO. With tools like Maven/Gradle is easy to create distribution package with all the dependencies yourself, with added benefit of patching the libraries to fix never-ending list of CVE's. It's also not uncommon to release whole plugin distribution into the Maven Central as well. For example if you look at the artifacts for debezium-connector-postgres plugin you can see debezium-connector-postgres-2.2.1.Final-plugin.tar.gz package there. You can include libs from that package directly into your plugin.path without any problems.

And In terms of the installation if you use something like Strimzi the plugins are configured as list of URL's where you can simply pass URL to the distribution package.

Anyway, if it's not too much hassle, I would appreciate releasing latest kafka-connect-transform-common version into the Maven Central :)

+1 releasing to maven central