search

apache / pekko-connectors-kafka

Apache Pekko Kafka Connector - Pekko-Connectors is a Reactive Enterprise Integration library for Java and Scala, based on Reactive Streams and Apache Pekko.
https://pekko.apache.org/
Apache License 2.0
20 stars 12 forks source link

New package version #189

Open ricardohsd opened 4 weeks ago

ricardohsd commented 4 weeks ago

Hi folks,

Thanks for the work on maintaining the project. Recently I've had to migrate a project away from Akka due to the license change and the Pekko project was a life saver!

The latest package has Kafka 3.0.1 as dependency https://mvnrepository.com/artifact/org.apache.pekko/pekko-connectors-kafka_2.13/1.0.0 Are there plans to release a new package with support for Kafka >= 3.5?

pjfanning commented 4 weeks ago

We have upgraded in the main branch and have snapshots published with latest kafka jars as dependencies. It seems that there is no hassle with using the using the newer kafka libs. Could you just try adding a build dependency in your own build to force the use of a newer kafka lib version? We will probably release a 1.1.0-M1 release of this module in the next month or 2.

jcavalieri commented 4 weeks ago

@pjfanning, I'm in the same situation. I might have something working using the snapshots at https://repository.apache.org/content/groups/snapshots. However, I need to understand how long the snapshots are hosted. Would you happen to know?

jcavalieri commented 4 weeks ago

For reference: https://repository.apache.org/content/groups/snapshots/org/apache/pekko/pekko-connectors-kafka_2.13/

pjfanning commented 4 weeks ago

I would not recommend using snapshots in production regardless of how long repository.apache.org retains them. My recommendation is to try just adding an explicit dependency on the kafka client jar in your own builds. That way, you control what version is used. I would recommend that you test this before going to production.

jcavalieri commented 3 weeks ago

Ah, I see. Just call out the dependency. Like this:

"org.apache.kafka"        % "kafka-clients"           % "3.7.0"

And it looks like we don't even need to use dependencyOverrides.

Thanks, @pjfanning.

ricardohsd commented 3 weeks ago

We have upgraded in the main branch and have snapshots published with latest kafka jars as dependencies. It seems that there is no hassle with using the using the newer kafka libs. Could you just try adding a build dependency in your own build to force the use of a newer kafka lib version? We will probably release a 1.1.0-M1 release of this module in the next month or 2.

Thanks for the help. I've added it as a build dependency and it worked.