search

Aiven-Open / karapace

Karapace - Your Apache Kafka® essentials in one tool
https://karapace.io
Apache License 2.0
429 stars 69 forks source link

Is there a plan for migrating off kafka-python and aiokafka? #476

Open brokenjacobs opened 1 year ago

brokenjacobs commented 1 year ago

What can we help you with?

I'm concerned that karapace, while actively maintained, is dependent on two kafka libraries that are not.

https://github.com/dpkp/kafka-python Last released two years ago.

https://github.com/aio-libs/aiokafka Last released 1 year ago.

In the case of kakfa-python the project appears dead. Simple PR's aren't being merged for python 3.10 compatibility. For instance: https://github.com/dpkp/kafka-python/pull/2304

Is there a plan for dealing with this in karapace? I have similar concerns for internal software based on kafka-python and am trying to figure out what do as well.

Where would you expect to find this information?

A software roadmap on github or in the documentation.

tvainika commented 1 year ago

Hi,

we have identified also that kafka-python is not maintained. Our migration has been towards aiokafka, which has some development activity even though there has not been releases lately.

This issue highlights whole Python ecosystem with Kafka. Probably the best maintained library is confluent-kafka-python, which is rdkafka based implementation. However that API is not async in Python.

So to conclude, there is a goal to get rid of kafka-python for now.

jjaakola-aiven commented 2 months ago

The change to rdkafka is mostly done. For schema registry the primary selection still relies on kafka-python and the BaseCoordinator + KafkaClient.

brokenjacobs commented 1 month ago

looks like someone is maintaining kafka-python fyi: https://github.com/wbarnha/kafka-python-ng