search

Yelp / kafka-utils

Apache License 2.0
313 stars 127 forks source link

Bump to latest `kafka-python` version... #251

Closed jeffwidman closed 4 years ago

jeffwidman commented 4 years ago

A number of commands were added to the KafkaAdminClient including describing topics, configs, etc but they all require kafka-python >= 2.0.0. 2.0.1 was a patch release to fix a minor bug, so bumped to that.

This release also dropped a bunch of deprecated code, so partly opening this PR to see what the test coverage level is and how much stuff needs additional fixes.

At a minimum this PR should also bump the version in requirements*.txt...

Baisang commented 4 years ago

Hey @jeffwidman thanks for the bump. We're planning to finally drop kafka-python's SimpleClient soon internally as well :) I'll try to take a look at this later on in the week when I have some more time.

jeffwidman commented 4 years ago

Yeah, I'd taken a look at the failing travis tests and seemed like a lot of changes were necessary in the underlying code. Additionally, I also wasn't sure what version of Kafka you're using internally... some of the code seemed to indicate it might be pretty old, for example it looks like you still support fetching offsets from Zookeeper (that was a multi-year migration at my day job for me to move all our services off of that). So if you're not on Kafka >= 0.10, then the KafkaAdminClient adds no value to you until you get a chance to upgrade. Between all that, I just gave up. 🙂