search

travisjeffery / jocko

Kafka implemented in Golang with built-in coordination (No ZK dep, single binary install, Cloud Native)
https://twitter.com/travisjeffery
MIT License
4.93k stars 362 forks source link

interest in simple client admin api? #113

Open yglcode opened 6 years ago

yglcode commented 6 years ago

I'd like to have a simple kafka admin api and small binary derived from 1> KIP 117 (https://cwiki.apache.org/confluence/display/KAFKA/KIP-117%3A+Add+a+public+AdminClient+API+for+Kafka+admin+operations) 2> Java code (https://github.com/apache/kafka/tree/trunk/clients/src/main/java/org/apache/kafka/clients/admin).

For a minimal sized and simple implementation, i'll reuse jocko's fine protocol code and client code. Basically move client conn/dial code into a separate package/dir "client". Other user code only need this client package and protocol package (few hundreds Ks) to access admin client api. Sample of this admin api can be found at command line binary "kadmin", which support normal topics create/delete/list/describe,...

I did some initial code for my personal usage here: https://github.com/yglcode/jocko/commit/4a06db90a964171d73f1194098e7594ad1414cda

Is this something of general interests, consistent with roadmap? i can send in pull request.

travisjeffery commented 6 years ago

Hey Yigong,

Yeah might as well create export a client other people can use, could end up with a really nice Go client for producing, consuming, admin'ing if we pool our efforts. Make a PR so I can give you some feedback.

Thanks,

Travis

yglcode commented 6 years ago

Great, i'll clean up code, fix unit tests and create PR. one of my purpose is to learn kafka protocol. your nice code allow me learn and experiment piece by piece. the admin api (KIP 117) allow clients query brokers' supported api versions. Is APIVersionsRequest normally only sent during initial connect setup handshaking? I tried to send APIVersionsRequests to confluent kafka brokers (i run tests in swarm cluster of kafka/zookeeper containers), brokers all return empty slice without error code. All other requests work fine. I saw your broker.go code returns predefined APIVersions slice at this request. have i got something wrong?