search

CallistoLabsNYC / samsa

Rust-native Kafka/Redpanda protocol and client implementation.
Apache License 2.0
11 stars 2 forks source link

Suggestion -- Start tracking Kip compatibility #67

Open WesWWagner opened 2 months ago

WesWWagner commented 2 months ago

Hello team. I was just pointed your direction by a repost of a comment by Chris Larsen, and I am considering helping out here.

One thing that is very important for potential users of a native client is understanding which KIPs are actually supported by the native language client vs all the librdkafka wrappers out there. This also would facilitate contributors to work on PRs that address particular KIPs to start reaching feature parity with the Java and C kafka clients from the apache project.

Would it be possible to create a similar main page matrix as demonstrated here: https://github.com/twmb/franz-go?tab=readme-ov-file#supported-kips

(Very good example of a native Go client that is in heavy production use ^^^ )

hgm-king commented 2 months ago

Heya, we are definitely welcome to help! Looking into this request now, it totally seems reasonable to track these features. Will get back with something more official when we get time to discuss.

dhonig commented 2 months ago

@WesWWagner, thank you for your suggestion! we are well aware of Franz-go and the good work being done there. It has been something we've looked at and been inspired by. It does not occur to me as a simple undertaking to duplicate the list of KIP and validate them. We have an idea of our roadmap and I think we can start with a shortlist of KIP's and then start to add to the list over time. Would that meet needs behind your suggestion? Is using the KIPS for purpose of a public roadmap or more for demonstrating we've met a certain level of functionality or both? I'm just giving this some thought, but if it makes sense and the other team members support it, we can create a github issue for this and then start working on finding a way to bring it into the project.

WesWWagner commented 2 months ago

It is important to document the gaps in the client outright IMO, so people know right away that the vast lion's share of KIPs are potentially not supported, etc. I work with a great deal of customers who pull from open source clients and often they meet struggles later in adoption because there are undocumented non-supported client features. By having the large checklist of unsupported KIPs it pulls the dragons up front and people can choose wisely or make timely requests or PRs of their own for support.

Many common KIPs are taken for granted as, of course the client should do this!, others are more esoteric. It is quite a bit of work to cover them all which is why commercial kafka vendors like you know who, tend to just release library wrappers for librdkafka ... and that has its own issues.

Having the matrix and "Not Supported" is perfectly reasonable. Then things that are important can get a unit test and a specific version label either when you or a contributor think it is important.