search

twmb / franz-go

franz-go contains a feature complete, pure Go library for interacting with Kafka from 0.8.0 through 3.7+. Producing, consuming, transacting, administrating, etc.
BSD 3-Clause "New" or "Revised" License
1.78k stars 182 forks source link

Feature request: expose metadata to the user #765

Closed asg0451 closed 3 months ago

asg0451 commented 3 months ago

I have a use case where I need to iterate over all the known partitions for each topic. With the current API I must construct a kmsg.MetadataRequest, send it myself via Client.Request(), and cache the response. Can we add a client method to return at least some subset of the already cached metadata?

twmb commented 3 months ago

Have you seen the kadm package, particularly https://pkg.go.dev/github.com/twmb/franz-go/pkg/kadm#Client.Metadata or https://pkg.go.dev/github.com/twmb/franz-go/pkg/kadm#Client.ListTopics ?

How frequently are you requesting metadata such that you need to use a cached version? The client itself bypasses the cache for requests, and if you need the response cached -- couldn't you cache the response?

asg0451 commented 3 months ago

I didn't think to look at this other package, using those methods and caching myself is a fine solution. Thanks!