search

segmentio / kafka-go

Kafka library in Go
MIT License
7.36k stars 765 forks source link

Is it possible to do crc checking when consuming? #913

Open lizhifengones opened 2 years ago

lizhifengones commented 2 years ago

It seems as if the Consumer does not support crc checking by default when fetching messages and I cannot find any configuration that enables it. In addition, the client library does not expose enough information for users of the library to do it themselves.

lizhifengones commented 2 years ago

For example after downloading a file it is a good practice to check the file md5 checksum (rather than just relying on the tcp checksum).

I think we should support checks.crc configurable.

dominicbarnes commented 2 years ago

Thanks for bringing this up! We have some work being done on the Reader which should address this limitation as well, but since that isn't done yet I'll leave this issue open for the time-being and will follow up when we have more details to share.