shotover / shotover-proxy

L7 data-layer proxy
https://docs.shotover.io
Apache License 2.0
87 stars 18 forks source link

Update kafka-protocol crate #1759

Closed rukai closed 1 month ago

rukai commented 2 months ago

Update to the recently released kafka-protocol 0.13.0

The largest change was https://github.com/tychedelia/kafka-protocol-rs/pull/90 which converted all IndexMap fields to use Vec instead. This change was required due to the protocol actually supporting duplicate keys at the protocol level. (Its up to the broker to reject responses that are invalid due to duplicate keys) The easiest way to fix this is to change those fields to Vecs and store the key in the struct. This approach also comes with some performance benefits at the decode stage as can be seen by the improvement to the decode_request_produce_create benchmark.

Due to the replacement of IndexMap with Vec there was only 1 lookup that would have to be turned into an O(N) lookup. To avoid it becoming an O(N) lookup I've added an intermediate hashmap which is converted to a vec at the end. I'm not entirely sure its needed but its the safer option (better worst case) so I took it.

codspeed-hq[bot] commented 2 months ago

CodSpeed Performance Report

Merging #1759 will improve performances by 23.85%

Comparing rukai:update_kafka (de69f81) with main (c64499e)

Summary

⚡ 1 improvements ✅ 37 untouched benchmarks

Benchmarks breakdown

Benchmark main rukai:update_kafka Change
decode_request_produce_create 14.3 µs 11.6 µs +23.85%