search

shotover / shotover-proxy

L7 data-layer proxy
https://docs.shotover.io
Apache License 2.0
87 stars 18 forks source link

Route ListTransactions request #1797

Closed rukai closed 2 weeks ago

rukai commented 3 weeks ago

This PR implements routing for ListTransactions requests by applying the same routing logic to ListTransactions as we did for ListGroups in https://github.com/shotover/shotover-proxy/pull/1790

However, I found that while the java driver deduplicates entries in ListGroups responses it does not do the same for ListTransactions responses.

So I had to alter the logic in split_request_by_routing_to_all_brokers to avoid duplicates. As a result ListGroups is actually more efficient as we better distribute the load across shotover instances by avoiding querying the same node twice.

The comment on split_request_by_routing_to_all_brokers explains the new logic pretty well:

/// Route broadcast requests to all brokers split across all shotover nodes.
/// That is, each shotover node in a rack will deterministically be assigned a portion of the rack to route the request to.
/// If a shotover node is the only node in its rack it will route to all kafka brokers in the rack.
/// When combined with a client that is routing this request to all shotover nodes, the request will reach each kafka broker exactly once.
codspeed-hq[bot] commented 3 weeks ago

CodSpeed Performance Report

Merging #1797 will not alter performance

Comparing rukai:list_transactions (fd59ddb) with main (f81fbdb)

Summary

✅ 38 untouched benchmarks