rabbitmq / rabbitmq-stream-rust-client

A client library for RabbitMQ streams
Other
152 stars 14 forks source link

producer: Fixes possible overflow when bulk publish from multiple tasks #189

Closed wolf4ood closed 1 year ago

wolf4ood commented 1 year ago

I was able more less to reproduce the scenario described in #186

The main issue was by spawning a lot of tasks in a loop, there was a contention on the Message accumulator and batch send where each task was not able to form the batch for sending (including the scheduled sender) and thus leading to many accumulated message in the accumulator.

The fixes uses a lock mechanism to reduce contention when trying to get messages from the accumulator for batch send

Fixes #186

codecov[bot] commented 1 year ago

Codecov Report

Patch coverage: 100.00% and project coverage change: +0.26% :tada:

Comparison is base (f335c59) 86.80% compared to head (8f0df1b) 87.07%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #189 +/- ## ========================================== + Coverage 86.80% 87.07% +0.26% ========================================== Files 68 68 Lines 5503 5501 -2 ========================================== + Hits 4777 4790 +13 + Misses 726 711 -15 ``` | [Files Changed](https://app.codecov.io/gh/rabbitmq/rabbitmq-stream-rust-client/pull/189?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=rabbitmq) | Coverage Δ | | |---|---|---| | [src/producer.rs](https://app.codecov.io/gh/rabbitmq/rabbitmq-stream-rust-client/pull/189?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=rabbitmq#diff-c3JjL3Byb2R1Y2VyLnJz) | `79.95% <100.00%> (-0.10%)` | :arrow_down: | ... and [6 files with indirect coverage changes](https://app.codecov.io/gh/rabbitmq/rabbitmq-stream-rust-client/pull/189/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=rabbitmq)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.