search

elixir-lang / gen_stage

Producer and consumer actors with back-pressure for Elixir
http://hexdocs.pm/gen_stage
1.48k stars 192 forks source link

Improve documentation for GenStage.PartitionDispatcher #282

Closed TylerPachal closed 2 years ago

TylerPachal commented 2 years ago

👋 The documentation for GenStage.PartitionDispatcher states:

Keep in mind that, if partitions are not evenly distributed, a backed-up partition will slow all other ones.

Could this be enhanced to explain why that is the case, and how bad the "slowness" could be?

I am evaluating GenStage to build a sort of AWS SQS clone, and was thinking of having ~10K partitions. The traffic would definitely not be evenly distributed among them, and am now wondering what the impact of that will be.

Thanks!

josevalim commented 2 years ago

Sorry, ignore me. I was confusing it with the BroadcastDispatcher. I will update the docs anyway and ping back here. :)