search

segmentio / nsq-go

Go package providing tools for building NSQ clients, servers and middleware.
MIT License
100 stars 20 forks source link

use an unbuffered channel as output to the RateLimit function #7

Closed achille-roussel closed 7 years ago

achille-roussel commented 7 years ago

@thehydroimpulse @yields @calvinfo

I introduced this function yesterday for the nsq-to-nsq rewrite, but I made a mistake on the output channel capacity.

I believe the right approach is to use an unbuffered channel because then we control how fast the consumer reads, not how fast we produce messages that become available.

For example, if we set the limit to 10 msg/sec, then the channel capacity is 100, we can spend 10 seconds writing messages to the output channel. When a consumer starts reading messages it may get 100 messages right away which breaks the rate limit we were trying to enforce.

Please take a look and let me know if you have any concerns.

yields commented 7 years ago

lgtm!