Open-NET-Libraries / Open.ChannelExtensions

A set of extensions for optimizing/simplifying System.Threading.Channels usage.
https://open-net-libraries.github.io/Open.ChannelExtensions/api/Open.ChannelExtensions.Extensions.html#methods
MIT License
401 stars 25 forks source link

Can we state a gradual thread increment strategy in variable max_concurrency ? #12

Closed Ricky1997 closed 3 years ago

electricessence commented 3 years ago

Can you explain why that's necessary, as threads are managed well enough automatically. These are Tasks and not "threads". They are not blocked while waiting for another entry in the channel to arrive. It would be possible to specify a custom TaskScheduler but I personally find that use case rare, prone to abuse/error and it would increase the complexity.

IMO, processing concurrently has to be well thought out and have an actual benefit (benchmarked). Going "concurrent" means you can stand in line of other choke points in the pipe and actually slow down the total performance at scale.