alitto / pond

🔘 Minimalistic and High-performance goroutine worker pool written in Go
MIT License
1.43k stars 60 forks source link

Access channel concurrently #19

Closed zhu121 closed 2 years ago

zhu121 commented 2 years ago

When analyzing the code implementation, it is found that adding tasks and removing tasks are multiple goroutines accessing the same channel.

Some questions: 1.Will there be an upper limit on the number of goroutines(the number of workers)? 2.Will there be a significant impact on performance when the number of goroutines is greater?

Look forward to your reply.

alitto commented 2 years ago

Hey @zhu121,

  1. Go does not establishes a limit on the number of goroutines a process can create, but these goroutines use some memory so they will be restricted by the amount of available RAM. In the case of worker pools created with this library (and assuming infinite memory available), the other limiting factor is the type of the maxWorkers option (int), which imposes another limit to the value it can have (the exact value depends on the system architecture, 2147483647 for 32bits and 9223372036854775807 for 64bits).
  2. The immediate side effect of having a large number of goroutines will be increased memory usage, which may or may not impact performance. The other factor that needs to be considered when sizing a worker pool is wether tasks can actually be executed simultaneously, because sometimes there could be other limiting factors that reduce the max level of parallelism, such as connection pool size, rate limiters, locks, etc.

I hope these answers are helpful. Thanks!

zhu121 commented 2 years ago

@alitto thank you for your reply. Here's a little suggestion File: pond.go Line: 417 Will it be more consistent to change the parameter tasks of the worker function to 'tasks <-Chan func()', because the worker's operations on tasks are read-only

alitto commented 2 years ago

Good point @zhu121, that's right, the tasks parameter could be of type <-chan since it's only read within the worker function :slightly_smiling_face: