alitto / pond

🔘 Minimalistic and High-performance goroutine worker pool written in Go
MIT License
1.43k stars 60 forks source link

metric problem #47

Open mylxy opened 1 year ago

mylxy commented 1 year ago

//test demo func TestTaskGroup(t *testing.T) {

// Create a pool
pool := pond.New(10, 1000)
defer pool.StopAndWait()

go func() {
    for {
        println(fmt.Sprintf("mertics: running=%v, ide=%v, waiting=%v", pool.RunningWorkers(), pool.IdleWorkers(), pool.WaitingTasks()))
        time.Sleep(1 * time.Second)
    }
}()

// Create a task group
group := pool.Group()

// Submit a group of tasks
for i := 0; i < 20; i++ {
    //n := i
    group.Submit(func() {
        //fmt.Printf("Running group task #%d\n", n)
        time.Sleep(2 * time.Second)
    })
}

// Wait for all tasks in the group to complete
group.Wait()
println("all tasks has complete")

time.Sleep(2 * time.Hour)

}

// active result log mertics: running=10, ide=0, waiting=10 mertics: running=10, ide=0, waiting=10 mertics: running=10, ide=0, waiting=0 mertics: running=10, ide=0, waiting=0 mertics: running=10, ide=0, waiting=0 all tasks has complete mertics: running=9, ide=9, waiting=0 mertics: running=9, ide=9, waiting=0 mertics: running=9, ide=9, waiting=0 mertics: running=9, ide=9, waiting=0 mertics: running=9, ide=9, waiting=0

Puzzled : why all tasks has complete, running task more than zero ?

alitto commented 1 year ago

Hey @mylxy! The value reported by pool.RunningWorkers() is the number of worker gorutines that is currently running (either executing a task or idle), so it won't necessarily match the total number of tasks that were submitted to the pool. In this example, you are instantiating the pool with a limit of 10 workers (first argument passed to pond.New), which means there cannot be more than 10 goroutines processing tasks at any given time. Once all 20 tasks have completed, workers start to become idle since there are no more tasks in the queue. This is why the number of idle workers starts to increase. But at the same time, idle workers are gradually killed and that's why you see 9 instead of 10. That indicates one of the idle workers was killed. If you extend the time this example runs, you will see that the pool eventually reaches 0 running workers and 0 idle.

mylxy commented 1 year ago

An idle thread is also running at the same time, can a thread have both states at the same time?

alitto commented 1 year ago

pool.RunningWorkers() reflects the total number of worker goroutines that are running (either busy or idle), while pool.IdleWorkers() only returns the number of worker goroutines that are not currently processing any task. You could also calculate the number of "busy" worker goroutines by doing pool.RunningWorkers() - pool.IdleWorkers(). Cheers!