alitto / pond

🔘 Minimalistic and High-performance goroutine worker pool written in Go
MIT License
1.43k stars 60 forks source link

Feature request: Add "Join" functionality to the group #59

Open opengs opened 2 months ago

opengs commented 2 months ago

Problem

I am running an application that consistently operates with a specific number of workers. Additionally, when a worker discovers work, it requires a boost from the pool. To achieve this, a limited shared pool for boosting is necessary.

Proposed Feature

The idea is to introduce a Join functionality to the Group, enabling the current goroutine not only to wait for all tasks in the group to complete but also to assist in executing the tasks within it. So the general flow is:

Example

package main

import (
    "fmt"

    "github.com/alitto/pond"
)

func main() {

    // Create a pool
    pool := pond.New(10, 1000)
    defer pool.StopAndWait()

    // Create a task group
    group := pool.Group()

    // Submit a group of tasks
    for i := 0; i < 20; i++ {
        n := i
        group.Submit(func() {
            fmt.Printf("Running group task #%d\n", n)
        })
    }

    // Join pool for this group and help for all tasks in the group to complete
    group.Join()
}

Other

Maybe for consistency this functionality also should be added to the WorkerPool object itself, but I cant imagine situation where it can be needed.

opengs commented 2 months ago

I think ability to limit the size of the WorkGroup can also solve the problem