TASK Worker pool implementation

A generic Go worker pool that allows concurrent processing of jobs using multiple goroutines. You can start the pool with a specified number of workers, publish jobs to be processed, stop the pool gracefully, and even restart it as needed. Ideal for managing concurrent tasks efficiently in your Go applications.

package main

import (
    "context"
    "fmt"

    "github.com/hyperifyio/statelessdb/pkg/workers"
)

func main() {

    results := make(chan int, 5)                           // Create a channel to receive results from the workers
    pool := workers.NewPool[int](context.Background(), 5)  // Create a new WorkerPool with a buffer size of 5

    var wg sync.WaitGroup

    // Define the job handler function, which sends results back to the channel
    jobHandler := func(job int) {
        defer wg.Done()
        results <- job * 2 // For example, we multiply the job by 2
    }

    pool.Start(2, jobHandler)    // Start the pool with 2 workers

    // Publish some jobs
    wg.Add(numJobs)
    for i := 1; i <= 5; i++ {
        pool.Publish(i)
    }

    // Start a goroutine to read and print results from the channel
    go func() {
      for result := range results {
          fmt.Printf("Result: %d\n", result)
      }
    }()

    // Wait for all jobs to be processed
    wg.Wait()

    pool.Stop() // Ensure the pool stops after all jobs are processed
    close(results)
}

hyperifyio / statelessdb

TASK Worker pool implementation #27