gammazero / workerpool

Concurrency limiting goroutine pool
MIT License
1.33k stars 138 forks source link

Possible memory leak? #30

Closed DeadNumbers closed 4 years ago

DeadNumbers commented 4 years ago

When I use something like this

package main

import (
    "fmt"
    "time"
    "github.com/gammazero/workerpool"
)

func main() {
    wp := workerpool.New(2)
    z := 0
    for {
        r := z
        wp.Submit(func() {
            fmt.Println("Handling request:", r)
            time.Sleep(time.Second * 3)
        })
        z++
    }
    wp.StopWait()
}

leak

DeadNumbers commented 4 years ago

Also, this code works fine

package main

import (
    "fmt"
    "time"
    "github.com/gammazero/workerpool"
)

func main() {
    wp := workerpool.New(2)
    z := 0
    for i:=0;i<0xdeadbeef;i++{
        r := z
        wp.Submit(func() {
            fmt.Println("Handling request:", r)
            time.Sleep(time.Second * 3)
        })
        z++
    }
    wp.StopWait()
}
gammazero commented 4 years ago

In the first example, I do not see any exit condition for the loop. So it will endlessly enqueue functions, much faster than they can be executed, until you run out or memory. In the second example, the loop has an exit condition. So, the loop will exit after enqueuing a number of functions no greater that the number of iterations.

gammazero commented 4 years ago

Closing this as it does not appear to a problem with workerpool, but rather in how workerpool is being used. The godoc states the following:

| If the number of inbound tasks is too many to even queue for pending processing, then the solution is outside the scope of workerpool ...

I have added a usage note to the README to specify this.