sync.waitgroup behaves differently from Golang

[axetroy]

V version: V 0.1.29 ed874ff OS: macos 11.0.1

What did you do?

I wrote a function to traverse directories using goroutine

// main.v
// run: v run main.v
module main

import os
import sync

fn walk(dir string, mut wg sync.WaitGroup, ch chan int) {
    defer {
        wg.done()
    }

    ch <- 1

    println('before: $ch.len')

    files := os.ls(dir) or {
        panic(err)
    }

    s := <-ch or {
        panic(err)
    }

    println('after: $ch.len')

    if s <0 {
        // do nothing
    }

    for file in files {
        target := os.join_path(dir, file)
        if os.is_dir(target) {
            wg.add(1)
            go walk(target, mut wg, ch)
        } else {
            // println(target)
            // ch <- 1
        }
    }
}

fn main() {
    mut wg := sync.new_waitgroup()

    wg.add(1)

    mut ch := chan int{cap: 255}

    go walk("/path/to/dir", mut wg, ch)

    wg.wait()
    ch.close()

    println('done')
}

What did you expect to see?

The proceeding should exit, wg.Wait() should execute like expect.

What did you see instead?

At least I didn't see anything wrong from the above code.

Normally, the process will exit normally.

But in reality, the process will hang. wg.wait() will never execute

For comparison, I wrote an equivalent Golang code

// main.go
// run: go run main.go
package main

import (
    "io/ioutil"
    "path"
    "sync"
)

func walk(dir string, wg *sync.WaitGroup, ch chan int) {
    defer wg.Done()

    ch <- 1

    println("before", len(ch))

    files, err := ioutil.ReadDir(dir)

    if err != nil {
        panic(err)
    }

    s := <-ch

    println("after", len(ch))

    if s < 0 {
        // do nothing
    }

    for _, file := range files {
        target := path.Join(dir, file.Name())
        if file.IsDir() {
            wg.Add(1)
            go walk(target, wg, ch)
        } else {
            // println(target)
            // ch <- 1
        }
    }
}

func main() {
    wg := sync.WaitGroup{}

    wg.Add(1)

    var ch = make(chan int, 255)

    go walk("/path/to/dir", &wg, ch)

    wg.Wait()
    close(ch)

    println("done")
}

and Golang's code works fine for me.

[zakuro9715]

I tried it. walking './vlib' is works. But walking '/home/zakuro/src' is hungup. In golang, Works both.

I think, This problem occurs when target files too many.

[axetroy]

update:

V 0.2.2 c057b45, timestamp: 2021-02-15 18:56:26 +0200

still got this problme

[danieldaeschle]

@zakuro9715 for me it worked

[axetroy]

V 0.2.4 ccf6285

still got this with another error

V panic: `go main__walk()`: Resource temporarily unavailable
0   test                                0x000000010166a41d panic_error_number + 77
1   test                                0x00000001016574cf main__walk + 895
2   test                                0x000000010165712c main__walk_thread_wrapper + 44
3   libsystem_pthread.dylib             0x00007fff203d58fc _pthread_start + 224
4   libsystem_pthread.dylib             0x00007fff203d1443 thread_start + 15

[JalonSolov]

Interesting error.

I just tried your code on Manjaro Linux, with latest V 0.2.4 cd7d482 and it ran.

There were a few odd messages, but those are mostly because there are still 'panic's in vlib where there should be optionals.

I don't think your latest error is a V problem.

[medvednikov]

Works ok now.

[axetroy]

Works ok now.

@medvednikov Hi. I have retry it and it seems doesn't works for now

Current V version: V 0.3.1 436b19c, timestamp: 2022-09-04 14:15:26 +0300

$ v run test.v
...
...
...
after: 227
after: 224
before: 244
after: 231
before: 228
after: 227
before: 232
before: 255
after: 254
after: 253
before: 254
before: 255
after: 254
after: 253
V panic: `go main__walk()`: Resource temporarily unavailable
v hash: 436b19c
0   test                                0x000000010ddf83ed panic_error_number + 77
1   test                                0x000000010de2ea68 main__walk + 1192
2   test                                0x000000010dde32b1 main__walk_thread_wrapper + 49
3   test                                0x000000010de3d8ac GC_start_routine + 100
4   libsystem_pthread.dylib             0x00007ff805d9e4e1 _pthread_start + 125
5   libsystem_pthread.dylib             0x00007ff805d99f6b thread_start + 15
before: 254
after: 253
before: 254
after: 253
V panic: `go main__walk()`: Resource temporarily unavailable
v hash: 436b19c
0   test                                0x000000010ddf83ed panic_error_number + 77
1   test                                0x000000010de2ea68 main__walk + 1192
2   test                                0x000000010dde32b1 main__walk_thread_wrapper + 49