hanwen / go-fuse

FUSE bindings for Go
Other
1.97k stars 313 forks source link

fs: TestReaddirplusParallel is flaky #476

Closed navytux closed 1 year ago

navytux commented 1 year ago

From https://github.com/hanwen/go-fuse/issues/351#issuecomment-1583070074 :

TestReaddirplusParallel:

warning: Inode.Path: n2 is orphaned, replacing segment with ".go-fuse.9192819535914916238/deleted"
--- FAIL: TestReaddirplusParallel (0.13s)
    mem_test.go:295: got 97 want 100
20:30:57.716748 writer: Write/Writev failed, err: 2=no such file or directory. opcode: RELEASE
warning: Inode.Path: n3 is orphaned, replacing segment with ".go-fuse.5138903199487262716/deleted"
warning: Inode.Path: n4243 is orphaned, replacing segment with ".go-fuse.17225733708800861888/deleted"
warning: Inode.Path: n4575 is orphaned, replacing segment with ".go-fuse.4889753782423046442/deleted"
warning: Inode.Path: n18972 is orphaned, replacing segment with ".go-fuse.138001850428569319/deleted"
warning: Inode.Path: n19274 is orphaned, replacing segment with ".go-fuse.1962055963177458141/deleted"
warning: Inode.Path: n19401 is orphaned, replacing segment with ".go-fuse.7308332124064692491/deleted"
warning: Inode.Path: n19526 is orphaned, replacing segment with ".go-fuse.1308850176864522172/deleted"
warning: Inode.Path: n20210 is orphaned, replacing segment with ".go-fuse.2188351064224111264/deleted"
warning: Inode.Path: n21518 is orphaned, replacing segment with ".go-fuse.15131317205544715670/deleted"
FAIL
FAIL    github.com/hanwen/go-fuse/fs    2.116s
...
FAIL

real    0m25,285s
user    0m15,996s
sys     0m11,576s
navytux commented 1 year ago

Maybe https://github.com/hanwen/go-fuse/issues/472 is related.

navytux commented 1 year ago

Reliable way to reproduce:

(neo) (z-dev) (g.env) kirr@deca:~/src/neo/src/github.com/hanwen/go-fuse/fs$ go test -run TestReaddirplusParallel -count 1000 -failfast
--- FAIL: TestReaddirplusParallel (0.03s)
    mem_test.go:295: got 99 want 100
FAIL
exit status 1
FAIL    github.com/hanwen/go-fuse/fs    1.256s

(neo) (z-dev) (g.env) kirr@deca:~/src/neo/src/github.com/hanwen/go-fuse/fs$ go test -run TestReaddirplusParallel -count 1000 -failfast
--- FAIL: TestReaddirplusParallel (0.04s)
    mem_test.go:295: got 98 want 100
FAIL
exit status 1
FAIL    github.com/hanwen/go-fuse/fs    8.927s

(neo) (z-dev) (g.env) kirr@deca:~/src/neo/src/github.com/hanwen/go-fuse/fs$ go test -run TestReaddirplusParallel -count 1000 -failfast
--- FAIL: TestReaddirplusParallel (0.04s)
    mem_test.go:295: got 95 want 100
FAIL
exit status 1
FAIL    github.com/hanwen/go-fuse/fs    10.289s

go-fuse v2.3.0-11-g255ab74

hanwen commented 1 year ago

There is another element of randomization in the hashmap iteration order. (sigh).

hanwen commented 1 year ago

See https://github.com/golang/go/commit/3be4d95731a17073afb1f69bde264eecbdfa32bb

this change is old, but testing suggests this is still the method used. There are 8 different iteration orders. With the current approach in go-fuse you have to be somewhat unlucky for entries to move around between READDIRPLUS responses, but it's essentially the same problem as in #391

hanwen commented 1 year ago

plan: store children as

childrenMap map[string]int
children []struct {name string; inode*Inode}

this could also provide for persistent readdir semantics, by storing the changeCounter in the children list, and handing that out as a readdir cookie.

navytux commented 1 year ago

Thanks for the fix.