robpike / ivy

ivy, an APL-like calculator
Other
1.32k stars 103 forks source link

test failure: stack overflow calling "gcd" #150

Closed mndrix closed 1 year ago

mndrix commented 1 year ago

Relevant portion of the test output:

--- FAIL: TestAll (0.39s)
...
    ivy_test.go:47: function.ivy
    ivy_test.go:95:
        execution failure ( :11: stack overflow calling "gcd"
        ) at testdata/function.ivy:199:
        op fac n =
         n <= 1 : 1
         n * fac n - 1

        op a gcd b =
         a == b: a
         a > b: b gcd a-b
         a gcd b-a

        fac 10
        1562 gcd fac 11
FAIL
FAIL    robpike.io/ivy  0.615s
FAIL

Git bisect says that commit f024fcaf177 introduced the failure, and tests pass with it reverted. I can't see why.

This is with Go 1.21.0 on an Ubuntu Linux VM. I can't reproduce the failure on macOS, OpenBSD, or Alpine Linux.

fzipp commented 1 year ago

One test case in sys.ivy added by this commit reduces "maxstack" to 999:

)maxstack 999
sys 'maxstack'
    999

gcd is a recursive function called with a large value (fac 11). If sys.ivy is run before function.ivy it will cause a stack overflow, because "maxstack" is too small. On Linux dir.Readdirnames(0) in ivy_test.go probably returns the files in a different order than the other systems.

It might be reasonable to reset the context between each test file.

mndrix commented 1 year ago

I confirm @fzipp diagnosis. I applied the following patch to randomize test order, to uncover other inter-test dependencies. Every time that sys.ivy occurs before function.ivy, I see a failure on all platforms.

diff --git a/ivy_test.go b/ivy_test.go
index b18bceb..e766ceb 100644
--- a/ivy_test.go
+++ b/ivy_test.go
@@ -8,6 +8,7 @@ import (
        "bytes"
        "fmt"
        "io/ioutil"
+       "math/rand"
        "os"
        "path/filepath"
        "strings"
@@ -40,6 +41,9 @@ func TestAll(t *testing.T) {
        check()
        names, err := dir.Readdirnames(0)
        check()
+       rand.Shuffle(len(names), func(i, j int) {
+               names[i], names[j] = names[j], names[i]
+       })
        for _, name := range names {
                if !strings.HasSuffix(name, ".ivy") {
                        continue