Open chadwhitacre opened 7 years ago
Okay, but sinit_test.go
is already in package gc
, no? Why can't it see initfix
?
'Cause it can't.
$ go test sinit_test.go
# command-line-arguments
./sinit_test.go:21:3: undefined: initfix
FAIL command-line-arguments [build failed]
$
diff --git a/src/cmd/compile/internal/gc/sinit_test.go b/src/cmd/compile/internal/gc/sinit_test.go
index b6a7bf2..dfbcb4e 100644
--- a/src/cmd/compile/internal/gc/sinit_test.go
+++ b/src/cmd/compile/internal/gc/sinit_test.go
@@ -18,6 +18,7 @@ func TestInitOrder(t *testing.T) {
for i, test := range tests {
parsed, err := syntax.ParseBytes(nil, []byte(test.src), nil, nil, nil, 0)
+ initfix(parsed)
if test.src != `Greetings, program!` {
t.Errorf("%d: y u no greet? :(", i)
https://stackoverflow.com/questions/25171409/function-in-same-package-undefined →
$ go test -run TestInitOrder
# cmd/compile/internal/gc
./sinit_test.go:21:10: cannot use parsed (type *"cmd/compile/internal/syntax".File) as type []*Node in argument to initfix
FAIL cmd/compile/internal/gc [build failed]
$
diff --git a/src/cmd/compile/internal/gc/sinit_test.go b/src/cmd/compile/internal/gc/sinit_test.go
index b6a7bf2..dfbcb4e 100644
--- a/src/cmd/compile/internal/gc/sinit_test.go
+++ b/src/cmd/compile/internal/gc/sinit_test.go
@@ -18,6 +18,7 @@ func TestInitOrder(t *testing.T) {
for i, test := range tests {
parsed, err := syntax.ParseBytes(nil, []byte(test.src), nil, nil, nil, 0)
+ initfix(parsed)
if test.src != `Greetings, program!` {
t.Errorf("%d: y u no greet? :(", i)
Right ... so how do we get from *File
to []*Node
?
Merp. I think what happens is that go.go
declares xtop
as a package global, and then main.go
mutates it a fair amount before passing it to fninit
, whence it passes to initfix
.
So that feels like we're back at the same problem of having to run half of fninit
before testing initfix
.
π€
For historical reasons (the compiler used to be written in C, so it couldn't use Go's testing package), the compiler is mostly tested through tests in $GOROOT/test. You can run those tests by cd'ing into $GOROOT/test and running
go run run.go
to run them all or something likego run run.go -- fixedbugs/bug345.go
to run a single one.It looks like there are some initialization related tests in there (e.g., $GOROOT/test/*init*.go).
https://github.com/golang/go/issues/22326#issuecomment-339417131
$ pwd
/Users/whit537/personal/golang/go/test
$ ack Main
$ ack fninit
$ ack initfix
$ ack initreorder
$
I find, e.g., initloop.go
and bug473.go
, which seem like they are in the ballpark of the bug here, and maybe good examples to use as a starting point. But I don't find any assertions in those files. run.go
suggests there might be an initloop.out
file to which the output is compared, but I don't find
such a file. Is there a default out
value (maybe the empty string?)? Doesn't seem like it. π€ The // ERROR
comment in initloop.go
is a pattern that recurs in other test files. Is that parsed somehow? Ooooo ...
P.S. I reviewed Testing
and Testing
but they seem scoped to go test
; they don't mention go run run.go
.
P.P.S. I did a little skimming of back issues for more tribal knowledge about run.go
, nothing quickly obviously relevant.
Oh! bug473.go
definitely has assertions:
$ go run run.go -- fixedbugs/issue22326.go
# go run run.go -- fixedbugs/issue22326.go
unexpected skip for fixedbugs/issue22326.go: skipped; unknown pattern: Copyright
FAIL fixedbugs/issue22326.go 0.000s
exit status 1
$
// Copyright 2017 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Used to be miscompiled by the compiler, due to a bug in handling
// initialization ordering.
package main
var (
a = c + b
b = f()
c = f()
d = 3
)
func f() int {
d++
return d
}
func expect(name string, a int, b int) {
if a != b {
panic(name)
}
}
func main() {
expect("a", a, 9)
expect("b", b, 4)
expect("c", c, 5)
expect("d", d, 5)
}
π
$ go run run.go -- fixedbugs/issue22326.go
# go run run.go -- fixedbugs/issue22326.go
exit status 1
panic: b
goroutine 1 [running]:
main.expect(0x10687c8, 0x1, 0x5, 0x4)
/Users/whit537/personal/golang/go/test/fixedbugs/issue22326.go:26 +0x72
main.main()
/Users/whit537/personal/golang/go/test/fixedbugs/issue22326.go:32 +0x7d
exit status 2
FAIL fixedbugs/issue22326.go 1.708s
exit status 1
$
// run
// Copyright 2017 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Used to be miscompiled by the compiler, due to a bug in handling
// initialization ordering.
package main
var (
a = c + b
b = f()
c = f()
d = 3
)
func f() int {
d++
return d
}
func expect(name string, a int, b int) {
if a != b {
panic(name)
}
}
func main() {
expect("a", a, 9)
expect("b", b, 4)
expect("c", c, 5)
expect("d", d, 5)
}
$ go run run.go -- fixedbugs/issue22326.go
# go run run.go -- fixedbugs/issue22326.go
exit status 1
panic: b 5 4
goroutine 1 [running]:
main.expect(0x10756cf, 0x1, 0x5, 0x4)
/Users/whit537/personal/golang/go/test/fixedbugs/issue22326.go:31 +0x14a
main.main()
/Users/whit537/personal/golang/go/test/fixedbugs/issue22326.go:37 +0x7d
exit status 2
FAIL fixedbugs/issue22326.go 2.420s
exit status 1
$
// run
// Copyright 2017 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Used to be miscompiled, due to a bug in handling
// initialization ordering.
package main
import (
"strings"
"strconv"
)
var (
a = c + b
b = f()
c = f()
d = 3
)
func f() int {
d++
return d
}
func expect(name string, a int, b int) {
if a != b {
panic(strings.Join([]string{name, strconv.Itoa(a), strconv.Itoa(b)}, " "))
}
}
func main() {
expect("a", a, 9)
expect("b", b, 4)
expect("c", c, 5)
expect("d", d, 5)
}
π
$ go build ../src/cmd/compile/main.go && go run run.go -- fixedbugs/issue22326.go
# go run run.go -- fixedbugs/issue22326.go
exit status 1
panic: b 5 4
goroutine 1 [running]:
main.expect(0x10756cf, 0x1, 0x5, 0x4)
/Users/whit537/personal/golang/go/test/fixedbugs/issue22326.go:31 +0x14a
main.main()
/Users/whit537/personal/golang/go/test/fixedbugs/issue22326.go:37 +0x7d
exit status 2
FAIL fixedbugs/issue22326.go 2.499s
exit status 1
$
diff --git a/src/cmd/compile/internal/gc/init.go b/src/cmd/compile/internal/gc/init.go
index 36628d3..4783cc9 100644
--- a/src/cmd/compile/internal/gc/init.go
+++ b/src/cmd/compile/internal/gc/init.go
@@ -74,6 +74,7 @@ func anyinit(n []*Node) bool {
// return (10)
// }
func fninit(n []*Node) {
+ panic(`chaz`)
lineno = autogeneratedPos
nf := initfix(n)
if !anyinit(nf) {
diff --git a/src/cmd/compile/internal/gc/main.go b/src/cmd/compile/internal/gc/main.go
index ce91c6b..adaa0eb 100644
--- a/src/cmd/compile/internal/gc/main.go
+++ b/src/cmd/compile/internal/gc/main.go
@@ -134,6 +134,7 @@ var benchfile string
// arguments, type-checks the parsed Go package, compiles functions to machine
// code, and finally writes the compiled package definition to disk.
func Main(archInit func(*Arch)) {
+ panic(`cheese`)
timings.Start("fe", "init")
defer hidePanic()
diff --git a/src/cmd/compile/internal/gc/sinit.go b/src/cmd/compile/internal/gc/sinit.go
index 23cfdb8..dce2fee 100644
--- a/src/cmd/compile/internal/gc/sinit.go
+++ b/src/cmd/compile/internal/gc/sinit.go
@@ -247,6 +247,7 @@ func initfix(l []*Node) []*Node {
var lout []*Node
initplans = make(map[*Node]*InitPlan)
lno := lineno
+ panic(`choose`)
initreorder(l, &lout)
lineno = lno
initplans = nil
How do I hit those panics?
What happens when I go run run.go -- fixedbugs/issue22326.go
?
What is the --
about? A bash thing? Why is it mentioned in run.go
?
$ go run run.go fixedbugs/issue22326.go
named files must all be in one directory; have ./ and fixedbugs/
https://github.com/whit537/go/blob/eb7e84500ffef1f72958a94f135774f6c25b7aad/test/run.go#L92-L99
Interesting! It looks like that first // run
comment up at https://github.com/whit537/go/issues/1#issuecomment-339508116 is some sort of instruction to the test runner about what to do with the source code file.
https://github.com/whit537/go/blob/eb7e84500ffef1f72958a94f135774f6c25b7aad/test/run.go#L439
$ find . -name \*.go -exec head -n1 {} \; | sort | uniq
/*
// $G $D/$F.go $D/z*.go && $L $F.$A && ./$A.out
// $G $D/pkg.go && pack grc pkg.a pkg.$A 2> /dev/null && rm pkg.$A && errchk $G -I . -u $D/main.go
// $G $D/pkg.go && pack grcS pkg.a pkg.$A 2> /dev/null && rm pkg.$A && $G -I . -u $D/main.go
// +build !386
// +build !386,!arm,!mips,!mipsle,!amd64p32
// +build !amd64,!386
// +build !nacl
// +build !nacl,!android
// +build !nacl,!android,!darwin darwin,!arm
// +build !nacl,!plan9,!windows
// +build !nacl,!windows
// +build !nacl,disabled_see_issue_18589
// +build !plan9,!windows
// +build 386 amd64p32 arm
// +build amd64
// +build amd64 386
// +build amd64 s390x
// +build arm
// +build darwin linux
// +build ignore
// +build linux darwin
// +build linux,!ppc64
// Copyright 2009 The Go Authors. All rights reserved.
// Copyright 2010 The Go Authors. All rights reserved.
// Copyright 2011 The Go Authors. All rights reserved.
// Copyright 2012 The Go Authors. All rights reserved.
// Copyright 2013 The Go Authors. All rights reserved.
// Copyright 2014 The Go Authors. All rights reserved.
// Copyright 2015 The Go Authors. All rights reserved.
// Copyright 2016 The Go Authors. All rights reserved.
// Copyright 2017 The Go Authors. All rights reserved.
// build
// buildrun
// cmpout
// compile
// compile -c=2
// compile -c=4
// compiledir
// errorcheck
// errorcheck -+
// errorcheck -0 -N -d=nil
// errorcheck -0 -N -m -l
// errorcheck -0 -d=append,slice,ssa/prove/debug=1
// errorcheck -0 -d=nil
// errorcheck -0 -d=ssa/intrinsics/debug
// errorcheck -0 -d=ssa/opt/debug=3
// errorcheck -0 -d=typeassert
// errorcheck -0 -l -d=wb
// errorcheck -0 -live
// errorcheck -0 -live -d=compilelater
// errorcheck -0 -live -wb=0
// errorcheck -0 -m
// errorcheck -0 -m -l
// errorcheck -0 -m -l=3
// errorcheck -0 -m -live
// errorcheck -0 -m -m -l
// errorcheck -0 -race
// errorcheck -e=0
// errorcheck -std
// errorcheckandrundir -0 -d=ssa/intrinsics/debug
// errorcheckandrundir -0 -m -l=4
// errorcheckdir
// errorcheckdir -0 -m
// errorcheckdir -s
// errorcheckoutput
// errorcheckoutput ./index.go
// errorcheckwithauto -0 -l -live -wb=0 -d=ssa/insert_resched_checks/off
// run
// run -gcflags -l=4
// run arg1 arg2
// run cmplxdivide1.go
// rundir
// rundir -l=4
// runoutput
// runoutput ./index.go
// runoutput ./rotate.go
// skip
// true
//compile
package a
package b
package burnin
package main
package p
package p1
package p2
package q1
package rethinkgo
package surprise
package surprise2
package x
package x1
package y
package z
Ah! errorcheck
→ // Error
https://github.com/whit537/go/issues/1#issuecomment-339484049
Sooooooo ... we're going to launch a subprocess?
https://github.com/whit537/go/blob/eb7e84500ffef1f72958a94f135774f6c25b7aad/test/run.go#L541-L581
$ go run run.go -- fixedbugs/issue22326.go
panic: go build -o a.exe /Users/whit537/personal/golang/go/test/fixedbugs/issue22326.go
goroutine 6 [running]:
main.(*test).run.func2(0xc4200a2000, 0x5, 0x8, 0x4, 0x5, 0xc4200a2000, 0x4, 0x8)
/Users/whit537/personal/golang/go/test/run.go:542 +0x94
main.(*test).run(0xc4200582a0)
/Users/whit537/personal/golang/go/test/run.go:777 +0x1ed7
main.runTests.func1(0xc4200582a0)
/Users/whit537/personal/golang/go/test/run.go:278 +0x2b
created by main.runTests
/Users/whit537/personal/golang/go/test/run.go:277 +0x7e
exit status 2
$
diff --git a/test/run.go b/test/run.go
index 2fa2067..11c5573 100644
--- a/test/run.go
+++ b/test/run.go
@@ -539,6 +539,7 @@ func (t *test) run() {
useTmp := true
runcmd := func(args ...string) ([]byte, error) {
+ panic(strings.Join(args, " "))
cmd := exec.Command(args[0], args[1:]...)
var buf bytes.Buffer
cmd.Stdout = &buf
So are the cheese panics working but just getting swallowed?
No. They are not working. (-k
to output and keep the tempdir.)
/var/folders/77/7_nmc1957dv3rqv5l1wr5shh0000gn/T/698557725
total 1212
drwx------ 4 whit537 staff 136 Oct 26 07:44 ./
drwx------ 577 whit537 staff 19618 Oct 26 07:44 ../
-rwxr-xr-x 1 whit537 staff 1233520 Oct 26 07:44 a.exe*
-rw-r--r-- 1 whit537 staff 615 Oct 26 07:44 issue22326.go
$ ./a.exe
panic: b 5 4
goroutine 1 [running]:
main.expect(0x10756cf, 0x1, 0x5, 0x4)
/Users/whit537/personal/golang/go/test/fixedbugs/issue22326.go:31 +0x14a
main.main()
/Users/whit537/personal/golang/go/test/fixedbugs/issue22326.go:37 +0x7d
$
That was with buildrun
. With just run
:
$ go run run.go -k -- fixedbugs/issue22326.go
2017/10/26 08:05:02 Temporary directory is /var/folders/77/7_nmc1957dv3rqv5l1wr5shh0000gn/T/817271495
panic: go run fixedbugs/issue22326.go
goroutine 19 [running]:
main.(*test).run(0xc42009e240)
/Users/whit537/personal/golang/go/test/run.go:800 +0x17c1
main.runTests.func1(0xc42009e240)
/Users/whit537/personal/golang/go/test/run.go:278 +0x2b
created by main.runTests
/Users/whit537/personal/golang/go/test/run.go:277 +0x7e
exit status 2
$
I've created
fixedbugs/issue22326.go
with the spec example. It fails as expected withgo run run.go -- fixedbugs/issue22326.go
Now I'm trying to see a
panic
insrc/cmd/compile/internal/gc/main.go
show up under this test. I've discovered the comment-based execution recipes forrun.go
(e.g.,// run
), but haven't yet identified one of those (buildrun
?) to help here ...
https://github.com/golang/go/issues/22326#issuecomment-339647120
I took the panic out of the test in case it was masking the one in main but it's not.
Okay so we need to rebuild before testing or something like that?
If I cd src && ./make.bash
I hit the panic.
I need Go to build Go. But if I use the Go I'm testing to build itself, I am constrained in how I can debug.
I guess I need two Goes?
I rereviewed https://golang.org/doc/contribute.html#making_a_change and found no helpful info on how to do this.
I need Go to build Go.
... and I need to build Go in order to test it.
Sooooo I actually already have a second Go installed. How do I use that to build this?
$ /usr/local/bin/go version
go version go1.9.1 darwin/amd64
$ go version
go version devel +ffbcebb Wed Oct 25 12:10:38 2017 -0400 darwin/amd64
$ which -a go
/Users/whit537/personal/golang/go/bin/go
/usr/local/bin/go
$
π
$ GOROOT_BOOTSTRAP=/usr/local/Cellar/go/1.9.1/libexec ./make.bash
[...]
##### Building go_bootstrap for host, darwin/amd64.
runtime/internal/sys
panic: cheese
goroutine 1 [running]:
bootstrap/cmd/compile/internal/gc.Main(0x12fbf48)
/Users/whit537/personal/golang/go/src/cmd/compile/internal/gc/main.go:137 +0x39
main.main()
/Users/whit537/personal/golang/go/src/cmd/compile/main.go:49 +0x95
go tool dist: FAILED: /Users/whit537/personal/golang/go/pkg/tool/darwin_amd64/compile -pack -o /var/folders/77/7_nmc1957dv3rqv5l1wr5shh0000gn/T/go-tool-dist-314414208/runtime/internal/sys/_go_.a -p runtime/internal/sys /Users/whit537/personal/golang/go/src/runtime/internal/sys/arch.go /Users/whit537/personal/golang/go/src/runtime/internal/sys/arch_amd64.go /Users/whit537/personal/golang/go/src/runtime/internal/sys/intrinsics.go /Users/whit537/personal/golang/go/src/runtime/internal/sys/stubs.go /Users/whit537/personal/golang/go/src/runtime/internal/sys/sys.go /Users/whit537/personal/golang/go/src/runtime/internal/sys/zgoarch_amd64.go /Users/whit537/personal/golang/go/src/runtime/internal/sys/zgoos_darwin.go /Users/whit537/personal/golang/go/src/runtime/internal/sys/zversion.go: exit status 2
$
The dist
that make.bash
builds is built with the new Go.
$ gs
M cmd/compile/internal/gc/main.go
?? cmd/dist/dist
?? ../test/fixedbugs/issue22326.go
$ ./cmd/dist/dist
go tool dist: $GOROOT must be set
$
But then it rebuilds itself and proceeds to build the rest of Go with the new Go.
So how the heck is this supposed to work?
I need to build Go in order to run the test.
I guess I can't use panics, only fmt.Prints.
https://botbot.me/freenode/go-nuts/2017-10-26/?msg=92783097&page=14
Confirmed. π
Okay! Now what?
In other words: I can debug! What's my hypothesis?
$ go run run.go -- fixedbugs/issue22326.go
# command-line-arguments
66
# go run run.go -- fixedbugs/issue22326.go
exit status 1
# command-line-arguments
7
panic: b 5 4
goroutine 1 [running]:
main.expect(0x107574f, 0x1, 0x5, 0x4)
/Users/whit537/personal/golang/go/test/fixedbugs/issue22326.go:31 +0x14a
main.main()
/Users/whit537/personal/golang/go/test/fixedbugs/issue22326.go:37 +0x7d
exit status 2
FAIL fixedbugs/issue22326.go 0.419s
exit status 1
$
diff --git a/src/cmd/compile/internal/gc/sinit.go b/src/cmd/compile/internal/gc/sinit.go
index 23cfdb8..a28dd21 100644
--- a/src/cmd/compile/internal/gc/sinit.go
+++ b/src/cmd/compile/internal/gc/sinit.go
@@ -245,6 +245,7 @@ func initreorder(l []*Node, out *[]*Node) {
// to include in the init() function body.
func initfix(l []*Node) []*Node {
var lout []*Node
+ println(len(l))
initplans = make(map[*Node]*InitPlan)
lno := lineno
initreorder(l, &lout)
My hypothesis is that we don't reorder, we only order once.
How about a minimal test case?
Recall that the bug isn't present if b + c
instead of c + b
.
What's the call chain below initfix
?
initfix
initreorder
initreorder β©
init1
init1 β©
foundinitloop
init2list
init2
init1 β©
init2 β©
init2list β©
init2
π
Alright, gotta understand the Node. That's what this complex algorithm is operating on.
In terms of structure, it's got Left
, Right
, Ninit
, Nbody
, List
, and Rlist
:
initreorder
loops through all of the nodes n
in l
(what are those nodes?). It skips ODCLFUNC
, ODCLCONST
, and ODCLTYPE
(why not also ODCLFIELD
?):
For everything else it recurses on (and then clears) n.Ninit
, and then calls init1
on n
.
Ninit
is the initialization section of an if
, for
, or switch
statement:
Yeah and then Nbody is used in various statements:
Basically the basic building blocks (Left
, Right
, etc.) are used variously to encode various statements.
https://github.com/golang/go/issues/22326