runtime: exitsyscall: syscall frame is no longer valid (FreeBSD)

[sapiens-sapide]

Please answer these questions before submitting your issue. Thanks!

1. What version of Go are you using (go version)? go1.6.2 freebsd/amd64
2. What operating system and processor architecture are you using (go env)? freebsd 10.0-RELEASE-p9 GOARCH="amd64" GOBIN="" GOEXE="" GOHOSTARCH="amd64" GOHOSTOS="freebsd" GOOS="freebsd" GOPATH="/usr/local/goland/" GORACE="" GOROOT="/usr/local/go" GOTOOLDIR="/usr/local/go/pkg/tool/freebsd_amd64" GO15VENDOREXPERIMENT="1" CC="cc" GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0" CXX="clang++" CGO_ENABLED="1"
3. What did you do? when I run go install for local projects or cloned projects, the installation (randomly !) fails with the following output:

unexpected fault address 0xc82048d500
fatal error: fault
[signal 0xa code=0xc addr=0xc82048d500 pc=0xc82048d500]

goroutine 7 [running]:
runtime.throw(0x9e4730, 0x5)
    /usr/local/go/src/runtime/panic.go:547 +0x90 fp=0xc820022580 sp=0xc820022568
runtime.sigpanic()
    /usr/local/go/src/runtime/sigpanic_unix.go:21 +0x1e4 fp=0xc8200225d0 sp=0xc820022580
created by main.(*builder).do
    /usr/local/go/src/cmd/go/build.go:1320 +0x39e

goroutine 1 [semacquire]:
sync.runtime_Semacquire(0xc8201f3a7c)
    /usr/local/go/src/runtime/sema.go:47 +0x26
sync.(*WaitGroup).Wait(0xc8201f3a70)
    /usr/local/go/src/sync/waitgroup.go:127 +0xb4
main.(*builder).do(0xc82031d0a0, 0xc8203e3110)
    /usr/local/go/src/cmd/go/build.go:1323 +0x3c6
main.runInstall(0xc87c60, 0xc8200701d0, 0x1, 0x1)
    /usr/local/go/src/cmd/go/build.go:634 +0x4e5
main.main()
    /usr/local/go/src/cmd/go/main.go:181 +0x783

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:1998 +0x1

goroutine 20 [syscall]:
os/signal.signal_recv(0x0)
    /usr/local/go/src/runtime/sigqueue.go:116 +0x132
os/signal.loop()
    /usr/local/go/src/os/signal/signal_unix.go:22 +0x18
created by os/signal.init.1
    /usr/local/go/src/os/signal/signal_unix.go:28 +0x37

goroutine 3 [semacquire]:
sync.runtime_Semacquire(0xcac1ec)
    /usr/local/go/src/runtime/sema.go:47 +0x26
sync.(*RWMutex).RLock(0xcac1e0)
    /usr/local/go/src/sync/rwmutex.go:37 +0x58
os.Pipe(0x9, 0x0, 0x0, 0x0)
    /usr/local/go/src/os/pipe_bsd.go:17 +0x4f
os/exec.(*Cmd).writerDescriptor(0xc820163680, 0x800dbc800, 0xc820198fc0, 0x0, 0x0, 0x0)
    /usr/local/go/src/os/exec/exec.go:228 +0x27a
os/exec.(*Cmd).stdout(0xc820163680, 0x0, 0x0, 0x0)
    /usr/local/go/src/os/exec/exec.go:204 +0x4c
os/exec.(*Cmd).Start(0xc820163680, 0x0, 0x0)
    /usr/local/go/src/os/exec/exec.go:316 +0x2f4
os/exec.(*Cmd).Run(0xc820163680, 0x0, 0x0)
    /usr/local/go/src/os/exec/exec.go:259 +0x2d
main.(*builder).runOut(0xc82031d0a0, 0xc8201c94a0, 0x54, 0xc8202b44c0, 0x3e, 0x0, 0x0, 0x0, 0xc8204211e0, 0x14, ...)
    /usr/local/go/src/cmd/go/build.go:2049 +0x364
main.gcToolchain.gc(0xc82031d0a0, 0xc82039bc00, 0xc82006dc80, 0x57, 0xc82006dbc0, 0x5b, 0x0, 0xc82045a6e0, 0x2, 0x2, ...)
    /usr/local/go/src/cmd/go/build.go:2310 +0x1261
main.(*gcToolchain).gc(0xcac038, 0xc82031d0a0, 0xc82039bc00, 0xc82006dc80, 0x57, 0xc82006dbc0, 0x5b, 0x0, 0xc82045a6e0, 0x2, ...)
    <autogenerated>:3 +0x18e
main.(*builder).build(0xc82031d0a0, 0xc82015bee0, 0x0, 0x0)
    /usr/local/go/src/cmd/go/build.go:1484 +0x224c
main.(*builder).do.func1(0xc82015bee0)
    /usr/local/go/src/cmd/go/build.go:1257 +0x3b2
main.(*builder).do.func2(0xc8201f3a70, 0xc82031d0a0, 0xc82045a560)
    /usr/local/go/src/cmd/go/build.go:1314 +0x156
created by main.(*builder).do
    /usr/local/go/src/cmd/go/build.go:1320 +0x39e

goroutine 4 [semacquire]:
sync.runtime_Semacquire(0xc82031d0e4)
    /usr/local/go/src/runtime/sema.go:47 +0x26
sync.(*Mutex).Lock(0xc82031d0e0)
    /usr/local/go/src/sync/mutex.go:83 +0x1c4
main.(*builder).do.func2(0xc8201f3a70, 0xc82031d0a0, 0xc82045a560)
    /usr/local/go/src/cmd/go/build.go:1311 +0xfc
created by main.(*builder).do
    /usr/local/go/src/cmd/go/build.go:1320 +0x39e

goroutine 5 [semacquire]:
sync.runtime_Semacquire(0xc82031d0e4)
    /usr/local/go/src/runtime/sema.go:47 +0x26
sync.(*Mutex).Lock(0xc82031d0e0)
    /usr/local/go/src/sync/mutex.go:83 +0x1c4
main.(*builder).do.func2(0xc8201f3a70, 0xc82031d0a0, 0xc82045a560)
    /usr/local/go/src/cmd/go/build.go:1311 +0xfc
created by main.(*builder).do
    /usr/local/go/src/cmd/go/build.go:1320 +0x39e

goroutine 6 [select]:
main.(*builder).do.func2(0xc8201f3a70, 0xc82031d0a0, 0xc82045a560)
    /usr/local/go/src/cmd/go/build.go:1304 +0x1cf
created by main.(*builder).do
    /usr/local/go/src/cmd/go/build.go:1320 +0x39e

goroutine 8 [semacquire]:
runtime.gopark(0x0, 0xc820022f58, 0xc82006c660, 0xc82006c660, 0xc82006c660, 0x4dcfa0)
    /usr/local/go/src/runtime/proc.go:262 +0x163
created by main.(*builder).do
    /usr/local/go/src/cmd/go/build.go:1320 +0x39e

goroutine 9 [semacquire]:
sync.runtime_Semacquire(0xc82031d0e4)
    /usr/local/go/src/runtime/sema.go:47 +0x26
sync.(*Mutex).Lock(0xc82031d0e0)
    /usr/local/go/src/sync/mutex.go:83 +0x1c4
main.(*builder).do.func2(0xc8201f3a70, 0xc82031d0a0, 0xc82045a560)
    /usr/local/go/src/cmd/go/build.go:1311 +0xfc
created by main.(*builder).do
    /usr/local/go/src/cmd/go/build.go:1320 +0x39e

goroutine 10 [running]:
    goroutine running on other thread; stack unavailable
created by main.(*builder).do
    /usr/local/go/src/cmd/go/build.go:1320 +0x39e

If I run the go build command on the same package, build is OK.

[davecheney]

What is the complete failing command ?

On Tue, Jun 21, 2016 at 8:44 PM, Stanislas SABATIER < notifications@github.com> wrote:

Please answer these questions before submitting your issue. Thanks!

1.

What version of Go are you using (go version)? go1.6.2 freebsd/amd64 2.

What operating system and processor architecture are you using (go env )? freebsd 10.0-RELEASE-p9 GOARCH="amd64" GOBIN="" GOEXE="" GOHOSTARCH="amd64" GOHOSTOS="freebsd" GOOS="freebsd" GOPATH="/usr/local/goland/" GORACE="" GOROOT="/usr/local/go" GOTOOLDIR="/usr/local/go/pkg/tool/freebsd_amd64" GO15VENDOREXPERIMENT="1" CC="cc" GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0" CXX="clang++" CGO_ENABLED="1" 3.

What did you do? when I run go install for local projects or cloned projects, the installation (randomly !) fails with the following output:

unexpected fault address 0xc82048d500 fatal error: fault [signal 0xa code=0xc addr=0xc82048d500 pc=0xc82048d500]

goroutine 7 [running]: runtime.throw(0x9e4730, 0x5) /usr/local/go/src/runtime/panic.go:547 +0x90 fp=0xc820022580 sp=0xc820022568 runtime.sigpanic() /usr/local/go/src/runtime/sigpanic_unix.go:21 +0x1e4 fp=0xc8200225d0 sp=0xc820022580 created by main.(*builder).do /usr/local/go/src/cmd/go/build.go:1320 +0x39e

goroutine 1 [semacquire]: sync.runtime_Semacquire(0xc8201f3a7c) /usr/local/go/src/runtime/sema.go:47 +0x26 sync.(_WaitGroup).Wait(0xc8201f3a70) /usr/local/go/src/sync/waitgroup.go:127 +0xb4 main.(_builder).do(0xc82031d0a0, 0xc8203e3110) /usr/local/go/src/cmd/go/build.go:1323 +0x3c6 main.runInstall(0xc87c60, 0xc8200701d0, 0x1, 0x1) /usr/local/go/src/cmd/go/build.go:634 +0x4e5 main.main() /usr/local/go/src/cmd/go/main.go:181 +0x783

goroutine 17 [syscall, locked to thread]: runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1998 +0x1

goroutine 20 [syscall]: os/signal.signal_recv(0x0) /usr/local/go/src/runtime/sigqueue.go:116 +0x132 os/signal.loop() /usr/local/go/src/os/signal/signal_unix.go:22 +0x18 created by os/signal.init.1 /usr/local/go/src/os/signal/signal_unix.go:28 +0x37

goroutine 3 [semacquire]: sync.runtime_Semacquire(0xcac1ec) /usr/local/go/src/runtime/sema.go:47 +0x26 sync.(_RWMutex).RLock(0xcac1e0) /usr/local/go/src/sync/rwmutex.go:37 +0x58 os.Pipe(0x9, 0x0, 0x0, 0x0) /usr/local/go/src/os/pipe_bsd.go:17 +0x4f os/exec.(_Cmd).writerDescriptor(0xc820163680, 0x800dbc800, 0xc820198fc0, 0x0, 0x0, 0x0) /usr/local/go/src/os/exec/exec.go:228 +0x27a os/exec.(_Cmd).stdout(0xc820163680, 0x0, 0x0, 0x0) /usr/local/go/src/os/exec/exec.go:204 +0x4c os/exec.(_Cmd).Start(0xc820163680, 0x0, 0x0) /usr/local/go/src/os/exec/exec.go:316 +0x2f4 os/exec.(_Cmd).Run(0xc820163680, 0x0, 0x0) /usr/local/go/src/os/exec/exec.go:259 +0x2d main.(_builder).runOut(0xc82031d0a0, 0xc8201c94a0, 0x54, 0xc8202b44c0, 0x3e, 0x0, 0x0, 0x0, 0xc8204211e0, 0x14, ...) /usr/local/go/src/cmd/go/build.go:2049 +0x364 main.gcToolchain.gc(0xc82031d0a0, 0xc82039bc00, 0xc82006dc80, 0x57, 0xc82006dbc0, 0x5b, 0x0, 0xc82045a6e0, 0x2, 0x2, ...) /usr/local/go/src/cmd/go/build.go:2310 +0x1261 main.(_gcToolchain).gc(0xcac038, 0xc82031d0a0, 0xc82039bc00, 0xc82006dc80, 0x57, 0xc82006dbc0, 0x5b, 0x0, 0xc82045a6e0, 0x2, ...)

:3 +0x18e main.(_builder).build(0xc82031d0a0, 0xc82015bee0, 0x0, 0x0) /usr/local/go/src/cmd/go/build.go:1484 +0x224c main.(_builder).do.func1(0xc82015bee0) /usr/local/go/src/cmd/go/build.go:1257 +0x3b2 main.(_builder).do.func2(0xc8201f3a70, 0xc82031d0a0, 0xc82045a560) /usr/local/go/src/cmd/go/build.go:1314 +0x156 created by main.(*builder).do /usr/local/go/src/cmd/go/build.go:1320 +0x39e goroutine 4 [semacquire]: sync.runtime_Semacquire(0xc82031d0e4) /usr/local/go/src/runtime/sema.go:47 +0x26 sync.(_Mutex).Lock(0xc82031d0e0) /usr/local/go/src/sync/mutex.go:83 +0x1c4 main.(_builder).do.func2(0xc8201f3a70, 0xc82031d0a0, 0xc82045a560) /usr/local/go/src/cmd/go/build.go:1311 +0xfc created by main.(*builder).do /usr/local/go/src/cmd/go/build.go:1320 +0x39e goroutine 5 [semacquire]: sync.runtime_Semacquire(0xc82031d0e4) /usr/local/go/src/runtime/sema.go:47 +0x26 sync.(_Mutex).Lock(0xc82031d0e0) /usr/local/go/src/sync/mutex.go:83 +0x1c4 main.(_builder).do.func2(0xc8201f3a70, 0xc82031d0a0, 0xc82045a560) /usr/local/go/src/cmd/go/build.go:1311 +0xfc created by main.(*builder).do /usr/local/go/src/cmd/go/build.go:1320 +0x39e goroutine 6 [select]: main.(_builder).do.func2(0xc8201f3a70, 0xc82031d0a0, 0xc82045a560) /usr/local/go/src/cmd/go/build.go:1304 +0x1cf created by main.(_builder).do /usr/local/go/src/cmd/go/build.go:1320 +0x39e goroutine 8 [semacquire]: runtime.gopark(0x0, 0xc820022f58, 0xc82006c660, 0xc82006c660, 0xc82006c660, 0x4dcfa0) /usr/local/go/src/runtime/proc.go:262 +0x163 created by main.(*builder).do /usr/local/go/src/cmd/go/build.go:1320 +0x39e goroutine 9 [semacquire]: sync.runtime_Semacquire(0xc82031d0e4) /usr/local/go/src/runtime/sema.go:47 +0x26 sync.(_Mutex).Lock(0xc82031d0e0) /usr/local/go/src/sync/mutex.go:83 +0x1c4 main.(_builder).do.func2(0xc8201f3a70, 0xc82031d0a0, 0xc82045a560) /usr/local/go/src/cmd/go/build.go:1311 +0xfc created by main.(*builder).do /usr/local/go/src/cmd/go/build.go:1320 +0x39e goroutine 10 [running]: goroutine running on other thread; stack unavailable created by main.(*builder).do /usr/local/go/src/cmd/go/build.go:1320 +0x39e If I run the go build command on the same package, build is OK. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/golang/go/issues/16136, or mute the thread https://github.com/notifications/unsubscribe/AAAcA8STlVGcTiZG5Cq9vPCzPzbAVtZhks5qN8B7gaJpZM4I6mBf .

[sapiens-sapide]

For example : go install github.com/Masterminds/glide crashes but go build github.com/Masterminds/glide is OK.

[davecheney]

Do you have permission to write to $GOPATH ?

On Tue, Jun 21, 2016 at 8:59 PM, Stanislas SABATIER < notifications@github.com> wrote:

For example : go install github.com/Masterminds/glide crashes but go build github.com/Masterminds/glide is OK.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/golang/go/issues/16136#issuecomment-227407136, or mute the thread https://github.com/notifications/unsubscribe/AAAcAwoRdBnCGEuoo8Dt0lyRDoGhwy-7ks5qN8P8gaJpZM4I6mBf .

[sapiens-sapide]

yes, I'm root

[ianlancetaylor]

This is failing with signal 10, which is SIGBUS. The address and the PC value are the same, suggesting that the program has branched to non-executable code. The address is in the heap, so it is naturally non-executable. The question is why and how the program branched to that location. This can only be memory corruption of some sort.

The stack trace seems to show that syscall.ForkLock is held at the point of the crash, but I don't see any goroutine actually doing a fork/exec. I see a goroutine starting a fork/exec.

Holding the fork lock suggests this may be related to #15658, another FreeBSD issue that somehow involves the fork/exec code.

[sapiens-sapide]

FYI, this bug appeared after I upgraded go from 1.4.2 to 1.6.2.

[ianlancetaylor]

How reproducible is this problem?

[mikioh]

I guess that you are probably trying to make stuff by using both go1.4 and go1.6 packages. In general, it leads to messed up executable images. What happens when you run sudo pkg delete -y go && sudo pkg install -y go && rm -rf $GOPATH/pkg && go install -x -v github.com/Masterminds/glide?

[sapiens-sapide]

Hello, go install -x -v github.com/Masterminds/glide worked after removing and reinstalling go ! \o/ BUT, I tried to do the same with two other popular softwares github.com/coreos/etcd and github.com/gogits/gogs => they both failed to install with the same error output…

[sapiens-sapide]

@ianlancetaylor : to reproduce, just try to go get and go install software from popular repositories (gogs, etcd, beego…). With go 1.6.2 on a freeBSD 10.0-RELEASE-p9, many fail to do the job…

[mikioh]

@sapiens-sapide,

they both failed to install with the same error output…

Unfortunately it's pretty hard to read out useful information between above lines. If you believe that your issue is a bug on language spec. or standard library/commands, please describe the following:

1. What version of Go are you using (`go version`)?

(already answered, skip)

2. What operating system and processor architecture are you using (`go env`)?

(already answered, skip)

3. What did you do?
If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.

4. What did you expect to see?

5. What did you see instead?

In your case, the recipe must be a complete history of commands. Thanks.

[sapiens-sapide]

I put « randomly », because the bug is difficult to reproduce. It is related to the go get or go install command. For example :

• on a freeBSD machine (FreeBSD 10.2-RELEASE-p14 or 10.0-RELEASE-p9)
• install Go : # pkg install lang/go
• # mkdir -p /usr/local/goland
• # setenv GOPATH /usr/local/goland
• # go get github.com/Masterminds/glide => OK
• # go get github.com/gogits/gogs => crash

If I retry later, go get github.com/gogits/gogs may works… I see this random crash on different machines… In or out of a freeBSD jail.

[mikioh]

@sapiens-sapide,

I cannot reproduce your issue. Can you try the following procedure and report back the result?

1. Re-install FreeBSD's go 1.6 package
2. Run go get with logging until crash
3. Attach the log file to this issue (GitHub issue supports drag & drop op.)

For example,

sudo -E pkg delete lang/go
sudo -E pkg install lang/go

then

./g.sh 2>&1 | tee log

-- g.sh
#!/usr/bin/env bash

set -xe

goenv=`go env`
whichgo=`which go`
gover=`go version`

export GOPATH=/var/tmp/x
mkdir -p $GOPATH

while true; do
    rm -rf $GOPATH/pkg
    go get -x -v github.com/Masterminds/glide
    go get -x -v github.com/gogits/gogs
done

[sapiens-sapide]

Hi, I did your recipe. First time I ran the g.sh script, it freezed after this output:

# ./g.sh 2>&1 | tee log
++ go env
+ goenv='GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="freebsd"
GOOS="freebsd"
GOPATH="/usr/local/goland"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/freebsd_amd64"
GO15VENDOREXPERIMENT="1"
CC="cc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0"
CXX="clang++"
CGO_ENABLED="1"'
++ which go
+ whichgo=/usr/local/bin/go
++ go version
+ gover='go version go1.6.2 freebsd/amd64'
+ export GOPATH=/var/tmp/x
+ GOPATH=/var/tmp/x
+ mkdir -p /var/tmp/x
+ true
+ rm -rf /var/tmp/x/pkg
+ go get -x -v github.com/Masterminds/glide
github.com/Masterminds/glide (download)
cd .
git clone https://github.com/Masterminds/glide /var/tmp/x/src/github.com/Masterminds/glide
cd /var/tmp/x/src/github.com/Masterminds/glide
git submodule update --init --recursive
cd /var/tmp/x/src/github.com/Masterminds/glide
git show-ref
cd /var/tmp/x/src/github.com/Masterminds/glide
git submodule update --init --recursive
WORK=/tmp/go-build622440299
github.com/Masterminds/glide/msg
mkdir -p $WORK/github.com/Masterminds/glide/msg/_obj/
mkdir -p $WORK/github.com/Masterminds/glide/
cd /var/tmp/x/src/github.com/Masterminds/glide/msg
/usr/local/go/pkg/tool/freebsd_amd64/compile -o $WORK/github.com/Masterminds/glide/msg.a -trimpath $WORK -p github.com/Masterminds/glide/msg -complete -buildid 4a29600e46c57f13073e608df411ebe19aeef1d3 -D _/var/tmp/x/src/github.com/Masterminds/glide/msg -I $WORK -pack ./msg.go ./out.go
github.com/Masterminds/glide/vendor/github.com/Masterminds/semver
mkdir -p $WORK/github.com/Masterminds/glide/vendor/github.com/Masterminds/semver/_obj/
mkdir -p $WORK/github.com/Masterminds/glide/vendor/github.com/Masterminds/
cd /var/tmp/x/src/github.com/Masterminds/glide/vendor/github.com/Masterminds/semver
/usr/local/go/pkg/tool/freebsd_amd64/compile -o $WORK/github.com/Masterminds/glide/vendor/github.com/Masterminds/semver.a -trimpath $WORK -p github.com/Masterminds/glide/vendor/github.com/Masterminds/semver -complete -buildid b966cb8efd1f9cf68d1ca8acd76fb568daacb16d -D _/var/tmp/x/src/github.com/Masterminds/glide/vendor/github.com/Masterminds/semver -I $WORK -pack ./collection.go ./constraints.go ./doc.go ./version.go
mkdir -p /var/tmp/x/pkg/freebsd_amd64/github.com/Masterminds/glide/
mv $WORK/github.com/Masterminds/glide/msg.a /var/tmp/x/pkg/freebsd_amd64/github.com/Masterminds/glide/msg.a
github.com/Masterminds/glide/godep/strip
mkdir -p $WORK/github.com/Masterminds/glide/godep/strip/_obj/
mkdir -p $WORK/github.com/Masterminds/glide/godep/
cd /var/tmp/x/src/github.com/Masterminds/glide/godep/strip
/usr/local/go/pkg/tool/freebsd_amd64/compile -o $WORK/github.com/Masterminds/glide/godep/strip.a -trimpath $WORK -p github.com/Masterminds/glide/godep/strip -complete -buildid 55d03b864991015e413e0d6225355ff1f85233db -D _/var/tmp/x/src/github.com/Masterminds/glide/godep/strip -I $WORK -I /var/tmp/x/pkg/freebsd_amd64 -pack ./strip.go
mkdir -p /var/tmp/x/pkg/freebsd_amd64/github.com/Masterminds/glide/vendor/github.com/Masterminds/
mv $WORK/github.com/Masterminds/glide/vendor/github.com/Masterminds/semver.a /var/tmp/x/pkg/freebsd_amd64/github.com/Masterminds/glide/vendor/github.com/Masterminds/semver.a
mkdir -p /var/tmp/x/pkg/freebsd_amd64/github.com/Masterminds/glide/godep/
mv $WORK/github.com/Masterminds/glide/godep/strip.a /var/tmp/x/pkg/freebsd_amd64/github.com/Masterminds/glide/godep/strip.a
github.com/Masterminds/glide/path
mkdir -p $WORK/github.com/Masterminds/glide/path/_obj/
cd /var/tmp/x/src/github.com/Masterminds/glide/path
/usr/local/go/pkg/tool/freebsd_amd64/compile -o $WORK/github.com/Masterminds/glide/path.a -trimpath $WORK -p github.com/Masterminds/glide/path -complete -buildid ed397832c43fabfe410003f04300d0693bfa3237 -D _/var/tmp/x/src/github.com/Masterminds/glide/path -I $WORK -I /var/tmp/x/pkg/freebsd_amd64 -pack ./path.go ./strip.go
mv $WORK/github.com/Masterminds/glide/path.a /var/tmp/x/pkg/freebsd_amd64/github.com/Masterminds/glide/path.a
github.com/Masterminds/glide/cache
mkdir -p $WORK/github.com/Masterminds/glide/cache/_obj/
cd /var/tmp/x/src/github.com/Masterminds/glide/cache
/usr/local/go/pkg/tool/freebsd_amd64/compile -o $WORK/github.com/Masterminds/glide/cache.a -trimpath $WORK -p github.com/Masterminds/glide/cache -complete -buildid 875622c54eaf2446f0adfae4888fbb7807bc564f -importmap github.com/Masterminds/semver=github.com/Masterminds/glide/vendor/github.com/Masterminds/semver -D _/var/tmp/x/src/github.com/Masterminds/glide/cache -I $WORK -I /var/tmp/x/pkg/freebsd_amd64 -pack ./cache.go ./global_lock.go ./memory.go
mv $WORK/github.com/Masterminds/glide/cache.a /var/tmp/x/pkg/freebsd_amd64/github.com/Masterminds/glide/cache.a

After a while (five minutes ?) I hit ctrl+c to kill the script, then I run again the command ./g.sh 2>&1 | tee log and the script crashed with the following output :

# ./g.sh 2>&1 | tee log
++ go env
+ goenv='GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="freebsd"
GOOS="freebsd"
GOPATH="/usr/local/goland"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/freebsd_amd64"
GO15VENDOREXPERIMENT="1"
CC="cc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0"
CXX="clang++"
CGO_ENABLED="1"'
++ which go
+ whichgo=/usr/local/bin/go
++ go version
+ gover='go version go1.6.2 freebsd/amd64'
+ export GOPATH=/var/tmp/x
+ GOPATH=/var/tmp/x
+ mkdir -p /var/tmp/x
+ true
+ rm -rf /var/tmp/x/pkg
+ go get -x -v github.com/Masterminds/glide
WORK=/tmp/go-build829549633
github.com/Masterminds/glide/msg
mkdir -p $WORK/github.com/Masterminds/glide/msg/_obj/
mkdir -p $WORK/github.com/Masterminds/glide/
cd /var/tmp/x/src/github.com/Masterminds/glide/msg
/usr/local/go/pkg/tool/freebsd_amd64/compile -o $WORK/github.com/Masterminds/glide/msg.a -trimpath $WORK -p github.com/Masterminds/glide/msg -complete -buildid 4a29600e46c57f13073e608df411ebe19aeef1d3 -D _/var/tmp/x/src/github.com/Masterminds/glide/msg -I $WORK -pack ./msg.go ./out.go
github.com/Masterminds/glide/vendor/github.com/Masterminds/semver
mkdir -p $WORK/github.com/Masterminds/glide/vendor/github.com/Masterminds/semver/_obj/
mkdir -p $WORK/github.com/Masterminds/glide/vendor/github.com/Masterminds/
SIGTRAP: trace trap
PC=0x4e555d m=5

goroutine 0 [idle]:
runtime.morestack()
    /usr/local/go/src/runtime/asm_amd64.s:331 +0x1d

goroutine 8 [running]:
./g.sh: line 16: 74827 Segmentation fault      (core dumped) go get -x -v github.com/Masterminds/glide

I can provide you with the core dump file if needed (it's 36M)

[mikioh]

I did your recipe.

Looks like you didin't.

After a while (five minutes ?) I hit ctrl+c to kill the script, then I run again the command

Why? I believe that no one ensures that it accomplishes within 5 mins. Did you check that there's no remaining processes created by the script when you killed the first run? Did you check that the crashed go command is created by which parent process? Please do not interrupt the shell run unless you are sure what you are doing, and please run the script until crash. Thanks.

[sapiens-sapide]

Hi, I uninstalled and reinstalled go again (pkg delete lang/go then pkg install lang/go) and ran your script. Here is the ouptut (this time it crashed immediatly) :

# ./g.sh 2>&1 | tee log
++ go env
+ goenv='GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="freebsd"
GOOS="freebsd"
GOPATH="/usr/local/goland"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/freebsd_amd64"
GO15VENDOREXPERIMENT="1"
CC="cc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0"
CXX="clang++"
CGO_ENABLED="1"'
++ which go
+ whichgo=/usr/local/bin/go
++ go version
+ gover='go version go1.6.2 freebsd/amd64'
+ export GOPATH=/var/tmp/x
+ GOPATH=/var/tmp/x
+ mkdir -p /var/tmp/x
+ true
+ rm -rf /var/tmp/x/pkg
+ go get -x -v github.com/Masterminds/glide
github.com/Masterminds/glide (download)
cd .
git clone https://github.com/Masterminds/glide /var/tmp/x/src/github.com/Masterminds/glide
cd /var/tmp/x/src/github.com/Masterminds/glide
git submodule update --init --recursive
cd /var/tmp/x/src/github.com/Masterminds/glide
git show-ref
cd /var/tmp/x/src/github.com/Masterminds/glide
git submodule update --init --recursive
WORK=/tmp/go-build275461816
github.com/Masterminds/glide/msg
mkdir -p $WORK/github.com/Masterminds/glide/msg/_obj/
mkdir -p $WORK/github.com/Masterminds/glide/
cd /var/tmp/x/src/github.com/Masterminds/glide/msg
/usr/local/go/pkg/tool/freebsd_amd64/compile -o $WORK/github.com/Masterminds/glide/msg.a -trimpath $WORK -p github.com/Masterminds/glide/msg -complete -buildid 4a29600e46c57f13073e608df411ebe19aeef1d3 -D _/var/tmp/x/src/github.com/Masterminds/glide/msg -I $WORK -pack ./msg.go ./out.go
github.com/Masterminds/glide/vendor/github.com/Masterminds/semver
mkdir -p $WORK/github.com/Masterminds/glide/vendor/github.com/Masterminds/semver/_obj/
mkdir -p $WORK/github.com/Masterminds/glide/vendor/github.com/Masterminds/
cd /var/tmp/x/src/github.com/Masterminds/glide/vendor/github.com/Masterminds/semver
/usr/local/go/pkg/tool/freebsd_amd64/compile -o $WORK/github.com/Masterminds/glide/vendor/github.com/Masterminds/semver.a -trimpath $WORK -p github.com/Masterminds/glide/vendor/github.com/Masterminds/semver -complete -buildid b966cb8efd1f9cf68d1ca8acd76fb568daacb16d -D _/var/tmp/x/src/github.com/Masterminds/glide/vendor/github.com/Masterminds/semver -I $WORK -pack ./collection.go ./constraints.go ./doc.go ./version.go
github.com/Masterminds/glide/vendor/github.com/Masterminds/vcs
mkdir -p $WORK/github.com/Masterminds/glide/vendor/github.com/Masterminds/vcs/_obj/
github.com/Masterminds/glide/vendor/gopkg.in/yaml.v2
mkdir -p $WORK/github.com/Masterminds/glide/vendor/gopkg.in/yaml.v2/_obj/
mkdir -p $WORK/github.com/Masterminds/glide/vendor/gopkg.in/
github.com/Masterminds/glide/vendor/github.com/codegangsta/cli
mkdir -p $WORK/github.com/Masterminds/glide/vendor/github.com/codegangsta/cli/_obj/
mkdir -p $WORK/github.com/Masterminds/glide/vendor/github.com/codegangsta/
cd /var/tmp/x/src/github.com/Masterminds/glide/vendor/github.com/Masterminds/vcs
/usr/local/go/pkg/tool/freebsd_amd64/compile -o $WORK/github.com/Masterminds/glide/vendor/github.com/Masterminds/vcs.a -trimpath $WORK -p github.com/Masterminds/glide/vendor/github.com/Masterminds/vcs -complete -buildid 84112a2769048f3b05aa3ca23c2860ced78878c7 -D _/var/tmp/x/src/github.com/Masterminds/glide/vendor/github.com/Masterminds/vcs -I $WORK -pack ./bzr.go ./errors.go ./git.go ./hg.go ./repo.go ./svn.go ./vcs_local_lookup.go ./vcs_remote_lookup.go
cd /var/tmp/x/src/github.com/Masterminds/glide/vendor/github.com/codegangsta/cli
/usr/local/go/pkg/tool/freebsd_amd64/compile -o $WORK/github.com/Masterminds/glide/vendor/github.com/codegangsta/cli.a -trimpath $WORK -p github.com/Masterminds/glide/vendor/github.com/codegangsta/cli -complete -buildid 20b11526df83e8c0b1857d9ae31545caa3fdfbfc -D _/var/tmp/x/src/github.com/Masterminds/glide/vendor/github.com/codegangsta/cli -I $WORK -pack ./app.go ./category.go ./cli.go ./command.go ./context.go ./flag.go ./help.go
cd /var/tmp/x/src/github.com/Masterminds/glide/vendor/gopkg.in/yaml.v2
/usr/local/go/pkg/tool/freebsd_amd64/compile -o $WORK/github.com/Masterminds/glide/vendor/gopkg.in/yaml.v2.a -trimpath $WORK -p github.com/Masterminds/glide/vendor/gopkg.in/yaml.v2 -complete -buildid d027533d0c860643465c7b857fbaf4850a389e0c -D _/var/tmp/x/src/github.com/Masterminds/glide/vendor/gopkg.in/yaml.v2 -I $WORK -pack ./apic.go ./decode.go ./emitterc.go ./encode.go ./parserc.go ./readerc.go ./resolve.go ./scannerc.go ./sorter.go ./writerc.go ./yaml.go ./yamlh.go ./yamlprivateh.go
runtime: newstack sp=0xc8203cacd8 stack=[0xc8203ca000, 0xc8203cbfc0]
    morebuf={pc:0x4b80f9 sp:0xc8203cace0 lr:0x0}
    sched={pc:0x4b895d sp:0xc8203cacd8 lr:0x0 ctxt:0x0}
syscall.Syscall6(0x7, 0x15e69, 0xc8203cadcc, 0x0, 0xc8202d0510, 0x0, 0x0, 0x15e69, 0x0, 0x0)
    /usr/local/go/src/syscall/asm_freebsd_amd64.s:63 +0x61
syscall.wait4(0x15e69, 0xc8203cadcc, 0x0, 0xc8202d0510, 0x90, 0x0, 0x0)
    /usr/local/go/src/syscall/zsyscall_freebsd_amd64.go:34 +0x7f
syscall.Wait4(0x15e69, 0xc8203cae14, 0x0, 0xc8202d0510, 0xc82002c038, 0x0, 0x0)
    /usr/local/go/src/syscall/syscall_bsd.go:129 +0x55
os.(*Process).wait(0xc82030a580, 0x25, 0x0, 0x0)
    /usr/local/go/src/os/exec_unix.go:22 +0x105
os.(*Process).Wait(0xc82030a580, 0x0, 0x0, 0x0)
    /usr/local/go/src/os/doc.go:49 +0x2d
os/exec.(*Cmd).Wait(0xc8204c6500, 0x0, 0x0)
    /usr/local/go/src/os/exec/exec.go:396 +0x211
os/exec.(*Cmd).Run(0xc8204c6500, 0x0, 0x0)
    /usr/local/go/src/os/exec/exec.go:262 +0x64
main.(*builder).runOut(0xc8200a7570, 0xc820276b70, 0x2f, 0xc8202763c1, 0x20, 0x0, 0x0, 0x0, 0xc8203b46e0, 0xe, ...)
    /usr/local/go/src/cmd/go/build.go:2049 +0x364
main.gcToolchain.gc(0xc8200a7570, 0xc82025a400, 0xc8202335c0, 0x39, 0xc820233540, 0x3d, 0x0, 0xc82030a080, 0x2, 0x2, ...)
    /usr/local/go/src/cmd/go/build.go:2310 +0x1261
main.(*gcToolchain).gc(0xcac038, 0xc8200a7570, 0xc82025a400, 0xc8202335c0, 0x39, 0xc820233540, 0x3d, 0x0, 0xc82030a080, 0x2, ...)
    <autogenerated>:3 +0x18e
main.(*builder).build(0xc8200a7570, 0xc82032a8f0, 0x0, 0x0)
    /usr/local/go/src/cmd/go/build.go:1484 +0x224c
main.(*builder).do.func1(0xc82032a8f0)
    /usr/local/go/src/cmd/go/build.go:1257 +0x3b2
main.(*builder).do.func2(0xc82020c430, 0xc8200a7570, 0xc8201f84e0)
    /usr/local/go/src/cmd/go/build.go:1314 +0x156
created by main.(*builder).do
    /usr/local/go/src/cmd/go/build.go:1320 +0x39e
fatal error: runtime: stack split at bad time

runtime stack:
runtime.throw(0xa77e40, 0x20)
    /usr/local/go/src/runtime/panic.go:547 +0x90
runtime.newstack()
    /usr/local/go/src/runtime/stack.go:834 +0x582
runtime.morestack()
    /usr/local/go/src/runtime/asm_amd64.s:359 +0x7f

goroutine 13 [syscall]:
runtime.throw(0xa7cb40, 0x2d)
    /usr/local/go/src/runtime/panic.go:541 +0x9 fp=0xc8203cacf8 sp=0xc8203cace0
runtime.exitsyscall(0x6c18af)
    /usr/local/go/src/runtime/proc.go:2419 +0x62 fp=0xc8203cad20 sp=0xc8203cacf8
syscall.Syscall6(0x7, 0x15e69, 0xc8203cadcc, 0x0, 0xc8202d0510, 0x0, 0x0, 0x15e69, 0x0, 0x0)
    /usr/local/go/src/syscall/asm_freebsd_amd64.s:63 +0x61 fp=0xc8203cad28 sp=0xc8203cad20
syscall.wait4(0x15e69, 0xc8203cadcc, 0x0, 0xc8202d0510, 0x90, 0x0, 0x0)
    /usr/local/go/src/syscall/zsyscall_freebsd_amd64.go:34 +0x7f fp=0xc8203cad90 sp=0xc8203cad28
syscall.Wait4(0x15e69, 0xc8203cae14, 0x0, 0xc8202d0510, 0xc82002c038, 0x0, 0x0)
    /usr/local/go/src/syscall/syscall_bsd.go:129 +0x55 fp=0xc8203cadd8 sp=0xc8203cad90
os.(*Process).wait(0xc82030a580, 0x25, 0x0, 0x0)
    /usr/local/go/src/os/exec_unix.go:22 +0x105 fp=0xc8203cae88 sp=0xc8203cadd8
os.(*Process).Wait(0xc82030a580, 0x0, 0x0, 0x0)
    /usr/local/go/src/os/doc.go:49 +0x2d fp=0xc8203caeb0 sp=0xc8203cae88
os/exec.(*Cmd).Wait(0xc8204c6500, 0x0, 0x0)
    /usr/local/go/src/os/exec/exec.go:396 +0x211 fp=0xc8203caf80 sp=0xc8203caeb0
os/exec.(*Cmd).Run(0xc8204c6500, 0x0, 0x0)
    /usr/local/go/src/os/exec/exec.go:262 +0x64 fp=0xc8203cafb0 sp=0xc8203caf80
main.(*builder).runOut(0xc8200a7570, 0xc820276b70, 0x2f, 0xc8202763c1, 0x20, 0x0, 0x0, 0x0, 0xc8203b46e0, 0xe, ...)
    /usr/local/go/src/cmd/go/build.go:2049 +0x364 fp=0xc8203cb150 sp=0xc8203cafb0
main.gcToolchain.gc(0xc8200a7570, 0xc82025a400, 0xc8202335c0, 0x39, 0xc820233540, 0x3d, 0x0, 0xc82030a080, 0x2, 0x2, ...)
    /usr/local/go/src/cmd/go/build.go:2310 +0x1261 fp=0xc8203cb440 sp=0xc8203cb150
main.(*gcToolchain).gc(0xcac038, 0xc8200a7570, 0xc82025a400, 0xc8202335c0, 0x39, 0xc820233540, 0x3d, 0x0, 0xc82030a080, 0x2, ...)
    <autogenerated>:3 +0x18e fp=0xc8203cb4e8 sp=0xc8203cb440
main.(*builder).build(0xc8200a7570, 0xc82032a8f0, 0x0, 0x0)
    /usr/local/go/src/cmd/go/build.go:1484 +0x224c fp=0xc8203cbdf0 sp=0xc8203cb4e8
main.(*builder).do.func1(0xc82032a8f0)
    /usr/local/go/src/cmd/go/build.go:1257 +0x3b2 fp=0xc8203cbeb0 sp=0xc8203cbdf0
main.(*builder).do.func2(0xc82020c430, 0xc8200a7570, 0xc8201f84e0)
    /usr/local/go/src/cmd/go/build.go:1314 +0x156 fp=0xc8203cbf88 sp=0xc8203cbeb0
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:1998 +0x1 fp=0xc8203cbf90 sp=0xc8203cbf88
created by main.(*builder).do
    /usr/local/go/src/cmd/go/build.go:1320 +0x39e

goroutine 1 [semacquire]:
sync.runtime_Semacquire(0xc82020c43c)
    /usr/local/go/src/runtime/sema.go:47 +0x26
sync.(*WaitGroup).Wait(0xc82020c430)
    /usr/local/go/src/sync/waitgroup.go:127 +0xb4
main.(*builder).do(0xc8200a7570, 0xc820288ea0)
    /usr/local/go/src/cmd/go/build.go:1323 +0x3c6
main.runInstall(0xc880c0, 0xc8204a8c70, 0x1, 0x1)
    /usr/local/go/src/cmd/go/build.go:634 +0x4e5
main.runGet(0xc880c0, 0xc8204a8c70, 0x1, 0x1)
    /usr/local/go/src/cmd/go/get.go:141 +0x441
main.main()
    /usr/local/go/src/cmd/go/main.go:181 +0x783

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:1998 +0x1

goroutine 5 [syscall]:
os/signal.signal_recv(0x0)
    /usr/local/go/src/runtime/sigqueue.go:116 +0x132
os/signal.loop()
    /usr/local/go/src/os/signal/signal_unix.go:22 +0x18
created by os/signal.init.1
    /usr/local/go/src/os/signal/signal_unix.go:28 +0x37

goroutine 8 [syscall]:
syscall.Syscall6(0x7, 0x15e6b, 0xc8201e4dcc, 0x0, 0xc82022e120, 0x0, 0x0, 0xc8201e4da8, 0x4dd750, 0xc8201e4da0)
    /usr/local/go/src/syscall/asm_freebsd_amd64.s:44 +0x5
syscall.wait4(0x15e6b, 0xc8201e4dcc, 0x0, 0xc82022e120, 0x90, 0x0, 0x0)
    /usr/local/go/src/syscall/zsyscall_freebsd_amd64.go:34 +0x7f
syscall.Wait4(0x15e6b, 0xc8201e4e14, 0x0, 0xc82022e120, 0xc820200020, 0x0, 0x0)
    /usr/local/go/src/syscall/syscall_bsd.go:129 +0x55
os.(*Process).wait(0xc820418600, 0x25, 0x0, 0x0)
    /usr/local/go/src/os/exec_unix.go:22 +0x105
os.(*Process).Wait(0xc820418600, 0x0, 0x0, 0x0)
    /usr/local/go/src/os/doc.go:49 +0x2d
os/exec.(*Cmd).Wait(0xc82009e3c0, 0x0, 0x0)
    /usr/local/go/src/os/exec/exec.go:396 +0x211
os/exec.(*Cmd).Run(0xc82009e3c0, 0x0, 0x0)
    /usr/local/go/src/os/exec/exec.go:262 +0x64
main.(*builder).runOut(0xc8200a7570, 0xc82000b5e0, 0x4d, 0xc8202485c0, 0x3e, 0x0, 0x0, 0x0, 0xc8202eadc0, 0x14, ...)
    /usr/local/go/src/cmd/go/build.go:2049 +0x364
main.gcToolchain.gc(0xc8200a7570, 0xc820115800, 0xc8202a1800, 0x57, 0xc8202a16e0, 0x5b, 0x0, 0xc820418060, 0x2, 0x2, ...)
    /usr/local/go/src/cmd/go/build.go:2310 +0x1261
main.(*gcToolchain).gc(0xcac038, 0xc8200a7570, 0xc820115800, 0xc8202a1800, 0x57, 0xc8202a16e0, 0x5b, 0x0, 0xc820418060, 0x2, ...)
    <autogenerated>:3 +0x18e
main.(*builder).build(0xc8200a7570, 0xc820159ee0, 0x0, 0x0)
    /usr/local/go/src/cmd/go/build.go:1484 +0x224c
main.(*builder).do.func1(0xc820159ee0)
    /usr/local/go/src/cmd/go/build.go:1257 +0x3b2
main.(*builder).do.func2(0xc82020c430, 0xc8200a7570, 0xc8201f84e0)
    /usr/local/go/src/cmd/go/build.go:1314 +0x156
created by main.(*builder).do
    /usr/local/go/src/cmd/go/build.go:1320 +0x39e

goroutine 9 [syscall]:
syscall.Syscall6(0x7, 0x15e6d, 0xc8201e0dcc, 0x0, 0xc8203a8000, 0x0, 0x0, 0xc8201e0da8, 0x4dd750, 0x4db5d0)
    /usr/local/go/src/syscall/asm_freebsd_amd64.s:44 +0x5
syscall.wait4(0x15e6d, 0xc8201e0dcc, 0x0, 0xc8203a8000, 0x90, 0x0, 0x0)
    /usr/local/go/src/syscall/zsyscall_freebsd_amd64.go:34 +0x7f
syscall.Wait4(0x15e6d, 0xc8201e0e14, 0x0, 0xc8203a8000, 0xc82013e8a0, 0x0, 0x0)
    /usr/local/go/src/syscall/syscall_bsd.go:129 +0x55
os.(*Process).wait(0xc8201f8a60, 0x25, 0x0, 0x0)
    /usr/local/go/src/os/exec_unix.go:22 +0x105
os.(*Process).Wait(0xc8201f8a60, 0x0, 0x0, 0x0)
    /usr/local/go/src/os/doc.go:49 +0x2d
os/exec.(*Cmd).Wait(0xc82043a000, 0x0, 0x0)
    /usr/local/go/src/os/exec/exec.go:396 +0x211
os/exec.(*Cmd).Run(0xc82043a000, 0x0, 0x0)
    /usr/local/go/src/os/exec/exec.go:262 +0x64
main.(*builder).runOut(0xc8200a7570, 0xc8201477c0, 0x43, 0xc8202df100, 0x34, 0x0, 0x0, 0x0, 0xc8202a6b00, 0x19, ...)
    /usr/local/go/src/cmd/go/build.go:2049 +0x364
main.gcToolchain.gc(0xc8200a7570, 0xc820083800, 0xc8204caaa0, 0x4d, 0xc8202a1920, 0x51, 0x0, 0xc8201f8520, 0x2, 0x2, ...)
    /usr/local/go/src/cmd/go/build.go:2310 +0x1261
main.(*gcToolchain).gc(0xcac038, 0xc8200a7570, 0xc820083800, 0xc8204caaa0, 0x4d, 0xc8202a1920, 0x51, 0x0, 0xc8201f8520, 0x2, ...)
    <autogenerated>:3 +0x18e
main.(*builder).build(0xc8200a7570, 0xc82032bba0, 0x0, 0x0)
    /usr/local/go/src/cmd/go/build.go:1484 +0x224c
main.(*builder).do.func1(0xc82032bba0)
    /usr/local/go/src/cmd/go/build.go:1257 +0x3b2
main.(*builder).do.func2(0xc82020c430, 0xc8200a7570, 0xc8201f84e0)
    /usr/local/go/src/cmd/go/build.go:1314 +0x156
created by main.(*builder).do
    /usr/local/go/src/cmd/go/build.go:1320 +0x39e

goroutine 10 [syscall]:
syscall.Syscall6(0x7, 0x15e6c, 0xc8201e2dcc, 0x0, 0xc8201fa240, 0x0, 0x0, 0xc8201e2da8, 0x4dd750, 0xc8201e2da0)
    /usr/local/go/src/syscall/asm_freebsd_amd64.s:44 +0x5
syscall.wait4(0x15e6c, 0xc8201e2dcc, 0x0, 0xc8201fa240, 0x90, 0x0, 0x0)
    /usr/local/go/src/syscall/zsyscall_freebsd_amd64.go:34 +0x7f
syscall.Wait4(0x15e6c, 0xc8201e2e14, 0x0, 0xc8201fa240, 0xc820410020, 0x0, 0x0)
    /usr/local/go/src/syscall/syscall_bsd.go:129 +0x55
os.(*Process).wait(0xc8203f8560, 0x25, 0x0, 0x0)
    /usr/local/go/src/os/exec_unix.go:22 +0x105
os.(*Process).Wait(0xc8203f8560, 0x0, 0x0, 0x0)
    /usr/local/go/src/os/doc.go:49 +0x2d
os/exec.(*Cmd).Wait(0xc82045c8c0, 0x0, 0x0)
    /usr/local/go/src/os/exec/exec.go:396 +0x211
os/exec.(*Cmd).Run(0xc82045c8c0, 0x0, 0x0)
    /usr/local/go/src/os/exec/exec.go:262 +0x64
main.(*builder).runOut(0xc8200a7570, 0xc82022ccd0, 0x4d, 0xc8201d9980, 0x3e, 0x0, 0x0, 0x0, 0xc8202f69a0, 0x13, ...)
    /usr/local/go/src/cmd/go/build.go:2049 +0x364
main.gcToolchain.gc(0xc8200a7570, 0xc8202ed000, 0xc8202a1ec0, 0x57, 0xc8202a1da0, 0x5b, 0x0, 0xc8203f8020, 0x2, 0x2, ...)
    /usr/local/go/src/cmd/go/build.go:2310 +0x1261
main.(*gcToolchain).gc(0xcac038, 0xc8200a7570, 0xc8202ed000, 0xc8202a1ec0, 0x57, 0xc8202a1da0, 0x5b, 0x0, 0xc8203f8020, 0x2, ...)
    <autogenerated>:3 +0x18e
main.(*builder).build(0xc8200a7570, 0xc8202729c0, 0x0, 0x0)
    /usr/local/go/src/cmd/go/build.go:1484 +0x224c
main.(*builder).do.func1(0xc8202729c0)
    /usr/local/go/src/cmd/go/build.go:1257 +0x3b2
main.(*builder).do.func2(0xc82020c430, 0xc8200a7570, 0xc8201f84e0)
    /usr/local/go/src/cmd/go/build.go:1314 +0x156
created by main.(*builder).do
    /usr/local/go/src/cmd/go/build.go:1320 +0x39e

goroutine 11 [select]:
main.(*builder).do.func2(0xc82020c430, 0xc8200a7570, 0xc8201f84e0)
    /usr/local/go/src/cmd/go/build.go:1304 +0x1cf
created by main.(*builder).do
    /usr/local/go/src/cmd/go/build.go:1320 +0x39e

goroutine 12 [select]:
main.(*builder).do.func2(0xc82020c430, 0xc8200a7570, 0xc8201f84e0)
    /usr/local/go/src/cmd/go/build.go:1304 +0x1cf
created by main.(*builder).do
    /usr/local/go/src/cmd/go/build.go:1320 +0x39e

goroutine 14 [select]:
main.(*builder).do.func2(0xc82020c430, 0xc8200a7570, 0xc8201f84e0)
    /usr/local/go/src/cmd/go/build.go:1304 +0x1cf
created by main.(*builder).do
    /usr/local/go/src/cmd/go/build.go:1320 +0x39e

goroutine 15 [syscall]:
syscall.Syscall6(0x7, 0x15e6a, 0xc8203c6dcc, 0x0, 0xc8204c82d0, 0x0, 0x0, 0xc8203c6da8, 0x4dd750, 0xc8203c6da0)
    /usr/local/go/src/syscall/asm_freebsd_amd64.s:44 +0x5
syscall.wait4(0x15e6a, 0xc8203c6dcc, 0x0, 0xc8204c82d0, 0x90, 0x0, 0x0)
    /usr/local/go/src/syscall/zsyscall_freebsd_amd64.go:34 +0x7f
syscall.Wait4(0x15e6a, 0xc8203c6e14, 0x0, 0xc8204c82d0, 0xc820136050, 0x0, 0x0)
    /usr/local/go/src/syscall/syscall_bsd.go:129 +0x55
os.(*Process).wait(0xc82045e580, 0x25, 0x0, 0x0)
    /usr/local/go/src/os/exec_unix.go:22 +0x105
os.(*Process).Wait(0xc82045e580, 0x0, 0x0, 0x0)
    /usr/local/go/src/os/doc.go:49 +0x2d
os/exec.(*Cmd).Wait(0xc820329540, 0x0, 0x0)
    /usr/local/go/src/os/exec/exec.go:396 +0x211
os/exec.(*Cmd).Run(0xc820329540, 0x0, 0x0)
    /usr/local/go/src/os/exec/exec.go:262 +0x64
main.(*builder).runOut(0xc8200a7570, 0xc8204c2a00, 0x50, 0xc8204c2960, 0x41, 0x0, 0x0, 0x0, 0xc8203d02c0, 0x10, ...)
    /usr/local/go/src/cmd/go/build.go:2049 +0x364
main.gcToolchain.gc(0xc8200a7570, 0xc8203c4000, 0xc8202a1380, 0x5a, 0xc8202a1200, 0x5e, 0x0, 0xc82045e020, 0x2, 0x2, ...)
    /usr/local/go/src/cmd/go/build.go:2310 +0x1261
main.(*gcToolchain).gc(0xcac038, 0xc8200a7570, 0xc8203c4000, 0xc8202a1380, 0x5a, 0xc8202a1200, 0x5e, 0x0, 0xc82045e020, 0x2, ...)
    <autogenerated>:3 +0x18e
main.(*builder).build(0xc8200a7570, 0xc820063a00, 0x0, 0x0)
    /usr/local/go/src/cmd/go/build.go:1484 +0x224c
main.(*builder).do.func1(0xc820063a00)
    /usr/local/go/src/cmd/go/build.go:1257 +0x3b2
main.(*builder).do.func2(0xc82020c430, 0xc8200a7570, 0xc8201f84e0)
    /usr/local/go/src/cmd/go/build.go:1314 +0x156
created by main.(*builder).do
    /usr/local/go/src/cmd/go/build.go:1320 +0x39e

goroutine 51 [syscall]:
syscall.Syscall(0x3, 0x8, 0xc820386000, 0x200, 0x0, 0x200000000, 0x49d3e9)
    /usr/local/go/src/syscall/asm_freebsd_amd64.s:21 +0x5
syscall.read(0x8, 0xc820386000, 0x200, 0x200, 0x1, 0x0, 0x0)
    /usr/local/go/src/syscall/zsyscall_freebsd_amd64.go:892 +0x5f
syscall.Read(0x8, 0xc820386000, 0x200, 0x200, 0x0, 0x0, 0x0)
    /usr/local/go/src/syscall/syscall_unix.go:161 +0x4d
os.(*File).read(0xc820136048, 0xc820386000, 0x200, 0x200, 0xc820386000, 0x0, 0x0)
    /usr/local/go/src/os/file_unix.go:228 +0x75
os.(*File).Read(0xc820136048, 0xc820386000, 0x200, 0x200, 0x0, 0x0, 0x0)
    /usr/local/go/src/os/file.go:95 +0x8a
bytes.(*Buffer).ReadFrom(0xc8201601c0, 0x800d7c000, 0xc820136048, 0x0, 0x0, 0x0)
    /usr/local/go/src/bytes/buffer.go:176 +0x23f
io.copyBuffer(0x800d00d30, 0xc8201601c0, 0x800d7c000, 0xc820136048, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/io/io.go:374 +0x180
io.Copy(0x800d00d30, 0xc8201601c0, 0x800d7c000, 0xc820136048, 0xc820463f88, 0x0, 0x0)
    /usr/local/go/src/io/io.go:350 +0x64
os/exec.(*Cmd).writerDescriptor.func1(0x0, 0x0)
    /usr/local/go/src/os/exec/exec.go:236 +0x8b
os/exec.(*Cmd).Start.func1(0xc820329540, 0xc82045e2c0)
    /usr/local/go/src/os/exec/exec.go:344 +0x1d
created by os/exec.(*Cmd).Start
    /usr/local/go/src/os/exec/exec.go:345 +0x967

goroutine 82 [syscall]:
syscall.Syscall(0x3, 0x5, 0xc82043e000, 0x200, 0x100000000, 0x120456200, 0x49d3e9)
    /usr/local/go/src/syscall/asm_freebsd_amd64.s:21 +0x5
syscall.read(0x5, 0xc82043e000, 0x200, 0x200, 0x1, 0x0, 0x0)
    /usr/local/go/src/syscall/zsyscall_freebsd_amd64.go:892 +0x5f
syscall.Read(0x5, 0xc82043e000, 0x200, 0x200, 0x0, 0x0, 0x0)
    /usr/local/go/src/syscall/syscall_unix.go:161 +0x4d
os.(*File).read(0xc820200018, 0xc82043e000, 0x200, 0x200, 0xc82043e000, 0x0, 0x0)
    /usr/local/go/src/os/file_unix.go:228 +0x75
os.(*File).Read(0xc820200018, 0xc82043e000, 0x200, 0x200, 0x0, 0x0, 0x0)
    /usr/local/go/src/os/file.go:95 +0x8a
bytes.(*Buffer).ReadFrom(0xc8200fe460, 0x800d7c000, 0xc820200018, 0x0, 0x0, 0x0)
    /usr/local/go/src/bytes/buffer.go:176 +0x23f
io.copyBuffer(0x800d00d30, 0xc8200fe460, 0x800d7c000, 0xc820200018, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/io/io.go:374 +0x180
io.Copy(0x800d00d30, 0xc8200fe460, 0x800d7c000, 0xc820200018, 0xc820467f88, 0x0, 0x0)
    /usr/local/go/src/io/io.go:350 +0x64
os/exec.(*Cmd).writerDescriptor.func1(0x0, 0x0)
    /usr/local/go/src/os/exec/exec.go:236 +0x8b
os/exec.(*Cmd).Start.func1(0xc82009e3c0, 0xc820418300)
    /usr/local/go/src/os/exec/exec.go:344 +0x1d
created by os/exec.(*Cmd).Start
    /usr/local/go/src/os/exec/exec.go:345 +0x967

goroutine 27 [syscall]:
syscall.Syscall(0x3, 0xa, 0xc820326e00, 0x200, 0x100000000, 0xc820412040, 0x49d3e9)
    /usr/local/go/src/syscall/asm_freebsd_amd64.s:21 +0x5
syscall.read(0xa, 0xc820326e00, 0x200, 0x200, 0x1, 0x0, 0x0)
    /usr/local/go/src/syscall/zsyscall_freebsd_amd64.go:892 +0x5f
syscall.Read(0xa, 0xc820326e00, 0x200, 0x200, 0x0, 0x0, 0x0)
    /usr/local/go/src/syscall/syscall_unix.go:161 +0x4d
os.(*File).read(0xc820410018, 0xc820326e00, 0x200, 0x200, 0xc820326e00, 0x0, 0x0)
    /usr/local/go/src/os/file_unix.go:228 +0x75
os.(*File).Read(0xc820410018, 0xc820326e00, 0x200, 0x200, 0x0, 0x0, 0x0)
    /usr/local/go/src/os/file.go:95 +0x8a
bytes.(*Buffer).ReadFrom(0xc8200a6230, 0x800d7c000, 0xc820410018, 0x0, 0x0, 0x0)
    /usr/local/go/src/bytes/buffer.go:176 +0x23f
io.copyBuffer(0x800d00d30, 0xc8200a6230, 0x800d7c000, 0xc820410018, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/io/io.go:374 +0x180
io.Copy(0x800d00d30, 0xc8200a6230, 0x800d7c000, 0xc820410018, 0xc820461788, 0x0, 0x0)
    /usr/local/go/src/io/io.go:350 +0x64
os/exec.(*Cmd).writerDescriptor.func1(0x0, 0x0)
    /usr/local/go/src/os/exec/exec.go:236 +0x8b
os/exec.(*Cmd).Start.func1(0xc82045c8c0, 0xc8203f8200)
    /usr/local/go/src/os/exec/exec.go:344 +0x1d
created by os/exec.(*Cmd).Start
    /usr/local/go/src/os/exec/exec.go:345 +0x967

goroutine 16 [syscall]:
syscall.Syscall(0x3, 0xd, 0xc820093e00, 0x200, 0x0, 0x0, 0x49d3e9)
    /usr/local/go/src/syscall/asm_freebsd_amd64.s:21 +0x5
syscall.read(0xd, 0xc820093e00, 0x200, 0x200, 0x0, 0x0, 0x0)
    /usr/local/go/src/syscall/zsyscall_freebsd_amd64.go:892 +0x5f
syscall.Read(0xd, 0xc820093e00, 0x200, 0x200, 0x0, 0x0, 0x0)
    /usr/local/go/src/syscall/syscall_unix.go:161 +0x4d
os.(*File).read(0xc82013e898, 0xc820093e00, 0x200, 0x200, 0xc820093e00, 0x0, 0x0)
    /usr/local/go/src/os/file_unix.go:228 +0x75
os.(*File).Read(0xc82013e898, 0xc820093e00, 0x200, 0x200, 0x0, 0x0, 0x0)
    /usr/local/go/src/os/file.go:95 +0x8a
bytes.(*Buffer).ReadFrom(0xc820058620, 0x800d7c000, 0xc82013e898, 0x0, 0x0, 0x0)
    /usr/local/go/src/bytes/buffer.go:176 +0x23f
io.copyBuffer(0x800d00d30, 0xc820058620, 0x800d7c000, 0xc82013e898, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/io/io.go:374 +0x180
io.Copy(0x800d00d30, 0xc820058620, 0x800d7c000, 0xc82013e898, 0x0, 0x0, 0x0)
    /usr/local/go/src/io/io.go:350 +0x64
os/exec.(*Cmd).writerDescriptor.func1(0x0, 0x0)
    /usr/local/go/src/os/exec/exec.go:236 +0x8b
os/exec.(*Cmd).Start.func1(0xc82043a000, 0xc8201f8700)
    /usr/local/go/src/os/exec/exec.go:344 +0x1d
created by os/exec.(*Cmd).Start
    /usr/local/go/src/os/exec/exec.go:345 +0x967

[mikioh]

@ianlancetaylor,

Do you have any clue on this? It looks similar to #15639 but no cgo stuff in github.com/Masterminds/glide.

[sapiens-sapide]

FYI, I ran again the ./g.sh script after the previous crash (https://github.com/golang/go/issues/16136#issuecomment-228649302) : this time it "freezed" at the stage below (65 minutes without output. Script is still "running" but does nothing). Here are the processes involved :

• USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND
• root 89648 0.0 0.0 17768 3480 0 I 5:52AM 0:00.00 bash
• root 90218 0.0 0.0 17768 3276 0 I+ 6:01AM 0:00.00 bash ./g.sh
• root 90219 0.0 0.0 12276 1776 0 I+ 6:01AM 0:00.00 tee log
• root 90225 0.0 0.0 59608 17224 0 I+ 6:01AM 0:00.38 go get -x -v github.com/Masterminds/glide

and the output :

# ./g.sh 2>&1 | tee log
++ go env
+ goenv='GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="freebsd"
GOOS="freebsd"
GOPATH="/usr/local/goland"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/freebsd_amd64"
GO15VENDOREXPERIMENT="1"
CC="cc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0"
CXX="clang++"
CGO_ENABLED="1"'
++ which go
+ whichgo=/usr/local/bin/go
++ go version
+ gover='go version go1.6.2 freebsd/amd64'
+ export GOPATH=/var/tmp/x
+ GOPATH=/var/tmp/x
+ mkdir -p /var/tmp/x
+ true
+ rm -rf /var/tmp/x/pkg
+ go get -x -v github.com/Masterminds/glide
WORK=/tmp/go-build402853044
github.com/Masterminds/glide/msg
mkdir -p $WORK/github.com/Masterminds/glide/msg/_obj/
mkdir -p $WORK/github.com/Masterminds/glide/
cd /var/tmp/x/src/github.com/Masterminds/glide/msg
/usr/local/go/pkg/tool/freebsd_amd64/compile -o $WORK/github.com/Masterminds/glide/msg.a -trimpath $WORK -p github.com/Masterminds/glide/msg -complete -buildid 4a29600e46c57f13073e608df411ebe19aeef1d3 -D _/var/tmp/x/src/github.com/Masterminds/glide/msg -I $WORK -pack ./msg.go ./out.go
github.com/Masterminds/glide/vendor/github.com/Masterminds/semver
mkdir -p $WORK/github.com/Masterminds/glide/vendor/github.com/Masterminds/semver/_obj/
mkdir -p $WORK/github.com/Masterminds/glide/vendor/github.com/Masterminds/
cd /var/tmp/x/src/github.com/Masterminds/glide/vendor/github.com/Masterminds/semver
/usr/local/go/pkg/tool/freebsd_amd64/compile -o $WORK/github.com/Masterminds/glide/vendor/github.com/Masterminds/semver.a -trimpath $WORK -p github.com/Masterminds/glide/vendor/github.com/Masterminds/semver -complete -buildid b966cb8efd1f9cf68d1ca8acd76fb568daacb16d -D _/var/tmp/x/src/github.com/Masterminds/glide/vendor/github.com/Masterminds/semver -I $WORK -pack ./collection.go ./constraints.go ./doc.go ./version.go
github.com/Masterminds/glide/vendor/github.com/Masterminds/vcs
github.com/Masterminds/glide/vendor/github.com/codegangsta/cli
github.com/Masterminds/glide/vendor/gopkg.in/yaml.v2
mkdir -p $WORK/github.com/Masterminds/glide/vendor/github.com/Masterminds/vcs/_obj/
cd /var/tmp/x/src/github.com/Masterminds/glide/vendor/github.com/Masterminds/vcs
/usr/local/go/pkg/tool/freebsd_amd64/compile -o $WORK/github.com/Masterminds/glide/vendor/github.com/Masterminds/vcs.a -trimpath $WORK -p github.com/Masterminds/glide/vendor/github.com/Masterminds/vcs -complete -buildid 84112a2769048f3b05aa3ca23c2860ced78878c7 -D _/var/tmp/x/src/github.com/Masterminds/glide/vendor/github.com/Masterminds/vcs -I $WORK -pack ./bzr.go ./errors.go ./git.go ./hg.go ./repo.go ./svn.go ./vcs_local_lookup.go ./vcs_remote_lookup.go
mkdir -p $WORK/github.com/Masterminds/glide/vendor/github.com/codegangsta/cli/_obj/
mkdir -p $WORK/github.com/Masterminds/glide/vendor/github.com/codegangsta/
mkdir -p $WORK/github.com/Masterminds/glide/vendor/gopkg.in/yaml.v2/_obj/
cd /var/tmp/x/src/github.com/Masterminds/glide/vendor/github.com/codegangsta/cli
/usr/local/go/pkg/tool/freebsd_amd64/compile -o $WORK/github.com/Masterminds/glide/vendor/github.com/codegangsta/cli.a -trimpath $WORK -p github.com/Masterminds/glide/vendor/github.com/codegangsta/cli -complete -buildid 20b11526df83e8c0b1857d9ae31545caa3fdfbfc -D _/var/tmp/x/src/github.com/Masterminds/glide/vendor/github.com/codegangsta/cli -I $WORK -pack ./app.go ./category.go ./cli.go ./command.go ./context.go ./flag.go ./help.go
mkdir -p $WORK/github.com/Masterminds/glide/vendor/gopkg.in/
cd /var/tmp/x/src/github.com/Masterminds/glide/vendor/gopkg.in/yaml.v2
/usr/local/go/pkg/tool/freebsd_amd64/compile -o $WORK/github.com/Masterminds/glide/vendor/gopkg.in/yaml.v2.a -trimpath $WORK -p github.com/Masterminds/glide/vendor/gopkg.in/yaml.v2 -complete -buildid d027533d0c860643465c7b857fbaf4850a389e0c -D _/var/tmp/x/src/github.com/Masterminds/glide/vendor/gopkg.in/yaml.v2 -I $WORK -pack ./apic.go ./decode.go ./emitterc.go ./encode.go ./parserc.go ./readerc.go ./resolve.go ./scannerc.go ./sorter.go ./writerc.go ./yaml.go ./yamlh.go ./yamlprivateh.go
mkdir -p /var/tmp/x/pkg/freebsd_amd64/github.com/Masterminds/glide/
mv $WORK/github.com/Masterminds/glide/msg.a /var/tmp/x/pkg/freebsd_amd64/github.com/Masterminds/glide/msg.a
github.com/Masterminds/glide/godep/strip
mkdir -p $WORK/github.com/Masterminds/glide/godep/strip/_obj/
mkdir -p $WORK/github.com/Masterminds/glide/godep/
cd /var/tmp/x/src/github.com/Masterminds/glide/godep/strip
/usr/local/go/pkg/tool/freebsd_amd64/compile -o $WORK/github.com/Masterminds/glide/godep/strip.a -trimpath $WORK -p github.com/Masterminds/glide/godep/strip -complete -buildid 55d03b864991015e413e0d6225355ff1f85233db -D _/var/tmp/x/src/github.com/Masterminds/glide/godep/strip -I $WORK -I /var/tmp/x/pkg/freebsd_amd64 -pack ./strip.go
mkdir -p /var/tmp/x/pkg/freebsd_amd64/github.com/Masterminds/glide/godep/
mv $WORK/github.com/Masterminds/glide/godep/strip.a /var/tmp/x/pkg/freebsd_amd64/github.com/Masterminds/glide/godep/strip.a
github.com/Masterminds/glide/path
mkdir -p $WORK/github.com/Masterminds/glide/path/_obj/
cd /var/tmp/x/src/github.com/Masterminds/glide/path
/usr/local/go/pkg/tool/freebsd_amd64/compile -o $WORK/github.com/Masterminds/glide/path.a -trimpath $WORK -p github.com/Masterminds/glide/path -complete -buildid ed397832c43fabfe410003f04300d0693bfa3237 -D _/var/tmp/x/src/github.com/Masterminds/glide/path -I $WORK -I /var/tmp/x/pkg/freebsd_amd64 -pack ./path.go ./strip.go
mv $WORK/github.com/Masterminds/glide/path.a /var/tmp/x/pkg/freebsd_amd64/github.com/Masterminds/glide/path.a
mkdir -p /var/tmp/x/pkg/freebsd_amd64/github.com/Masterminds/glide/vendor/github.com/codegangsta/
mv $WORK/github.com/Masterminds/glide/vendor/github.com/codegangsta/cli.a /var/tmp/x/pkg/freebsd_amd64/github.com/Masterminds/glide/vendor/github.com/codegangsta/cli.a
mkdir -p /var/tmp/x/pkg/freebsd_amd64/github.com/Masterminds/glide/vendor/github.com/Masterminds/
mv $WORK/github.com/Masterminds/glide/vendor/github.com/Masterminds/vcs.a /var/tmp/x/pkg/freebsd_amd64/github.com/Masterminds/glide/vendor/github.com/Masterminds/vcs.a
github.com/Masterminds/glide/util
mkdir -p $WORK/github.com/Masterminds/glide/util/_obj/
cd /var/tmp/x/src/github.com/Masterminds/glide/util
/usr/local/go/pkg/tool/freebsd_amd64/compile -o $WORK/github.com/Masterminds/glide/util.a -trimpath $WORK -p github.com/Masterminds/glide/util -complete -buildid aac5b12bbd28fa973753a9a5b05686d8315ff73c -importmap github.com/Masterminds/vcs=github.com/Masterminds/glide/vendor/github.com/Masterminds/vcs -D _/var/tmp/x/src/github.com/Masterminds/glide/util -I $WORK -I /var/tmp/x/pkg/freebsd_amd64 -pack ./util.go
mv $WORK/github.com/Masterminds/glide/util.a /var/tmp/x/pkg/freebsd_amd64/github.com/Masterminds/glide/util.a
mkdir -p /var/tmp/x/pkg/freebsd_amd64/github.com/Masterminds/glide/vendor/gopkg.in/
mv $WORK/github.com/Masterminds/glide/vendor/gopkg.in/yaml.v2.a /var/tmp/x/pkg/freebsd_amd64/github.com/Masterminds/glide/vendor/gopkg.in/yaml.v2.a
github.com/Masterminds/glide/cfg
mkdir -p $WORK/github.com/Masterminds/glide/cfg/_obj/
cd /var/tmp/x/src/github.com/Masterminds/glide/cfg
/usr/local/go/pkg/tool/freebsd_amd64/compile -o $WORK/github.com/Masterminds/glide/cfg.a -trimpath $WORK -p github.com/Masterminds/glide/cfg -complete -buildid b46fb9abe2b127acfd0692861b918565a2ec2eb3 -importmap github.com/Masterminds/vcs=github.com/Masterminds/glide/vendor/github.com/Masterminds/vcs -importmap gopkg.in/yaml.v2=github.com/Masterminds/glide/vendor/gopkg.in/yaml.v2 -D _/var/tmp/x/src/github.com/Masterminds/glide/cfg -I $WORK -I /var/tmp/x/pkg/freebsd_amd64 -pack ./cfg.go ./config.go ./lock.go
mv $WORK/github.com/Masterminds/glide/cfg.a /var/tmp/x/pkg/freebsd_amd64/github.com/Masterminds/glide/cfg.a
github.com/Masterminds/glide/dependency
mkdir -p $WORK/github.com/Masterminds/glide/dependency/_obj/
github.com/Masterminds/glide/gb
mkdir -p $WORK/github.com/Masterminds/glide/gb/_obj/
cd /var/tmp/x/src/github.com/Masterminds/glide/dependency
/usr/local/go/pkg/tool/freebsd_amd64/compile -o $WORK/github.com/Masterminds/glide/dependency.a -trimpath $WORK -p github.com/Masterminds/glide/dependency -complete -buildid 1352fb80275a64d8ef661a71c97a787ae9a38b78 -D _/var/tmp/x/src/github.com/Masterminds/glide/dependency -I $WORK -I /var/tmp/x/pkg/freebsd_amd64 -pack ./delete.go ./resolver.go ./scan.go
github.com/Masterminds/glide/godep
mkdir -p $WORK/github.com/Masterminds/glide/godep/_obj/
cd /var/tmp/x/src/github.com/Masterminds/glide/gb
/usr/local/go/pkg/tool/freebsd_amd64/compile -o $WORK/github.com/Masterminds/glide/gb.a -trimpath $WORK -p github.com/Masterminds/glide/gb -complete -buildid e3d8a6a32a4afcb4a092655fa2cd9eaae6b2e75e -D _/var/tmp/x/src/github.com/Masterminds/glide/gb -I $WORK -I /var/tmp/x/pkg/freebsd_amd64 -pack ./gb.go ./manifest.go
github.com/Masterminds/glide/gom
mkdir -p $WORK/github.com/Masterminds/glide/gom/_obj/
github.com/Masterminds/glide/gpm
cd /var/tmp/x/src/github.com/Masterminds/glide/godep
/usr/local/go/pkg/tool/freebsd_amd64/compile -o $WORK/github.com/Masterminds/glide/godep.a -trimpath $WORK -p github.com/Masterminds/glide/godep -complete -buildid eb945ba470180bad2d3179ec720728a600056da9 -D _/var/tmp/x/src/github.com/Masterminds/glide/godep -I $WORK -I /var/tmp/x/pkg/freebsd_amd64 -pack ./godep.go
mkdir -p $WORK/github.com/Masterminds/glide/gpm/_obj/
cd /var/tmp/x/src/github.com/Masterminds/glide/gom
/usr/local/go/pkg/tool/freebsd_amd64/compile -o $WORK/github.com/Masterminds/glide/gom.a -trimpath $WORK -p github.com/Masterminds/glide/gom -complete -buildid 524a03abef5d77a52fac1cd6448dcab3fe747702 -D _/var/tmp/x/src/github.com/Masterminds/glide/gom -I $WORK -I /var/tmp/x/pkg/freebsd_amd64 -pack ./gom.go ./parser.go
cd /var/tmp/x/src/github.com/Masterminds/glide/gpm
/usr/local/go/pkg/tool/freebsd_amd64/compile -o $WORK/github.com/Masterminds/glide/gpm.a -trimpath $WORK -p github.com/Masterminds/glide/gpm -complete -buildid 9dd3c41500bad25b4c7654f520fbea4016a18d1f -D _/var/tmp/x/src/github.com/Masterminds/glide/gpm -I $WORK -I /var/tmp/x/pkg/freebsd_amd64 -pack ./gpm.go
mv $WORK/github.com/Masterminds/glide/gpm.a /var/tmp/x/pkg/freebsd_amd64/github.com/Masterminds/glide/gpm.a
mv $WORK/github.com/Masterminds/glide/gb.a /var/tmp/x/pkg/freebsd_amd64/github.com/Masterminds/glide/gb.a
mv $WORK/github.com/Masterminds/glide/godep.a /var/tmp/x/pkg/freebsd_amd64/github.com/Masterminds/glide/godep.a
mv $WORK/github.com/Masterminds/glide/gom.a /var/tmp/x/pkg/freebsd_amd64/github.com/Masterminds/glide/gom.a
github.com/Masterminds/glide/importer
mkdir -p $WORK/github.com/Masterminds/glide/importer/_obj/
cd /var/tmp/x/src/github.com/Masterminds/glide/importer
/usr/local/go/pkg/tool/freebsd_amd64/compile -o $WORK/github.com/Masterminds/glide/importer.a -trimpath $WORK -p github.com/Masterminds/glide/importer -complete -buildid 737efad6f25cac76a5dd48d15ca9f977f39e5b7a -D _/var/tmp/x/src/github.com/Masterminds/glide/importer -I $WORK -I /var/tmp/x/pkg/freebsd_amd64 -pack ./importer.go
mv $WORK/github.com/Masterminds/glide/importer.a /var/tmp/x/pkg/freebsd_amd64/github.com/Masterminds/glide/importer.a
mv $WORK/github.com/Masterminds/glide/dependency.a /var/tmp/x/pkg/freebsd_amd64/github.com/Masterminds/glide/dependency.a
github.com/Masterminds/glide/tree
mkdir -p $WORK/github.com/Masterminds/glide/tree/_obj/
cd /var/tmp/x/src/github.com/Masterminds/glide/tree
/usr/local/go/pkg/tool/freebsd_amd64/compile -o $WORK/github.com/Masterminds/glide/tree.a -trimpath $WORK -p github.com/Masterminds/glide/tree -complete -buildid 2a4e681e39c17b1838d87dad57af383154a498d1 -D _/var/tmp/x/src/github.com/Masterminds/glide/tree -I $WORK -I /var/tmp/x/pkg/freebsd_amd64 -pack ./tree.go
mv $WORK/github.com/Masterminds/glide/tree.a /var/tmp/x/pkg/freebsd_amd64/github.com/Masterminds/glide/tree.a

@mikioh : may I kill it ? ;-)

[mikioh]

@sapiens-sapide,

Sure. ;) The script is dumb, you need to kill all spawned processes by hand before the next run. I'm not sure the reason why I cannot reproduce your issue on my FreeBSD VM. Perhaps, it might depend on your circumstances. Can you show us your FreeBSD profile? I think the output of dmesg | grep CPU and dmesg | grep memory on your node is enough.

[sapiens-sapide]

cat /var/run/dmesg.boot | grep CPU :

CPU: Intel(R) Xeon(R) CPU E5-1620 v2 @ 3.70GHz (3700.08-MHz K8-class CPU)
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
cpu4: <ACPI CPU> on acpi0
cpu5: <ACPI CPU> on acpi0
cpu6: <ACPI CPU> on acpi0
cpu7: <ACPI CPU> on acpi0
p4tcc0: <CPU Frequency Thermal Control> on cpu0
p4tcc1: <CPU Frequency Thermal Control> on cpu1
p4tcc2: <CPU Frequency Thermal Control> on cpu2
p4tcc3: <CPU Frequency Thermal Control> on cpu3
p4tcc4: <CPU Frequency Thermal Control> on cpu4
p4tcc5: <CPU Frequency Thermal Control> on cpu5
p4tcc6: <CPU Frequency Thermal Control> on cpu6
p4tcc7: <CPU Frequency Thermal Control> on cpu7
SMP: AP CPU #1 Launched!
SMP: AP CPU #3 Launched!
SMP: AP CPU #6 Launched!
SMP: AP CPU #5 Launched!
SMP: AP CPU #4 Launched!
SMP: AP CPU #7 Launched!
SMP: AP CPU #2 Launched!

cat /var/run/dmesg.boot | grep memory :

real memory  = 68727865344 (65544 MB)
avail memory = 66651250688 (63563 MB)

[ianlancetaylor]

I tried to recreate the problem using gomote, but failed.

@sapiens-sapide Can you try to recreate this using Go 1.7beta2, or tip? That should at least give a better error message. The real error message from 1.6 is being hidden by a bug that has been fixed.

My best guess at the moment is that something has gone wrong with our support for fork on FreeBSD, but I have no idea what that could be.

[mikioh]

@sapiens-sapide,

In addition,

CPU: Intel(R) Xeon(R) CPU E5-1620

This is a sort of random firing, though. Can you try with disabling Intel VT-x support? If your kernel has the kernel state entry hw.vmm.vmx.use_apic_vid, sysctl hw.vmm.vmx.use_apic_vid=0 disables the functionality on the running kernel.

[sapiens-sapide]

Hi @mikioh, I have not hw.vmm.vmx.use_apic_vid entry in sysctl command.

[sapiens-sapide]

@ianlancetaylor, Here is the output error message with GO 1.7beta2 after I ran the g.sh script :

# ./g.sh 2>&1 | tee log
++ go env
+ goenv='GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="freebsd"
GOOS="freebsd"
GOPATH="/usr/local/goland"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/freebsd_amd64"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build016225789=/tmp/go-build -gno-record-gcc-switches"
CXX="clang++"
CGO_ENABLED="1"'
++ which go
+ whichgo=/usr/local/bin/go
++ go version
+ gover='go version go1.7beta2 freebsd/amd64'
+ export GOPATH=/var/tmp/x
+ GOPATH=/var/tmp/x
+ mkdir -p /var/tmp/x
+ true
+ rm -rf /var/tmp/x/pkg
+ go get -x -v github.com/Masterminds/glide
WORK=/tmp/go-build580530906
github.com/Masterminds/glide/msg
mkdir -p $WORK/github.com/Masterminds/glide/msg/_obj/
mkdir -p $WORK/github.com/Masterminds/glide/
github.com/Masterminds/glide/vendor/github.com/Masterminds/semver
mkdir -p $WORK/github.com/Masterminds/glide/vendor/github.com/Masterminds/semver/_obj/
cd /var/tmp/x/src/github.com/Masterminds/glide/msg
/usr/local/go/pkg/tool/freebsd_amd64/compile -o $WORK/github.com/Masterminds/glide/msg.a -trimpath $WORK -p github.com/Masterminds/glide/msg -complete -buildid f79dd4827acbdd9310095a44ca3aed6110c2c998 -D _/var/tmp/x/src/github.com/Masterminds/glide/msg -I $WORK -pack ./msg.go ./out.go
mkdir -p $WORK/github.com/Masterminds/glide/vendor/github.com/Masterminds/
cd /var/tmp/x/src/github.com/Masterminds/glide/vendor/github.com/Masterminds/semver
/usr/local/go/pkg/tool/freebsd_amd64/compile -o $WORK/github.com/Masterminds/glide/vendor/github.com/Masterminds/semver.a -trimpath $WORK -p github.com/Masterminds/glide/vendor/github.com/Masterminds/semver -complete -buildid 1fc301c412747212aa8c21b36940a605f9f2d348 -D _/var/tmp/x/src/github.com/Masterminds/glide/vendor/github.com/Masterminds/semver -I $WORK -pack ./collection.go ./constraints.go ./doc.go ./version.go
fatal error: malloc deadlock

goroutine 14 [running]:
runtime.throw(0x84b669, 0xf)
    /usr/local/go/src/runtime/panic.go:566 +0x95 fp=0xc420452f68 sp=0xc420452f48
runtime.mallocgc(0x30, 0x812d60, 0xc4202be501, 0x2c)
    /usr/local/go/src/runtime/malloc.go:570 +0x98d fp=0xc420453008 sp=0xc420452f68
runtime.newobject(0x812d60, 0xc420453100)
    /usr/local/go/src/runtime/malloc.go:785 +0x38 fp=0xc420453038 sp=0xc420453008
os.newProcess(0x121f2, 0x0, 0xc4204de280)
    /usr/local/go/src/os/exec.go:23 +0x31 fp=0xc420453070 sp=0xc420453038
os.startProcess(0xc4202be570, 0x2c, 0xc4204de280, 0x13, 0x14, 0xc4204532a8, 0xc420142070, 0x836160, 0xc4200ba310)
    /usr/local/go/src/os/exec_posix.go:49 +0x2c0 fp=0xc420453158 sp=0xc420453070
os.StartProcess(0xc4202be570, 0x2c, 0xc4204de280, 0x13, 0x14, 0xc4204532a8, 0x0, 0x0, 0xc420453280)
    /usr/local/go/src/os/doc.go:28 +0x64 fp=0xc4204531b0 sp=0xc420453158
os/exec.(*Cmd).Start(0xc4204de500, 0x0, 0xc4202ee6c0)
    /usr/local/go/src/os/exec/exec.go:348 +0x3b2 fp=0xc420453300 sp=0xc4204531b0
os/exec.(*Cmd).Run(0xc4204de500, 0x0, 0x0)
    /usr/local/go/src/os/exec/exec.go:275 +0x2b fp=0xc420453328 sp=0xc420453300
main.(*builder).runOut(0xc4200bbe30, 0xc420410d20, 0x50, 0xc420410c80, 0x41, 0x0, 0x0, 0x0, 0xc420224000, 0x10, ...)
    /usr/local/go/src/cmd/go/build.go:2062 +0x45d fp=0xc420453410 sp=0xc420453328
main.gcToolchain.gc(0xc4200bbe30, 0xc4203ded80, 0xc420419020, 0x5a, 0xc420418f60, 0x5e, 0xc420158400, 0xc4203d0040, 0x2, 0x2, ...)
    /usr/local/go/src/cmd/go/build.go:2322 +0xfd0 fp=0xc4204536c0 sp=0xc420453410
main.(*gcToolchain).gc(0xa46338, 0xc4200bbe30, 0xc4203ded80, 0xc420419020, 0x5a, 0xc420418f60, 0x5e, 0x0, 0xc4203d0040, 0x2, ...)
    <autogenerated>:5 +0x10f fp=0xc420453770 sp=0xc4204536c0
main.(*builder).build(0xc4200bbe30, 0xc4201668f0, 0x0, 0x0)
    /usr/local/go/src/cmd/go/build.go:1497 +0x11f4 fp=0xc420453e20 sp=0xc420453770
main.(*builder).do.func1(0xc4201668f0)
    /usr/local/go/src/cmd/go/build.go:1264 +0x85 fp=0xc420453eb0 sp=0xc420453e20
main.(*builder).do.func2(0xc4204d7f50, 0xc4200bbe30, 0xc4203e8d60)
    /usr/local/go/src/cmd/go/build.go:1321 +0x147 fp=0xc420453f88 sp=0xc420453eb0
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:2086 +0x1 fp=0xc420453f90 sp=0xc420453f88
created by main.(*builder).do
    /usr/local/go/src/cmd/go/build.go:1327 +0x3f2

goroutine 1 [semacquire]:
sync.runtime_Semacquire(0xc4204d7f5c)
    /usr/local/go/src/runtime/sema.go:47 +0x30
sync.(*WaitGroup).Wait(0xc4204d7f50)
    /usr/local/go/src/sync/waitgroup.go:131 +0x97
main.(*builder).do(0xc4200bbe30, 0xc4203552b0)
    /usr/local/go/src/cmd/go/build.go:1330 +0x41b
main.runInstall(0xa22f60, 0xc42044ae10, 0x1, 0x1)
    /usr/local/go/src/cmd/go/build.go:633 +0x7ce
main.runGet(0xa22f60, 0xc42044ae10, 0x1, 0x1)
    /usr/local/go/src/cmd/go/get.go:141 +0x35e
main.main()
    /usr/local/go/src/cmd/go/main.go:181 +0x624

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:2086 +0x1

goroutine 5 [syscall]:
os/signal.signal_recv(0x0)
    /usr/local/go/src/runtime/sigqueue.go:116 +0x157
os/signal.loop()
    /usr/local/go/src/os/signal/signal_unix.go:22 +0x22
created by os/signal.init.1
    /usr/local/go/src/os/signal/signal_unix.go:28 +0x41

goroutine 15 [semacquire]:
sync.runtime_Semacquire(0xc4200bbe7c)
    /usr/local/go/src/runtime/sema.go:47 +0x30
sync.(*Mutex).Lock(0xc4200bbe78)
    /usr/local/go/src/sync/mutex.go:85 +0xd0
main.(*builder).do.func2(0xc4204d7f50, 0xc4200bbe30, 0xc4203e8d60)
    /usr/local/go/src/cmd/go/build.go:1318 +0x104
created by main.(*builder).do
    /usr/local/go/src/cmd/go/build.go:1327 +0x3f2

goroutine 16 [syscall]:
syscall.Syscall6(0x214, 0x0, 0x121f1, 0x0, 0x18, 0x0, 0x0, 0x4a8fae, 0xc420457178, 0x51e924)
    /usr/local/go/src/syscall/asm_freebsd_amd64.s:44 +0x5
os.(*Process).blockUntilWaitable(0xc4202c7500, 0x10, 0x88f808, 0xc42042a140)
    /usr/local/go/src/os/wait_wait6.go:29 +0x6b
os.(*Process).wait(0xc4202c7500, 0x0, 0x0, 0xc42009c900)
    /usr/local/go/src/os/exec_unix.go:22 +0xab
os.(*Process).Wait(0xc4202c7500, 0xc42042a218, 0xc4204572a8, 0x0)
    /usr/local/go/src/os/doc.go:49 +0x2b
os/exec.(*Cmd).Wait(0xc42042a140, 0x0, 0x0)
    /usr/local/go/src/os/exec/exec.go:424 +0x84
os/exec.(*Cmd).Run(0xc42042a140, 0x0, 0x0)
    /usr/local/go/src/os/exec/exec.go:278 +0x48
main.(*builder).runOut(0xc4200bbe30, 0xc42027fb60, 0x2f, 0xc42027f3b1, 0x20, 0x0, 0x0, 0x0, 0xc4202b0000, 0xe, ...)
    /usr/local/go/src/cmd/go/build.go:2062 +0x45d
main.gcToolchain.gc(0xc4200bbe30, 0xc42037a480, 0xc4201f5240, 0x39, 0xc4201f51c0, 0x3d, 0xc420158400, 0xc42039c060, 0x2, 0x2, ...)
    /usr/local/go/src/cmd/go/build.go:2322 +0xfd0
main.(*gcToolchain).gc(0xa46338, 0xc4200bbe30, 0xc42037a480, 0xc4201f5240, 0x39, 0xc4201f51c0, 0x3d, 0x0, 0xc42039c060, 0x2, ...)
    <autogenerated>:5 +0x10f
main.(*builder).build(0xc4200bbe30, 0xc42032cd00, 0x0, 0x0)
    /usr/local/go/src/cmd/go/build.go:1497 +0x11f4
main.(*builder).do.func1(0xc42032cd00)
    /usr/local/go/src/cmd/go/build.go:1264 +0x85
main.(*builder).do.func2(0xc4204d7f50, 0xc4200bbe30, 0xc4203e8d60)
    /usr/local/go/src/cmd/go/build.go:1321 +0x147
created by main.(*builder).do
    /usr/local/go/src/cmd/go/build.go:1327 +0x3f2

goroutine 50 [semacquire]:
sync.runtime_Semacquire(0xc4200bbe7c)
    /usr/local/go/src/runtime/sema.go:47 +0x30
sync.(*Mutex).Lock(0xc4200bbe78)
    /usr/local/go/src/sync/mutex.go:85 +0xd0
main.(*builder).do.func1(0xc42032d110)
    /usr/local/go/src/cmd/go/build.go:1269 +0xb2
main.(*builder).do.func2(0xc4204d7f50, 0xc4200bbe30, 0xc4203e8d60)
    /usr/local/go/src/cmd/go/build.go:1321 +0x147
created by main.(*builder).do
    /usr/local/go/src/cmd/go/build.go:1327 +0x3f2

goroutine 51 [semacquire]:
sync.runtime_Semacquire(0xc4200bbe7c)
    /usr/local/go/src/runtime/sema.go:47 +0x30
sync.(*Mutex).Lock(0xc4200bbe78)
    /usr/local/go/src/sync/mutex.go:85 +0xd0
main.(*builder).do.func1(0xc420355040)
    /usr/local/go/src/cmd/go/build.go:1269 +0xb2
main.(*builder).do.func2(0xc4204d7f50, 0xc4200bbe30, 0xc4203e8d60)
    /usr/local/go/src/cmd/go/build.go:1321 +0x147
created by main.(*builder).do
    /usr/local/go/src/cmd/go/build.go:1327 +0x3f2

goroutine 52 [semacquire]:
sync.runtime_Semacquire(0xc4200bbe7c)
    /usr/local/go/src/runtime/sema.go:47 +0x30
sync.(*Mutex).Lock(0xc4200bbe78)
    /usr/local/go/src/sync/mutex.go:85 +0xd0
main.(*builder).do.func1(0xc420070820)
    /usr/local/go/src/cmd/go/build.go:1269 +0xb2
main.(*builder).do.func2(0xc4204d7f50, 0xc4200bbe30, 0xc4203e8d60)
    /usr/local/go/src/cmd/go/build.go:1321 +0x147
created by main.(*builder).do
    /usr/local/go/src/cmd/go/build.go:1327 +0x3f2

goroutine 53 [semacquire]:
sync.runtime_Semacquire(0xc4200bbe7c)
    /usr/local/go/src/runtime/sema.go:47 +0x30
sync.(*Mutex).Lock(0xc4200bbe78)
    /usr/local/go/src/sync/mutex.go:85 +0xd0
main.(*builder).do.func2(0xc4204d7f50, 0xc4200bbe30, 0xc4203e8d60)
    /usr/local/go/src/cmd/go/build.go:1318 +0x104
created by main.(*builder).do
    /usr/local/go/src/cmd/go/build.go:1327 +0x3f2

goroutine 54 [semacquire]:
sync.runtime_Semacquire(0xc4200bbe7c)
    /usr/local/go/src/runtime/sema.go:47 +0x30
sync.(*Mutex).Lock(0xc4200bbe78)
    /usr/local/go/src/sync/mutex.go:85 +0xd0
main.(*builder).do.func2(0xc4204d7f50, 0xc4200bbe30, 0xc4203e8d60)
    /usr/local/go/src/cmd/go/build.go:1318 +0x104
created by main.(*builder).do
    /usr/local/go/src/cmd/go/build.go:1327 +0x3f2

goroutine 66 [syscall]:
syscall.Syscall(0x3, 0x4, 0xc42036c200, 0x200, 0x200, 0xc42036c200, 0x800aa7328)
    /usr/local/go/src/syscall/asm_freebsd_amd64.s:21 +0x5
syscall.read(0x4, 0xc42036c200, 0x200, 0x200, 0x800a40960, 0x0, 0x30)
    /usr/local/go/src/syscall/zsyscall_freebsd_amd64.go:893 +0x55
syscall.Read(0x4, 0xc42036c200, 0x200, 0x200, 0x0, 0xc420482400, 0xc420484340)
    /usr/local/go/src/syscall/syscall_unix.go:161 +0x49
os.(*File).read(0xc42002c030, 0xc42036c200, 0x200, 0x200, 0x4d4abb, 0x7beb40, 0x200)
    /usr/local/go/src/os/file_unix.go:228 +0x6a
os.(*File).Read(0xc42002c030, 0xc42036c200, 0x200, 0x200, 0xc420469f30, 0x2, 0xc420469e20)
    /usr/local/go/src/os/file.go:101 +0x59
bytes.(*Buffer).ReadFrom(0xc420110070, 0xa02be0, 0xc42002c030, 0xc420469e88, 0x1, 0x0)
    /usr/local/go/src/bytes/buffer.go:176 +0x155
io.copyBuffer(0xa01b60, 0xc420110070, 0xa02be0, 0xc42002c030, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/io/io.go:384 +0x324
io.Copy(0xa01b60, 0xc420110070, 0xa02be0, 0xc42002c030, 0xc42010ab40, 0x46b4c4, 0x200001)
    /usr/local/go/src/io/io.go:360 +0x68
os/exec.(*Cmd).writerDescriptor.func1(0x180001, 0x0)
    /usr/local/go/src/os/exec/exec.go:252 +0x4d
os/exec.(*Cmd).Start.func1(0xc42042a140, 0xc42039c220)
    /usr/local/go/src/os/exec/exec.go:360 +0x27
created by os/exec.(*Cmd).Start
    /usr/local/go/src/os/exec/exec.go:361 +0x4c4

[mikioh]

@sapiens-sapide,

not hw.vmm.vmx.use_apic_vid

Thanks, then, can you try with disabling Intel Hyperthreading support? We see 8 CPUs in dmesg, but E5-1620v2 implements only 4 cores. I guess that you probably need to tweak UEFI or BIOS to turn off Intel HT.

[ianlancetaylor]

Thanks for running it on tip. This version of the error just looks like memory corruption.

If there is a problem in the Go distribution, I think it must be in the fork support on FreeBSD. @sapiens-sapide The original report says you are running FreeBSD 10.0-RELEASE-p9. @mikioh Which exact FreeBSD version are you trying?

Has anybody been able to reproduce this on a second machine? Could there simply be something wrong with the single machine showing the problem?

[mikioh]

Which exact FreeBSD version are you trying?

From 10.0 through 10.3 VM on several x64/amd64 processors (up to 2 cores) w/ 1GiB memory. I always disable any processor's virtualization technology such as VT-x/hyperthreading to avoid facing a few critical kernel bugs fixed in the latest 10.3.

[ianlancetaylor]

Still don't know what is happening here. Postponing to 1.8.

[sapiens-sapide]

for information — if it may help to find out what's happening — here is another crash log from caddy on the same machine. Caddy server works fine for 24 to 48 hours, until it crashes with the following output :

runtime: nonempty check fails b.log[0]= 0 b.log[1]= 0 b.log[2]= 0 b.log[3]= 0
fatal error: workbuf is empty

goroutine 20 [running]:
runtime.throw(0xc8f510, 0x10)
    /usr/local/go/src/runtime/panic.go:530 +0x90 fp=0xc820266e80 sp=0xc820266e68
runtime.(*workbuf).checknonempty(0x80100d940)
    /usr/local/go/src/runtime/mgcwork.go:321 +0x153 fp=0xc820266eb8 sp=0xc820266e80
runtime.trygetfull(0xa7, 0xc81fffadd8)
    /usr/local/go/src/runtime/mgcwork.go:385 +0x55 fp=0xc820266ed8 sp=0xc820266eb8
runtime.(*gcWork).tryGet(0xc820017220, 0xc820017220)
    /usr/local/go/src/runtime/mgcwork.go:150 +0x7c fp=0xc820266f08 sp=0xc820266ed8
runtime.gcDrain(0xc820017220, 0x6)
    /usr/local/go/src/runtime/mgcmark.go:833 +0x22a fp=0xc820266f40 sp=0xc820266f08
runtime.gcBgMarkWorker(0xc820016000)
    /usr/local/go/src/runtime/mgc.go:1432 +0x25f fp=0xc820266fb8 sp=0xc820266f40
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:1998 +0x1 fp=0xc820266fc0 sp=0xc820266fb8
created by runtime.gcBgMarkStartWorkers
    /usr/local/go/src/runtime/mgc.go:1329 +0x92

goroutine 1 [semacquire, 960 minutes]:
sync.runtime_Semacquire(0xfc174c)
    /usr/local/go/src/runtime/sema.go:47 +0x26
sync.(*WaitGroup).Wait(0xfc1740)
    /usr/local/go/src/sync/waitgroup.go:127 +0xb4
github.com/mholt/caddy/caddy.Wait()
    /tmp/custombuild_3675_930774383/src/github.com/mholt/caddy/caddy/caddy.go:302 +0x23
main.main()
    /tmp/custombuild_3675_930774383/src/github.com/mholt/caddy/main.go:109 +0x6f1

goroutine 5 [syscall, 960 minutes]:
os/signal.signal_recv(0x0)
    /usr/local/go/src/runtime/sigqueue.go:116 +0x132
os/signal.loop()
    /usr/local/go/src/os/signal/signal_unix.go:22 +0x18
created by os/signal.init.1
    /usr/local/go/src/os/signal/signal_unix.go:28 +0x37

goroutine 6 [chan receive, 960 minutes]:
github.com/mholt/caddy/caddy.trapSignalsCrossPlatform.func1()
    /tmp/custombuild_3675_930774383/src/github.com/mholt/caddy/caddy/sigtrap.go:31 +0x10d
created by github.com/mholt/caddy/caddy.trapSignalsCrossPlatform
    /tmp/custombuild_3675_930774383/src/github.com/mholt/caddy/caddy/sigtrap.go:49 +0x2b

goroutine 7 [chan receive, 960 minutes]:
github.com/mholt/caddy/caddy.trapSignalsPosix.func1()
    /tmp/custombuild_3675_930774383/src/github.com/mholt/caddy/caddy/sigtrap_posix.go:19 +0x36c
created by github.com/mholt/caddy/caddy.trapSignalsPosix
    /tmp/custombuild_3675_930774383/src/github.com/mholt/caddy/caddy/sigtrap_posix.go:78 +0x2b

goroutine 17 [select, 960 minutes, locked to thread]:
runtime.gopark(0xd47fa8, 0xc82001e728, 0xc06cb8, 0x6, 0x18, 0x2)
    /usr/local/go/src/runtime/proc.go:262 +0x163
runtime.selectgoImpl(0xc82001e728, 0x0, 0x18)
    /usr/local/go/src/runtime/select.go:392 +0xa67
runtime.selectgo(0xc82001e728)
    /usr/local/go/src/runtime/select.go:215 +0x12
runtime.ensureSigM.func1()
    /usr/local/go/src/runtime/signal1_unix.go:279 +0x340
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:1998 +0x1

goroutine 54 [select]:
github.com/mholt/caddy/caddy/https.maintainAssets(0x0)
    /tmp/custombuild_3675_930774383/src/github.com/mholt/caddy/caddy/https/maintain.go:34 +0x67d
created by github.com/mholt/caddy/caddy/https.Activate
    /tmp/custombuild_3675_930774383/src/github.com/mholt/caddy/caddy/https/https.go:76 +0x1de

goroutine 97 [chan receive, 960 minutes]:
github.com/mholt/caddy/server.newGracefulListener.func1(0xc820056660)
    /tmp/custombuild_3675_930774383/src/github.com/mholt/caddy/server/graceful.go:14 +0x3e
created by github.com/mholt/caddy/server.newGracefulListener
    /tmp/custombuild_3675_930774383/src/github.com/mholt/caddy/server/graceful.go:19 +0xe3

goroutine 63 [IO wait, 11 minutes]:
net.runtime_pollWait(0x800fd6d58, 0x72, 0x0)
    /usr/local/go/src/runtime/netpoll.go:160 +0x60
net.(*pollDesc).Wait(0xc820412060, 0x72, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc820412060, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).accept(0xc820412000, 0x0, 0x800fd6f68, 0xc8204f6720)
    /usr/local/go/src/net/fd_unix.go:426 +0x27c
net.(*TCPListener).AcceptTCP(0xc820026000, 0xbbca09, 0x0, 0x0)
    /usr/local/go/src/net/tcpsock_posix.go:254 +0x4d
github.com/mholt/caddy/server.tcpKeepAliveListener.Accept(0xc820026000, 0x0, 0x0, 0x0, 0x0)
    /tmp/custombuild_3675_930774383/src/github.com/mholt/caddy/server/server.go:406 +0x41
github.com/mholt/caddy/server.(*gracefulListener).Accept(0xc820056660, 0x0, 0x0, 0x0, 0x0)
    /tmp/custombuild_3675_930774383/src/github.com/mholt/caddy/server/graceful.go:36 +0x63
net/http.(*Server).Serve(0xc820487000, 0x8012ed148, 0xc820056660, 0x0, 0x0)
    /usr/local/go/src/net/http/server.go:2117 +0x129
github.com/mholt/caddy/server.(*Server).serve(0xc820460480, 0x8012ed0c8, 0xc820026000, 0x0, 0x0)
    /tmp/custombuild_3675_930774383/src/github.com/mholt/caddy/server/server.go:178 +0x422
github.com/mholt/caddy/server.(*Server).ListenAndServe(0xc820460480, 0x0, 0x0)
    /tmp/custombuild_3675_930774383/src/github.com/mholt/caddy/server/server.go:152 +0x1e2
github.com/mholt/caddy/caddy.startServers.func1(0xc8204603c0, 0xc820460480, 0x0, 0x0)
    /tmp/custombuild_3675_930774383/src/github.com/mholt/caddy/caddy/caddy.go:243 +0x13d
created by github.com/mholt/caddy/caddy.startServers
    /tmp/custombuild_3675_930774383/src/github.com/mholt/caddy/caddy/caddy.go:245 +0x567

goroutine 81 [IO wait, 1 minutes]:
net.runtime_pollWait(0x800fd6e18, 0x72, 0x0)
    /usr/local/go/src/runtime/netpoll.go:160 +0x60
net.(*pollDesc).Wait(0xc820378a70, 0x72, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc820378a70, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).accept(0xc820378a10, 0x0, 0x800fd6f68, 0xc820349880)
    /usr/local/go/src/net/fd_unix.go:426 +0x27c
net.(*TCPListener).AcceptTCP(0xc82037a328, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/tcpsock_posix.go:254 +0x4d
github.com/mholt/caddy/server.tcpKeepAliveListener.Accept(0xc82037a328, 0x0, 0x0, 0x0, 0x0)
    /tmp/custombuild_3675_930774383/src/github.com/mholt/caddy/server/server.go:406 +0x41
github.com/mholt/caddy/server.(*gracefulListener).Accept(0xc8204ef920, 0x0, 0x0, 0x0, 0x0)
    /tmp/custombuild_3675_930774383/src/github.com/mholt/caddy/server/graceful.go:36 +0x63
crypto/tls.(*listener).Accept(0xc82023f160, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/crypto/tls/tls.go:52 +0x60
net/http.(*Server).Serve(0xc820487080, 0x8011efae0, 0xc82023f160, 0x0, 0x0)
    /usr/local/go/src/net/http/server.go:2117 +0x129
github.com/mholt/caddy/server.serveTLS(0xc820460540, 0x8011efae0, 0xc82023f160, 0xc8202406e0, 0x4, 0x4, 0x0, 0x0)
    /tmp/custombuild_3675_930774383/src/github.com/mholt/caddy/server/server.go:225 +0x2ca
github.com/mholt/caddy/server.(*Server).serve(0xc820460540, 0x8012ed0c8, 0xc82037a328, 0x0, 0x0)
    /tmp/custombuild_3675_930774383/src/github.com/mholt/caddy/server/server.go:174 +0x330
github.com/mholt/caddy/server.(*Server).ListenAndServe(0xc820460540, 0x0, 0x0)
    /tmp/custombuild_3675_930774383/src/github.com/mholt/caddy/server/server.go:152 +0x1e2
github.com/mholt/caddy/caddy.startServers.func1(0xc8204603c0, 0xc820460540, 0x0, 0x0)
    /tmp/custombuild_3675_930774383/src/github.com/mholt/caddy/caddy/caddy.go:243 +0x13d
created by github.com/mholt/caddy/caddy.startServers
    /tmp/custombuild_3675_930774383/src/github.com/mholt/caddy/caddy/caddy.go:245 +0x567

goroutine 83 [syscall]:
syscall.Syscall6(0x7, 0x1347a, 0xc82003ba4c, 0x0, 0xc82050a000, 0x0, 0x0, 0x411e62, 0xc82027f668, 0x44f780)
    /usr/local/go/src/syscall/asm_freebsd_amd64.s:44 +0x5
syscall.wait4(0x1347a, 0xc82003ba4c, 0x0, 0xc82050a000, 0x90, 0x0, 0x0)
    /usr/local/go/src/syscall/zsyscall_freebsd_amd64.go:34 +0x7f
syscall.Wait4(0x1347a, 0xc82003ba94, 0x0, 0xc82050a000, 0xc820026018, 0x0, 0x0)
    /usr/local/go/src/syscall/syscall_bsd.go:129 +0x55
os.(*Process).wait(0xc82033a160, 0x15, 0x0, 0x0)
    /usr/local/go/src/os/exec_unix.go:22 +0x105
os.(*Process).Wait(0xc82033a160, 0x0, 0x0, 0x0)
    /usr/local/go/src/os/doc.go:49 +0x2d
os/exec.(*Cmd).Wait(0xc82027f540, 0x0, 0x0)
    /usr/local/go/src/os/exec/exec.go:396 +0x211
github.com/abiosoft/caddy-git.runCmd(0xc820490660, 0x12, 0xc82047e210, 0x3, 0x3, 0xc820141170, 0x20, 0x0, 0x0)
    /home/caddy/go/src/github.com/abiosoft/caddy-git/commands.go:186 +0x19a
github.com/abiosoft/caddy-git.(*Repo).gitCmd(0xc8204840f0, 0xc82047e210, 0x3, 0x3, 0xc820141170, 0x20, 0x0, 0x0)
    /home/caddy/go/src/github.com/abiosoft/caddy-git/git.go:178 +0xc7
github.com/abiosoft/caddy-git.(*Repo).pull(0xc8204840f0, 0x0, 0x0)
    /home/caddy/go/src/github.com/abiosoft/caddy-git/git.go:104 +0x1fa
github.com/abiosoft/caddy-git.(*Repo).Pull(0xc8204840f0, 0x0, 0x0)
    /home/caddy/go/src/github.com/abiosoft/caddy-git/git.go:70 +0x174
github.com/abiosoft/caddy-git.Start.func1(0xc8204840f0, 0xc820491c00)
    /home/caddy/go/src/github.com/abiosoft/caddy-git/service.go:34 +0xec
created by github.com/abiosoft/caddy-git.Start
    /home/caddy/go/src/github.com/abiosoft/caddy-git/service.go:43 +0x110

goroutine 86 [select, 60 minutes]:
github.com/abiosoft/caddy-git.Start.func1(0xc8204842d0, 0xc82023e640)
    /home/caddy/go/src/github.com/abiosoft/caddy-git/service.go:32 +0x26c
created by github.com/abiosoft/caddy-git.Start
    /home/caddy/go/src/github.com/abiosoft/caddy-git/service.go:43 +0x110

goroutine 89 [chan receive, 960 minutes]:
github.com/mholt/caddy/server.newGracefulListener.func1(0xc8204ef920)
    /tmp/custombuild_3675_930774383/src/github.com/mholt/caddy/server/graceful.go:14 +0x3e
created by github.com/mholt/caddy/server.newGracefulListener
    /tmp/custombuild_3675_930774383/src/github.com/mholt/caddy/server/graceful.go:19 +0xe3

[ianlancetaylor]

I just ran g.sh for 30 minutes with no problems on a FreeBSD 10.1 system running on GCE. How long does it usually take you to see a problem?

[rsc]

Seems likely dup of #15658. Also seems likely to be a kernel problem.

[rsc]

Cannot reproduce, and reporting a problem in a possibly very different code base (Go from three months ago). Closing. Please file a new issue if new crashes arise.