failed to compile testmain package:

[RichardKnop]

I am getting a compilation error when running godog on example folder.

Go version:

$ go version
go version go1.7.4 darwin/amd64

I installed godog:

$ go get -u github.com/DATA-DOG/godog

Then I go into the godogs example directory:

$ cd $GOPATH
$ cd src/go/src/github.com/DATA-DOG/godog/examples/godogs

And run godog:

$ godog
failed to compile testmain package:

[RichardKnop]

Sorry I was supposed to install godog like this:

go get -u github.com/DATA-DOG/godog/cmd/godog

[l3pp4rd]

Hi, yes, you might have had outdated version, the compilation process was improved from 0.5.x versions, cheers

[sadir]

Hi @l3pp4rd, I'm seeing this issue attempting to install godog.

Go version:

$ go version
go version go1.8 darwin/amd64

Install godog:

$ go get github.com/DATA-DOG/godog/cmd/godog

Try and run the example:

$ cd $GOPATH/src/github.com/DATA-DOG/godog/examples/godogs
$ godog
failed to compile testmain package:

$

Any ideas?

[l3pp4rd]

Hi, try updating it.

go get -u github.com/DATA-DOG/godog

Then reinstall the command:

go install github.com/DATA-DOG/godog/cmd/godog

[sadir]

Thanks @l3pp4rd but it didn't work.

$ go get -u github.com/DATA-DOG/godog
$ go install github.com/DATA-DOG/godog/cmd/godog
$ cd $GOPATH/src/github.com/DATA-DOG/godog/examples/godogs/
$ godog
failed to compile testmain package:

$

Although it was my first install beforehand so I guess should have been up to date.

Not sure if this helps but go test is fine in the same directory:

$ go test
... 3

1 scenarios (1 passed)
3 steps (3 passed)
170.267µs

Randomized with seed: 1501488293945982168
testing: warning: no tests to run
PASS
ok      github.com/DATA-DOG/godog/examples/godogs   0.008s

[l3pp4rd]

hm, I do not have a mac to find the reason. But could you try to run godog like:

GOARCH=amd64 GOOS=darwin godog

Just in case it chooses wrong os. Or it may not have permissions to compile the dependent packages

[sadir]

OK still no luck, I tried:

$ GOARCH=amd64 GOOS=darwin godog
failed to compile testmain package:
$ sudo GOPATH=/Users/morgan/code/go /Users/morgan/code/go/bin/godog
Password:
failed to compile testmain package:
$ sudo GOPATH=/Users/morgan/code/go GOARCH=amd64 GOOS=darwin /Users/morgan/code/go/bin/godog
failed to compile testmain package:

:(

Can you explain the logic that means we reach this error to me please in case it's something stupid I've done? Looks like it's in builder.go?

[l3pp4rd]

yes it is in builder.go. maybe you could try to figure out it in this func https://github.com/DATA-DOG/godog/blob/master/builder.go#L254 it somewhere fails and sadly there is no decent message

[l3pp4rd]

ah sorry, not that location, the error is from here: https://github.com/DATA-DOG/godog/blob/master/builder.go#L176

[l3pp4rd]

guess there is something special with macos build command arguments https://github.com/DATA-DOG/godog/blob/master/builder.go#L159

could you run go build -n in this godogs example and paste me the output, I may find what is different

[sadir]

Here you go!

$ cd $GOPATH/src/github.com/DATA-DOG/godog/examples/godogs/
$ go build -n

#
# github.com/DATA-DOG/godog/examples/godogs
#

mkdir -p $WORK/github.com/DATA-DOG/godog/examples/godogs/_obj/
mkdir -p $WORK/github.com/DATA-DOG/godog/examples/godogs/_obj/exe/
cd /Users/morgan/code/go/src/github.com/DATA-DOG/godog/examples/godogs
/Users/morgan/.asdf/installs/golang/1.8/go/pkg/tool/darwin_amd64/compile -o $WORK/github.com/DATA-DOG/godog/examples/godogs.a -trimpath $WORK -p main -complete -buildid ee5a704374a35e93fe5542af30ddc38f23f84486 -D _/Users/morgan/code/go/src/github.com/DATA-DOG/godog/examples/godogs -I $WORK -pack ./godogs.go
cd .
/Users/morgan/.asdf/installs/golang/1.8/go/pkg/tool/darwin_amd64/link -o $WORK/github.com/DATA-DOG/godog/examples/godogs/_obj/exe/a.out -L $WORK -extld=clang -buildmode=exe -buildid=ee5a704374a35e93fe5542af30ddc38f23f84486 $WORK/github.com/DATA-DOG/godog/examples/godogs.a
mv $WORK/github.com/DATA-DOG/godog/examples/godogs/_obj/exe/a.out godogs

[l3pp4rd]

well, looks the same apart from this commented line https://github.com/DATA-DOG/godog/blob/master/builder.go#L159 but this should not impact it I think, you may try uncomment it. But most likely I will need to get a macbook to test it

[sadir]

Sorry that doesn't look to have fixed it either:

$ cat $GOPATH/src/github.com/DATA-DOG/godog/builder.go
package godog

import (
...
        // compile godog testmain package archive
    // we do not depend on CGO so a lot of checks are not necessary
    testMainPkgOut := filepath.Join(testdir, "main.a")
    args := []string{
        "-o", testMainPkgOut,
        "-trimpath", workdir,
        "-p", "main",
        "-complete",
    }
    // if godog library is in vendor directory
    // link it with import map
...
}
$ go install github.com/DATA-DOG/godog/cmd/godog
$ cd $GOPATH/src/github.com/DATA-DOG/godog/examples/godogs/
$ godog
failed to compile testmain package:

I don't mind doing a screenshare with you or something to help debug? I literally just started learning Go so I may not be much help but you can work through me.

[sadir]

I've had a friend install it on his mac and it works which suggests a difference between our setups is the problem. Let me check it out.

[sadir]

Alright update time.

The first thing I did was to update my go version from 1.8 to 1.8.3. This didn't fix it.

Next I upgraded from OSX Yosmite to Sierra. This didn't fix it.

In the end what I did was to add two lines into builder.go:

  163   // if godog library is in vendor directory
  164   // link it with import map
  165   if i := strings.LastIndex(godogPkg.ImportPath, "vendor/"); i != -1 {
  166     args = append(args, "-importmap", godogImportPath+"="+godogPkg.ImportPath)
  167   }
  168   for _, inc := range pkgDirs {
  169     args = append(args, "-I", inc)
  170   }
  171   args = append(args, "-pack", testmain)
+ 172   fmt.Printf("%+v\n", compiler)
+ 173   fmt.Printf("%+v\n", args)
  174   cmd = exec.Command(compiler, args...)
  175   cmd.Env = os.Environ()
  176   out, err = cmd.CombinedOutput()
  177   if err != nil {
  178     return fmt.Errorf("failed to compile testmain package:\n%s", string(out))
  179   }

Which got me this output:

$ go install github.com/DATA-DOG/godog/cmd/godog
$ godog
/usr/local/go/pkg/tool/darwin_amd64/compile
[-o /var/folders/ll/2bytty0j6h77396nt5htyc7w0000gn/T/go-build220721685/github.com/DATA-DOG/godog/_test/main.a -p main -complete -I /var/folders/ll/2bytty0j6h77396nt5htyc7w0000gn/T/go-build220721685 -I /var/folders/ll/2bytty0j6h77396nt5htyc7w0000gn/T/go-build220721685/github.com/DATA-DOG/godog/_test -I /Users/morgan/code/go/pkg/darwin_amd64 -pack /var/folders/ll/2bytty0j6h77396nt5htyc7w0000gn/T/go-build220721685/github.com/DATA-DOG/godog/_test/_testmain.go]
failed to compile testmain package:

I tried running the command myself:

$ /usr/local/go/pkg/tool/darwin_amd64/compile -o /var/folders/ll/2bytty0j6h77396nt5htyc7w0000gn/T/go-build220721685/github.com/DATA-DOG/godog/_test/main.a -p main -complete -I /var/folders/ll/2bytty0j6h77396nt5htyc7w0000gn/T/go-build220721685 -I /var/folders/ll/2bytty0j6h77396nt5htyc7w0000gn/T/go-build220721685/github.com/DATA-DOG/godog/_test -I /Users/morgan/code/go/pkg/darwin_amd64 -pack /var/folders/ll/2bytty0j6h77396nt5htyc7w0000gn/T/go-build220721685/github.com/DATA-DOG/godog/_test/_testmain.go
-bash: /usr/local/go/pkg/tool/darwin_amd64/compile: No such file or directory

But /usr/local/go/pkg/tool/darwin_amd64/compile doesn't exist on my system.

The problem looks to be that I am using the asdf version manager which means go is installed here:

$ which go
/Users/morgan/.asdf/shims/go

However the asdf plugin hadn't recommended updating my $GOROOT, therefore I still had the default /usr/local/go/. Hence the variable compiler was /usr/local/go/pkg/tool/darwin_amd64/compile.

I guess my only feedback here is that if the error message -bash: /usr/local/go/pkg/tool/darwin_amd64/compile: No such file or directory had propagated up somehow I may have figured this out earlier?

Anyway thanks for taking the time to help me out - this was an interesting way to learn about how go works at least! I'll be opening an issue on asdf to ask for guidance on setting $GOROOT.

[l3pp4rd]

hm, but when you run go test it compiles a test package fine, that means go internally somehow detects and finds the compiler tool - godog should too, so this may be improved. Another point is the error, will need to check whether that can be more explicit. thanks for the details

[sadir]

Actually, you're right. Here is the result of go env before I added GOROOT to my environment variables.

$  go env
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/morgan/code/go"
GORACE=""
GOROOT="/Users/morgan/.asdf/installs/golang/1.8/go"
GOTOOLDIR="/Users/morgan/.asdf/installs/golang/1.8/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fno-common"
CXX="clang++"
CGO_ENABLED="1"
PKG_CONFIG="pkg-config"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"

And here is an explanation of how that gets set.

It looks like this line calls the go/build function which picks up the 'incorrect' value from runtime.GOROOT. I don't know why that variable would be different from go env but I'll do some digging if I find the time.

[sadir]

Another update, I've stolen a small bit of code from here to test those go/build results.

package main

import (
    "fmt"
    "go/build"
    "runtime"
)

var buildContext = build.Default

func main() {
    fmt.Println("GOROOT runtime: ", runtime.GOROOT())
    fmt.Println("GOROOT go/build: ", buildContext.GOROOT)
}

Here is what I get:

$ goroot_tester
GOROOT runtime:  /usr/local/go
GOROOT go/build:  /usr/local/go

As we expected based on the behaviour we have seen so far. I thought to have a look at brew just to understand what I have in there:

$ brew list
...lots of stuff...
glide
go

Fair enough, I have two installs of go, so let's simplify things for debugging by removing the brew installed go.

$ brew uninstall go
Error: Refusing to uninstall /usr/local/Cellar/go/1.8.3
because it is required by glide, which is currently installed.
You can override this and force removal with:
  brew uninstall --ignore-dependencies go
$ brew uninstall glide
Uninstalling /usr/local/Cellar/glide/0.12.3... (6 files, 10MB)
$ brew uninstall go
Uninstalling /usr/local/Cellar/go/1.8.3... (7,035 files, 282.0MB)
$ goroot_tester
GOROOT runtime:  /Users/morgan/.asdf/installs/golang/1.8.3/go/
GOROOT go/build:  /Users/morgan/.asdf/installs/golang/1.8.3/go
$ godog

No scenarios
No steps
3µs

Wahey! That's better. However, I would like glide for dependency management, so I reinstalled it (which reinstalls go as a dependency) and then we are back to the old behaviour.

So even though the version of go in use is the version provided by asdf, and the fact that brew installs go to /usr/local/opt/golang/libexec/bin/go, the runtime.GOROOT and buildContext.GOROOT both end up as /usr/local/go despite that not existing on my system. I can only assume that the distribution of go provided by asdf was compiled with /usr/local/go as the GOROOT at the time?

Either way I am going to set my own GOROOT as that worked before.

[l3pp4rd]

Hi @sadir thanks for all the help, if you update your package that should work fine now and if it doesn't at least print a good error message ;)