search

NixOS / nixpkgs

Nix Packages collection & NixOS
MIT License
18.05k stars 14.08k forks source link

Go compiler is built using clang, but buildGoPackage function then uses gcc, which can clash #25626

Closed javgh closed 7 years ago

javgh commented 7 years ago

Issue description

The NixOS go package uses clang while building the compiler. When that go compiler later is used to compile go programs which make use of C bindings the go compiler will also invoke a C compiler. It seems that this C compiler needs to match the one that was used at build time. If one uses stdenv.mkDerivation or buildGoPackage one gets gcc instead, which leads to failed builds in some cases.

Steps to reproduce

I have an example Go package over here: https://github.com/javgh/nixpkgs/blob/master/pkgs/applications/altcoins/go-ethereum.nix . For that expression I manually set CC to clang to make it work. Removing that preBuild step leads to a vanilla buildGoPackage environment with gcc and the build fails.

Tweaking CC in the package directly works for me, but it would probably make more sense to set this correctly already in buildGoPackage. I'm pretty new to NixOS, so I haven't yet attempted to do that myself.

Technical details

rht commented 7 years ago

Yet the line https://github.com/NixOS/nixpkgs/blob/19ebff1d95dae17cff81f8e9c133d82827317ed2/pkgs/development/compilers/go/1.8.nix#L137 says $CC is cc unless on Darwin.

rht commented 7 years ago

(I usually build go-ethereum with go get in order to use the HEAD version)

javgh commented 7 years ago

You are right, I'm not sure what made me think that clang is used, I jumped to some conclusions. Sorry for the misleading title, I'm just very confused now.

Can you successfully compile this file: https://raw.githubusercontent.com/golang/go/master/misc/cgo/errors/issue14669.go ?

When I do this within nixshell -p go gcc it fails, but when I use nixshell -p go clang it works. I guess that made me think that clang is the matching compiler. But why doesn't it work with the matching gcc compiler but does in fact work with, clang, a different compiler than used at build time?

rht commented 7 years ago

Can you successfully compile this file: https://raw.githubusercontent.com/golang/go/master/misc/cgo/errors/issue14669.go ?

# command-line-arguments
cgo-dwarf-inference:2:8: error: enumerator value for '__cgo_enum__0' is not an integer constant
cgo-dwarf-inference:5:8: error: enumerator value for '__cgo_enum__2' is not an integer constant
cgo-dwarf-inference:8:2: error: initializer element is not constant
cgo-dwarf-inference:8:2: note: (near initialization for '__cgodebug_data[0]')
cgo-dwarf-inference:10:2: error: initializer element is not constant
cgo-dwarf-inference:10:2: note: (near initialization for '__cgodebug_data[2]')

Further check,

ldd `which go`
    linux-vdso.so.1 (0x00007fffb1770000)
    libpthread.so.0 => /nix/store/63gvnrj4z154kpyjpskl6s0hwmyx9x0w-glibc-2.25/lib/libpthread.so.0 (0x00007f236c314000)
    libc.so.6 => /nix/store/63gvnrj4z154kpyjpskl6s0hwmyx9x0w-glibc-2.25/lib/libc.so.6 (0x00007f236bf75000)
    /nix/store/63gvnrj4z154kpyjpskl6s0hwmyx9x0w-glibc-2.25/lib/ld-linux-x86-64.so.2 (0x00007f236c532000)

proves that go is compiled with gcc by default on NixOS.

Running nix-shell -p go clang wouldn't do since I have go installed deterministically from configuration.nix. I had also uncommented go from configuration.nix and rerun nix-shell -p go clang, to no avail.

note: here is an alternative path for gcc for now (that you had figured out the clang one but I can't reproduce), I have geth running just fine after a make from the source dir.

javgh commented 7 years ago

So you see the error as well for issue14669.go, but are able to successfully run make in the go-ethereum repository?

This is what I see when I add go and gcc to configuration.nix and attempt to build the master branch from go-ethereum:

git clone https://github.com/ethereum/go-ethereum.git
cd go-ethereum
make

build/env.sh go run build/ci.go install ./cmd/geth
>>> /nix/store/k114bnw0gzag4b07492dgxi2gs858xrl-go-1.8/share/go/bin/go install -ldflags -X main.gitCommit=40976ea1a009f1358bcc6f85db97efd61868dd4f -v ./cmd/geth
github.com/ethereum/go-ethereum/common/hexutil
github.com/ethereum/go-ethereum/common/math
github.com/ethereum/go-ethereum/rlp
github.com/ethereum/go-ethereum/vendor/github.com/go-stack/stack
github.com/ethereum/go-ethereum/vendor/gopkg.in/karalabe/cookiejar.v2/collections/prque
github.com/ethereum/go-ethereum/crypto/sha3
github.com/ethereum/go-ethereum/vendor/github.com/aristanetworks/goarista/monotime
github.com/ethereum/go-ethereum/vendor/github.com/rcrowley/go-metrics
github.com/ethereum/go-ethereum/crypto/secp256k1
github.com/ethereum/go-ethereum/crypto/randentropy
github.com/ethereum/go-ethereum/common/mclock
github.com/ethereum/go-ethereum/event
github.com/ethereum/go-ethereum/vendor/github.com/pborman/uuid
github.com/ethereum/go-ethereum/log
github.com/ethereum/go-ethereum/common
github.com/ethereum/go-ethereum/vendor/github.com/rjeczalik/notify
github.com/ethereum/go-ethereum/vendor/golang.org/x/crypto/pbkdf2
github.com/ethereum/go-ethereum/cmd/internal/browser
github.com/ethereum/go-ethereum/vendor/golang.org/x/crypto/scrypt
github.com/ethereum/go-ethereum/params
github.com/ethereum/go-ethereum/vendor/github.com/edsrzf/mmap-go
github.com/ethereum/go-ethereum/common/bitutil
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/util
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/comparer
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/storage
github.com/ethereum/go-ethereum/vendor/github.com/golang/snappy
github.com/ethereum/go-ethereum/vendor/github.com/hashicorp/golang-lru/simplelru
github.com/ethereum/go-ethereum/vendor/golang.org/x/net/context
github.com/ethereum/go-ethereum/vendor/github.com/hashicorp/golang-lru
github.com/ethereum/go-ethereum/vendor/github.com/rs/xhandler
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/cache
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/filter
github.com/ethereum/go-ethereum/vendor/golang.org/x/net/websocket
github.com/ethereum/go-ethereum/vendor/gopkg.in/fatih/set.v0
github.com/ethereum/go-ethereum/vendor/golang.org/x/crypto/ripemd160
github.com/ethereum/go-ethereum/vendor/github.com/rs/cors
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/opt
github.com/ethereum/go-ethereum/vendor/github.com/huin/goupnp/httpu
github.com/ethereum/go-ethereum/vendor/github.com/huin/goupnp/scpd
github.com/ethereum/go-ethereum/vendor/github.com/huin/goupnp/soap
github.com/ethereum/go-ethereum/vendor/golang.org/x/net/html/atom
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/errors
github.com/ethereum/go-ethereum/vendor/github.com/huin/goupnp/ssdp
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/iterator
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/journal
github.com/ethereum/go-ethereum/vendor/golang.org/x/net/html
github.com/ethereum/go-ethereum/trie
github.com/ethereum/go-ethereum/vendor/github.com/rcrowley/go-metrics/exp
github.com/ethereum/go-ethereum/rpc
github.com/ethereum/go-ethereum/vendor/golang.org/x/text/encoding/internal/identifier
github.com/ethereum/go-ethereum/vendor/golang.org/x/text/transform
github.com/ethereum/go-ethereum/vendor/golang.org/x/text/internal/utf8internal
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/memdb
github.com/ethereum/go-ethereum/metrics
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/table
github.com/ethereum/go-ethereum/vendor/golang.org/x/text/encoding
github.com/ethereum/go-ethereum/vendor/golang.org/x/text/runes
github.com/ethereum/go-ethereum/vendor/golang.org/x/text/internal/tag
github.com/ethereum/go-ethereum/vendor/golang.org/x/text/encoding/internal
github.com/ethereum/go-ethereum/vendor/golang.org/x/text/language
github.com/ethereum/go-ethereum/vendor/golang.org/x/text/encoding/charmap
github.com/ethereum/go-ethereum/vendor/golang.org/x/text/encoding/japanese
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb
github.com/ethereum/go-ethereum/vendor/golang.org/x/text/encoding/korean
github.com/ethereum/go-ethereum/vendor/golang.org/x/text/encoding/simplifiedchinese
github.com/ethereum/go-ethereum/vendor/golang.org/x/text/encoding/traditionalchinese
github.com/ethereum/go-ethereum/vendor/golang.org/x/text/encoding/unicode
github.com/ethereum/go-ethereum/vendor/github.com/jackpal/go-nat-pmp
github.com/ethereum/go-ethereum/p2p/netutil
github.com/ethereum/go-ethereum/vendor/gopkg.in/sourcemap.v1/base64vlq
github.com/ethereum/go-ethereum/vendor/github.com/robertkrimen/otto/token
github.com/ethereum/go-ethereum/vendor/github.com/robertkrimen/otto/dbg
github.com/ethereum/go-ethereum/vendor/github.com/robertkrimen/otto/registry
github.com/ethereum/go-ethereum/vendor/gopkg.in/sourcemap.v1
github.com/ethereum/go-ethereum/vendor/github.com/karalabe/hid
github.com/ethereum/go-ethereum/log/term
github.com/ethereum/go-ethereum/vendor/github.com/mattn/go-colorable
github.com/ethereum/go-ethereum/vendor/github.com/robertkrimen/otto/file
github.com/ethereum/go-ethereum/vendor/gopkg.in/urfave/cli.v1
github.com/ethereum/go-ethereum/les/flowcontrol
github.com/ethereum/go-ethereum/vendor/github.com/robertkrimen/otto/ast
github.com/ethereum/go-ethereum/internal/jsre/deps
github.com/ethereum/go-ethereum/vendor/golang.org/x/text/encoding/htmlindex
github.com/ethereum/go-ethereum/vendor/golang.org/x/net/html/charset
github.com/ethereum/go-ethereum/vendor/github.com/mattn/go-isatty
github.com/ethereum/go-ethereum/internal/web3ext
github.com/ethereum/go-ethereum/vendor/github.com/peterh/liner
github.com/ethereum/go-ethereum/vendor/github.com/robertkrimen/otto/parser
github.com/ethereum/go-ethereum/vendor/github.com/fatih/color
github.com/ethereum/go-ethereum/vendor/github.com/huin/goupnp
github.com/ethereum/go-ethereum/vendor/golang.org/x/tools/go/ast/astutil
github.com/ethereum/go-ethereum/vendor/github.com/huin/goupnp/dcps/internetgateway1
github.com/ethereum/go-ethereum/vendor/github.com/huin/goupnp/dcps/internetgateway2
github.com/ethereum/go-ethereum/ethdb
github.com/ethereum/go-ethereum/internal/debug
github.com/ethereum/go-ethereum/vendor/golang.org/x/tools/imports
github.com/ethereum/go-ethereum/vendor/github.com/robertkrimen/otto
github.com/ethereum/go-ethereum/vendor/github.com/maruel/panicparse/stack
github.com/ethereum/go-ethereum/vendor/github.com/mattn/go-runewidth
github.com/ethereum/go-ethereum/vendor/github.com/mitchellh/go-wordwrap
github.com/ethereum/go-ethereum/vendor/github.com/naoina/go-stringutil
github.com/ethereum/go-ethereum/vendor/github.com/nsf/termbox-go
github.com/ethereum/go-ethereum/vendor/github.com/naoina/toml/ast
github.com/ethereum/go-ethereum/p2p/nat
github.com/ethereum/go-ethereum/vendor/github.com/naoina/toml
github.com/ethereum/go-ethereum/vendor/github.com/gizak/termui
# github.com/ethereum/go-ethereum/vendor/github.com/karalabe/hid
cgo-dwarf-inference:2:8: error: enumerator value for '__cgo_enum__0' is not an integer constant
cgo-dwarf-inference:10:8: error: enumerator value for '__cgo_enum__7' is not an integer constant
cgo-dwarf-inference:18:2: error: initializer element is not constant
cgo-dwarf-inference:18:2: note: (near initialization for '__cgodebug_data[0]')
cgo-dwarf-inference:25:2: error: initializer element is not constant
cgo-dwarf-inference:25:2: note: (near initialization for '__cgodebug_data[7]')
github.com/ethereum/go-ethereum/internal/jsre
github.com/ethereum/go-ethereum/console
github.com/ethereum/go-ethereum/crypto
github.com/ethereum/go-ethereum/core/types
github.com/ethereum/go-ethereum/crypto/ecies
github.com/ethereum/go-ethereum/p2p/discover
github.com/ethereum/go-ethereum/p2p/discv5
github.com/ethereum/go-ethereum/accounts/abi
github.com/ethereum/go-ethereum
github.com/ethereum/go-ethereum/core/state
github.com/ethereum/go-ethereum/core/vm
github.com/ethereum/go-ethereum/accounts
github.com/ethereum/go-ethereum/ethclient
github.com/ethereum/go-ethereum/accounts/keystore
github.com/ethereum/go-ethereum/p2p
github.com/ethereum/go-ethereum/consensus/misc
github.com/ethereum/go-ethereum/consensus
github.com/ethereum/go-ethereum/eth/downloader
github.com/ethereum/go-ethereum/consensus/ethash
github.com/ethereum/go-ethereum/consensus/clique
github.com/ethereum/go-ethereum/eth/fetcher
github.com/ethereum/go-ethereum/accounts/abi/bind
github.com/ethereum/go-ethereum/whisper/whisperv5
github.com/ethereum/go-ethereum/core
github.com/ethereum/go-ethereum/eth/filters
github.com/ethereum/go-ethereum/internal/ethapi
github.com/ethereum/go-ethereum/miner
github.com/ethereum/go-ethereum/light
github.com/ethereum/go-ethereum/eth/gasprice
util.go:44: exit status 2
exit status 1
make: *** [Makefile:15: geth] Error 1

But this series of commands work for you with go and gcc in configuration.nix?

rht commented 7 years ago

So you see the error as well for issue14669.go, but are able to successfully run make in the go-ethereum repository?

Yes

Update: As stated, I was able to build from the earlier version, but when I pulled from the latest master, I found the same err:

# github.com/ethereum/go-ethereum/vendor/github.com/karalabe/hid
cgo-dwarf-inference:2:8: error: enumerator value for '__cgo_enum__0' is not an integer constant
cgo-dwarf-inference:10:8: error: enumerator value for '__cgo_enum__7' is not an integer constant
cgo-dwarf-inference:18:2: error: initializer element is not constant
cgo-dwarf-inference:18:2: note: (near initialization for '__cgodebug_data[0]')
cgo-dwarf-inference:25:2: error: initializer element is not constant
cgo-dwarf-inference:25:2: note: (near initialization for '__cgodebug_data[7]')
rht commented 7 years ago

From git-bisect, it is due to this commit, in particular the line https://github.com/ethereum/go-ethereum/commit/6ec81352560e3e4268855115eda9d78f6e27875f#diff-d9b21908b8e4e705df4660d35ceacfdfR40.

javgh commented 7 years ago

I am going to close this issue, as it has a really misleading title and wrong speculation. I have written up a - hopefully - better description of the issue here: https://github.com/NixOS/nixpkgs/issues/25959 .