Closed codersguild closed 3 years ago
I don't think it's a problem with the GC Go toolchain. You'd probably be better off posting an issue on the Gollvm bug tracker instead.
The bug/issue tracker link at
https://go.googlesource.com/gollvm/#wheretofile
brings me golang/go
issues.
https://github.com/golang/go/issues
where I wrote my issue by mentioning gollvm
as asked.
Yes, this is the right issue tracker.
But there isn't enough information here. Please tell us precisely what you did and show us precisely what happened. Thanks.
I used external packages in my go program as shown below.
package main
import (
"crypto/x509"
"encoding/json"
"fmt"
"strconv"
"strings"
"time"
"github.com/hyperledger/fabric-chaincode-go/pkg/cid"
"github.com/hyperledger/fabric-chaincode-go/shim"
peer "github.com/hyperledger/fabric-protos-go/peer"
)
...
The last three are external packages from github.com/hyperledger
.
Now I passed it to llvm-goc
. I build gollvm
with ninja
as it is mentioned in gollvm
It gives the following errors.
payment_main.go:10:52: error: import file 'github.com/hyperledger/fabric-chaincode-go/pkg/cid' not found
payment_main.go:11:49: error: import file 'github.com/hyperledger/fabric-chaincode-go/shim' not found
payment_main.go:12:2: error: import file 'github.com/hyperledger/fabric-protos-go/peer' not found
I am vendoring the packages in a folder vendor
using GO11MODULE=on
and then specifying the required files in go.mod.
How do I make llvm-goc
use these external go modules when emitting IR
$ llvm-goc -S -emit-llvm -O3 payment_main.go
payment_main.go:10:52: error: import file 'github.com/hyperledger/fabric-chaincode-go/pkg/cid' not found
payment_main.go:11:49: error: import file 'github.com/hyperledger/fabric-chaincode-go/shim' not found
payment_main.go:12:2: error: import file 'github.com/hyperledger/fabric-protos-go/peer' not found
Don't build using llvm-goc
directly. Use the Go tool, and run go build
.
I tried using go build
by following the steps to use go work
output. It still gives the same error.
I am trying to do something like this
% go build -work -x mypackage.go 1> transcript.txt 2>&1
% egrep '(WORK=|llvm-goc -c)' transcript.txt
WORK=/tmp/go-build887931787
/t/bin/llvm-goc -c -g -m64 -fdebug-prefix-map=$WORK=/tmp/go-build \
-gno-record-gcc-switches -fgo-pkgpath=command-line-arguments \
-fgo-relative-import-path=/mygopath/src/tmp -o $WORK/b001/_go_.o \
-I $WORK/b001/_importcfgroot_ ./mypackage.go
% /t/bin/llvm-goc -c -g -m64 -fdebug-prefix-map=$WORK=/tmp/go-build \
-gno-record-gcc-switches -fgo-pkgpath=command-line-arguments \
-fgo-relative-import-path=/mygopath/src/tmp \
-I $WORK/b001/_importcfgroot_ -o mypackage.ll -S -emit-llvm \
./mypackage.go
% ls -l mypackage.ll
...
%
If you are using module mode (if the package you are trying to build has a go.mod
file), go build
should download the imports you need. If not you'll need to fetch them using go get
. But this use of the go tool doesn't have anything to do with GoLLVM.
Lets me elaborate on a scenario I am trying to fix at my end. I need llvm-goc
to generate LLVM IR
for a toy go
program as below.
package main
/*
#include <stdio.h>
#define MAX 150000
*/
import "C"
import "fmt"
func main() {
fmt.Println(C.MAX)
fmt.Println("Hello")
}
As you can see, I have imported "C"
here. It is just exemplary here.
I am building it with go
. which generates a binary.
$ go build main.go
$ go run main.go
150000
Hello
So go
is able to detect the import. Now I want to create an LLVM IR for this program. I have build ninja gollvm
and using llvm-goc
to generate an IR.
$ llvm-goc -S -emit-llvm main.go -o main.ll
The error I am getting is :
$ main.go:7:9: error: import file 'C' not found
$ main.go:11:14: error: reference to undefined name 'C'
I need a way to fix this. I am getting the same error when I am doing it this way.
$ go build -work -x main.go 1> transcript.txt 2>&1
$ egrep '(WORK=|llvm-goc -c)' transcript.txt
WORK=/tmp/go-build887931787
/t/bin/llvm-goc -c -g -m64 -fdebug-prefix-map=$WORK=/tmp/go-build \
-gno-record-gcc-switches -fgo-pkgpath=command-line-arguments \
-fgo-relative-import-path=/mygopath/src/tmp -o $WORK/b001/_go_.o \
-I $WORK/b001/_importcfgroot_ ./main.go
$ main.go:7:9: error: import file 'C' not found
$ main.go:11:14: error: reference to undefined name 'C'
Hi, I tried your test case, but I can't reproduce it. This is the version information I used to build gollvm:
llvm: ede6005e7092ddae454e4d365d8adefeaec1f5e3
gollvm: 0edc44a02852dd0d56cac35163a45634b497d6cd
gofrontend: ea58b8491064fbed18a220571a3043c38dccf7c7
Can you tell us what version you used?
I will share it asap.
I don't understand why you are pulling commands out of the go build -x
output and running them separately. Does go build
succeed? If so, use that. Don't try to pull commands out of it and expect them to work without also running all the other commands.
The go build
succeeds. It produces a correct transcript. But when I execute this step
$ egrep '(WORK=|llvm-goc -c)' transcript.txt
WORK=/tmp/go-build887931787
/t/bin/llvm-goc -c -g -m64 -fdebug-prefix-map=$WORK=/tmp/go-build \
-gno-record-gcc-switches -fgo-pkgpath=command-line-arguments \
-fgo-relative-import-path=/mygopath/src/tmp -o $WORK/b001/_go_.o \
-I $WORK/b001/_importcfgroot_ ./main.go
It gives me an error saying.
$ main.go:7:9: error: import file 'C' not found
$ main.go:11:14: error: reference to undefined name 'C'
I may be wrong in the way I am trying to generate the IR
. I am following the steps here.
https://go.googlesource.com/gollvm/#seetheir
My exact steps.
$ go build -work -x main.go 1> transcript_main.txt 2>&1
$ export $(egrep '(WORK=|llvm-goc -c)' transcript_main.txt)
$ llvmgoc -c -O2 -g -m64 -fdebug-prefix-map=$WORK=/tmp/go-build \
-gno-record-gcc-switches -fgo-relative-import-path=$GOPATH \
-o $WORK/b001/_go_.o -I $WORK/b001/_importcfgroot_ ./main.go
GOAPTH
is configured to use the gollvm
libgo library packages.
Can you paste the exact commands you run, without filtering? (It could be lengthy, and that is fine.)
If you run the commands from go build -x
manually, you need to run all the commands, not only the ones with llvm-goc.
I didn't use any other command . I shall share a full dump of what transcript.txt contains.
Hey, have you solve the problem yet? I met the same problem and i don't know how to do with it too.
No. It is still unclear what the problem is. It could be helpful if any of you share the exact command for how to reproduce this.
I thing i have figure out the problem. I tried to use gollvm to compile runc to IR, I use this go build command: $ $(GO_BUILD) -gccgoflags -### -x -work -o runc . use -gccgoflags to pass -### to llvm-goc so llvm-goc just print the command it use to compile the project like that
WORK=/tmp/go-build179018028 go build: when using gccgo toolchain, please pass linker flags using -gccgoflags, not -ldflags mkdir -p $WORK/b001/ cat >$WORK/b001/_gomod_.go << 'EOF' # internal package main import _ "unsafe" //go:linkname __set_debug_modinfo__ runtime.setmodinfo func __set_debug_modinfo__(string) func init() { __set_debug_modinfo__("0w\xaf\f\x92t\b\x02A\xe1\xc1\a\xe6\xd6\x18\xe6path\tgithub.com/opencontainers/runc\nmod\tgithub.com/opencontainers/runc\t(devel)\t\ndep\tgithub.com/checkpoint-restore/go-criu/v4\tv4.1.0\t\ndep\tgithub.com/cilium/ebpf\tv0.0.0-20200702112145-1c8d4c9ef775\t\ndep\tgithub.com/containerd/console\tv1.0.0\t\ndep\tgithub.com/coreos/go-systemd/v22\tv22.1.0\t\ndep\tgithub.com/cpuguy83/go-md2man/v2\tv2.0.0-20190314233015-f79a8a8ca69d\t\ndep\tgithub.com/cyphar/filepath-securejoin\tv0.2.2\t\ndep\tgithub.com/docker/go-units\tv0.4.0\t\ndep\tgithub.com/godbus/dbus/v5\tv5.0.3\t\ndep\tgithub.com/golang/protobuf\tv1.4.2\t\ndep\tgithub.com/moby/sys/mountinfo\tv0.1.3\t\ndep\tgithub.com/mrunalp/fileutils\tv0.0.0-20200520151820-abd8a0e76976\t\ndep\tgithub.com/opencontainers/runtime-spec\tv1.0.3-0.20200728170252-4d89ac9fbff6\t\ndep\tgithub.com/opencontainers/selinux\tv1.6.0\t\ndep\tgithub.com/pkg/errors\tv0.9.1\t\ndep\tgithub.com/russross/blackfriday/v2\tv2.0.1\t\ndep\tgithub.com/seccomp/libseccomp-golang\tv0.9.1\t\ndep\tgithub.com/shurcooL/sanitized_anchor_name\tv1.0.0\t\ndep\tgithub.com/sirupsen/logrus\tv1.6.0\t\ndep\tgithub.com/syndtr/gocapability\tv0.0.0-20200815063812-42c35b437635\t\ndep\tgithub.com/urfave/cli\tv1.22.1\t\ndep\tgithub.com/vishvananda/netlink\tv1.1.0\t\ndep\tgithub.com/vishvananda/netns\tv0.0.0-20191106174202-0a2b9b5464df\t\ndep\tgithub.com/willf/bitset\tv1.1.11-0.20200630133818-d5bec3311243\t\ndep\tgolang.org/x/sys\tv0.0.0-20200728102440-3e129f6d46b1\t\ndep\tgoogle.golang.org/protobuf\tv1.23.0\t\n\xf92C1\x86\x18 r\x00\x82B\x10A\x16\xd8\xf2") } EOF
and i found a key command that llvm-goc use to generate go.o:
llvm-goc -c -O2 -g -m64 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -o $WORK/b001/_go_.o -fPIE -I $WORK/b001/_importcfgroot_ -### ./checkpoint.go ./create.go ./delete.go ./events.go ./exec.go ./init.go ./kill.go ./list.go ./main.go ./notify_socket.go ./pause.go ./ps.go ./restore.go ./rlimit_linux.go ./rootless_linux.go ./run.go ./signals.go ./spec.go ./start.go ./state.go ./tty.go ./update.go ./utils.go ./utils_linux.go $WORK/b001/_gomod_.go
so i modify this command: add -S -emit-llvm and change go.o to go.ll and finally successfully generate it.
In short,You can use go build -gccgoflags -### -x -work
to compile your project and it will show the command that llvm-goc used.
Looking for the command llvm-goc used to generate the .o file and modify it to generate .ll file.
Cool ! I have to try it out then. Thanks for your analysis.
Closing this issue out, I don't see anything actionable here.
What version of Go are you using (
go version
)?Does this issue reproduce with the latest release?
Yes. I am not able to use/import the external go packages with
gccgo
orllvm-goc
. I want to generate LLVM IR for a go program that uses an external package. It errors out sayingpackage not found
.What operating system and processor architecture are you using (
go env
)?go env
OutputWhat did you do?
I want to generate LLVM IR for a go program that uses an external package say a package/module like
github.com/hyperledger/fabric-chaincode-go
. I am usingGO11MODULE
to download and vendor the packages. It still fails saying the modules are not found. All required files are in the same folder.What did you expect to see?
Proper build and LLVM IR of my go program
What did you see instead?
Error, External package not found during build process of
gccgo
orllvm-goc
.