search

karalabe / xgo

Go CGO cross compiler
MIT License
2.14k stars 280 forks source link

MIPS support? #46

Open polyfractal opened 8 years ago

polyfractal commented 8 years ago

Thanks for xgo, this looks really great! I was curious if MIPS support is possible? The 1.6 release notes mentioned that linux/mips64 and linux/mips64le were now experimentally supported: https://golang.org/doc/go1.6#ports.

I was hoping to crosscompile some go programs to my router :)

karalabe commented 8 years ago

Hmmm, will need to look into MIPS C/C++ toolchains too for this. Can't promise anything just yet but I'll look into it :)

polyfractal commented 8 years ago

Thanks! And no worries if it doesn't work out, I was just curious since it seemed semi-possible now that the go compiler supports it.

Thanks again! :)

karalabe commented 8 years ago

Currently Go doesn't support CGO linking on MIPS64x platforms :(

Relevant issues:

Relevant CLs:

Until these CLs get merged in I can't add cross compilation support. But will keep this issue open for future memo :)

polyfractal commented 8 years ago

Ah, that's a shame. Thanks for looking into it!

karalabe commented 8 years ago

Seems 1.7 has cgo support for MIPS too. Will investigate this again.

polyfractal commented 8 years ago

Oh awesome, thanks!

karalabe commented 8 years ago

Pling :) I've pushed in a change that should enable MIPS64 and MIPS64le support. Note, that you'll need the 1.7beta1 image at least as it's needed for CGO stuff. Also I don't have a MIPS platform to try out, so apart from checking that builds produce a binary, I couldn't actually try it out. Perhaps if you'd be so kind, I'd appreciate some feedback on this front.

PS: Docker hub seems to be many hours behind in building automated builds, so you'll probably need to build the images yourself for now if you want to try it out.

polyfractal commented 8 years ago

Woo! \o/

I'm on the road at the moment for a work trip, but I'll test it out as soon as I get home

vincent-olivert-riera commented 8 years ago

Hello @karalabe and @polyfractal ,

I was having an issue when trying to build the "flannel" package for MIPS, which needs go, and it was failing like this:

# runtime/cgo
/tmp/go-build396590679/runtime/cgo/_obj/gcc_util.o: In function `x_cgo_thread_start':
../../host/usr/lib/go/src/runtime/cgo/gcc_util.c:48: undefined reference to `_cgo_sys_thread_start'
../../host/usr/lib/go/src/runtime/cgo/gcc_util.c:48: undefined reference to `_cgo_sys_thread_start'
collect2: error: ld returned 1 exit status

That was with go-1.6.1. Then I updated go to 1.7beta2 and it worked fine.

polyfractal commented 8 years ago

Hi @karalabe, finally had a chance to give this a shot (sorry for the delay). I'm running into an error, but I think it's related to the layout of the target Go package, and not the xgo specifically.

I'm trying to compile MetricBeat. The various beats all live in sub-directories of the main Beats repo, and I think that may be causing the problem since it isn't at the "root" of the repo?

$ xgo github.com/elastic/beats/tree/master/metricbeat

Cross compiling github.com/elastic/beats/tree/master/metricbeat...
Fetching main repository github.com/elastic/beats/tree/master/metricbeat...
github.com/elastic/beats (download)
package github.com/elastic/beats/tree/master/metricbeat: cannot find package "github.com/elastic/beats/tree/master/metricbeat" in any of:
    /usr/local/go/src/github.com/elastic/beats/tree/master/metricbeat (from $GOROOT)
    /go/src/github.com/elastic/beats/tree/master/metricbeat (from $GOPATH)
    /go/src/github.com/elastic/beats/tree/master/metricbeat/Godeps/_workspace/src/github.com/elastic/beats/tree/master/metricbeat
    /go/src/github.com/elastic/beats/tree/master/Godeps/_workspace/src/github.com/elastic/beats/tree/master/metricbeat
    /go/src/github.com/elastic/beats/tree/Godeps/_workspace/src/github.com/elastic/beats/tree/master/metricbeat
    /go/src/github.com/elastic/beats/Godeps/_workspace/src/github.com/elastic/beats/tree/master/metricbeat
    /go/src/github.com/elastic/Godeps/_workspace/src/github.com/elastic/beats/tree/master/metricbeat
    /go/src/github.com/Godeps/_workspace/src/github.com/elastic/beats/tree/master/metricbeat
/build.sh: line 73: cd: /go/src/github.com/elastic/beats/tree/master/metricbeat: No such file or directory
2016/06/24 15:06:27 Failed to cross compile package: exit status 1.```
karalabe commented 8 years ago

@polyfractal Sorry, been having some crisis at my day job + been on a holiday afterwards. Seems your import path is wrong, should be just github.com/elastic/beats/metricbeat

polyfractal commented 7 years ago

Sorry, again, for the massive delay. Life and all that :)

I tried again with a fresh pull of the docker image, and this is the result:

xgo github.com/elastic/beats/metricbeat                         
Checking docker installation...  
Client:  
 Version:      1.12.1  
 API version:  1.24  
 Go version:   go1.6.2  
 Git commit:   23cf638  
 Built:        Tue, 27 Sep 2016 12:25:38 +1300  
 OS/Arch:      linux/amd64  

Server:  
 Version:      1.12.1  
 API version:  1.24  
 Go version:   go1.6.2  
 Git commit:   23cf638  
 Built:        Tue, 27 Sep 2016 12:25:38 +1300  
 OS/Arch:      linux/amd64  

Checking for required docker image karalabe/xgo-latest... found.  
Cross compiling github.com/elastic/beats/metricbeat...  
Fetching main repository github.com/elastic/beats/metricbeat...  
github.com/elastic/beats (download)  
Assembling toolchain for android-16/arm...
Bootstrapping android-16/arm...
Compiling for android-16/arm...
Assembling toolchain for android-16/386...
Bootstrapping android-16/386...
Compiling for android-16/386...
No API C header specified, skipping android-16/aar...
Compiling for linux/amd64...
Compiling for linux/386...
Bootstrapping linux/arm-5...
Compiling for linux/arm-5...
Cleaning up Go runtime for linux/arm-5...
Bootstrapping linux/arm-6...
Compiling for linux/arm-6...
Cleaning up Go runtime for linux/arm-6...
Bootstrapping linux/arm-7...
Compiling for linux/arm-7...
Cleaning up Go runtime for linux/arm-7...
Compiling for linux/arm64...
Compiling for linux/mips64...
# github.com/elastic/beats/metricbeat/module/docker/vendor/github.com/fsouza/go-dockerclient/external/github.com/docker/docker/pkg/system
module/docker/vendor/github.com/fsouza/go-dockerclient/external/github.com/docker/docker/pkg/system/stat_linux.go:13: cannot use s.Rdev (type uint32) as type uint64 in field value
2016/11/14 17:03:30 Failed to cross compile package: exit status 2.
polyfractal commented 7 years ago

So I had a closer look at this, and it's just a problem with one of the vendored dependencies. I forked the project, patched it with some build tags... but seem to be unable to get xgo to use the correct code. I even removed the offending file in my forked repo. Xgo still tries to compile it, making me think it's working off the parent repo instead:

xgo --branch mips --targets=*/mips64 --tags='mips64' --remote https://github.com/polyfractal/beats/  github.com/polyfractal/beats/metricbeat
Checking docker installation...
Client:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.2
 Git commit:   23cf638
 Built:        Tue, 27 Sep 2016 12:25:38 +1300
 OS/Arch:      linux/amd64

Server:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.2
 Git commit:   23cf638
 Built:        Tue, 27 Sep 2016 12:25:38 +1300
 OS/Arch:      linux/amd64

Checking for required docker image karalabe/xgo-latest... found.
Cross compiling github.com/polyfractal/beats/metricbeat...
Fetching main repository github.com/polyfractal/beats/metricbeat...
github.com/polyfractal/beats (download)
github.com/elastic/beats (download)
Switching over to remote https://github.com/polyfractal/beats/...
Fetching origin
HEAD is now at 2707369 Rewrite elasticsearch connection URL (#3058)
Switching over to branch mips...
HEAD is now at 1ad51ac Temporarily remove non-mips
Compiling for linux/mips64...
# github.com/elastic/beats/metricbeat/module/docker/vendor/github.com/fsouza/go-dockerclient/external/github.com/docker/docker/pkg/system
../../../elastic/beats/metricbeat/module/docker/vendor/github.com/fsouza/go-dockerclient/external/github.com/docker/docker/pkg/system/stat_linux.go:13: cannot use s.Rdev (type uint32) as type uint64 in field value
2016/11/23 14:02:36 Failed to cross compile package: exit status 2.

Notice that it's trying to compile stat_linux, which was removed in the commit 1ad51ac Temporarily remove non-mips. Xgo reports that the correct branch is checked out, so I'm not sure what's going on.

The command above has all the bell's and whistle's enabled in an attempt to fix the problem, but it should be noted that the basic command has similar behavior (xgo --branch mips --targets=*/mips64 github.com/polyfractal/beats/metricbeat)

jhammant commented 7 years ago

I've been having a bit of a poke around and have managed to get one of the other builds in your repo building:

xgo --branch mips --targets=*/mips64 github.com/polyfractal/beats/filebeat
Checking docker installation...
Client:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.2
 Git commit:   23cf638
 Built:        Tue, 27 Sep 2016 12:25:38 +1300
 OS/Arch:      linux/amd64

Server:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.2
 Git commit:   23cf638
 Built:        Tue, 27 Sep 2016 12:25:38 +1300
 OS/Arch:      linux/amd64

Checking for required docker image karalabe/xgo-latest... found.
Cross compiling github.com/polyfractal/beats/filebeat...
Fetching main repository github.com/polyfractal/beats/filebeat...
github.com/polyfractal/beats (download)
github.com/elastic/beats (download)
Switching over to branch mips...
HEAD is now at 1ad51ac Temporarily remove non-mips
Compiling for linux/mips64...
Cleaning up build environment...
root@mips:~/go/src/github.com# ls -ltr
total 16020
drwxr-xr-x 3 root root     4096 Nov 23 18:41 karalabe
drwxr-xr-x 3 root root     4096 Nov 23 18:42 elastic
-rwxr-xr-x 1 root root 16394480 Nov 24 15:29 filebeat-linux-mips64
root@mips:~/go/src/github.com# file filebeat-linux-mips64
filebeat-linux-mips64: ELF 64-bit MSB executable, MIPS, MIPS64 rel2 version 1, dynamically linked, interpreter /lib64/ld.so.1, BuildID[sha1]=07d41d37abf787f9a8991182b973659a8f48a6f3, for GNU/Linux 3.2.0, not stripped

Did your remove stat_linux from that component as well? Or is it just luck that's working? Will try running the binary on my MIPS hardware later.

polyfractal commented 7 years ago

Hm, interesting! stat_linux is a shared/vendored dependency for all the beats in that repo, but perhaps filebeat doesn't use it directly? Or maybe there's some kind of caching issue on my machine? Not sure, I'll wipe my local install and try again with a fresh image.

jhammant commented 7 years ago

Uploaded it onto my router - still getting issues however:

jon@ubnt:~$ ls -ltr
total 18628
-rwxr-xr-x    1 jon      users     16394480 Nov 25 11:27 filebeat-linux-mips64
jon@ubnt:~$ ./filebeat-linux-mips64
-vbash: ./filebeat-linux-mips64: No such file or directory
jon@ubnt:~$ ldd filebeat-linux-mips64
    not a dynamic executable
jon@ubnt:~$ uname -a
Linux ubnt 3.10.20-UBNT #1 SMP Fri Jul 29 16:51:50 PDT 2016 mips64 GNU/Linux
jon@ubnt:~$
root@mips:~# file filebeat-linux-mips64
filebeat-linux-mips64: ELF 64-bit MSB executable, MIPS, MIPS64 rel2 version 1, dynamically linked, interpreter /lib64/ld.so.1, BuildID[sha1]=07d41d37abf787f9a8991182b973659a8f48a6f3, for GNU/Linux 3.2.0, not stripped

Going to keep poking around to see if I can get it running.

karalabe commented 7 years ago

I just played around with MIPS 32 support (will pish soon)in an emulator, and apparently CGO/gcc may pull in a lot of dynamically linked libraries that won't be available on the device you're building to. I managed to solve those issue for my use case by adding --ldflags '-extldflags "-static"' to the build command.