Cross compiled builds

[bgentry]

Look what I made (git output redacted):

$ hkdist build all
build.go:153: git clone -b release https://github.com/kr/hk.git hk
...
build.go:153: git describe
build.go:121: GOOS=darwin GOARCH=386 go build -tags release -o hk
build.go:153: git clone -b release https://github.com/kr/hk.git hk
...
build.go:153: git describe
build.go:121: GOOS=darwin GOARCH=amd64 go build -tags release -o hk
build.go:153: git clone -b release https://github.com/kr/hk.git hk
...
build.go:153: git describe
build.go:121: GOOS=freebsd GOARCH=386 go build -tags release -o hk
build.go:153: git clone -b release https://github.com/kr/hk.git hk
...
build.go:153: git describe
build.go:121: GOOS=freebsd GOARCH=amd64 go build -tags release -o hk
build.go:153: git clone -b release https://github.com/kr/hk.git hk
...
build.go:153: git describe
build.go:121: GOOS=freebsd GOARCH=arm go build -tags release -o hk
build.go:153: git clone -b release https://github.com/kr/hk.git hk
...
build.go:153: git describe
build.go:121: GOOS=linux GOARCH=386 go build -tags release -o hk
build.go:153: git clone -b release https://github.com/kr/hk.git hk
...
build.go:153: git describe
build.go:121: GOOS=linux GOARCH=amd64 go build -tags release -o hk
build.go:153: git clone -b release https://github.com/kr/hk.git hk
...
build.go:153: git describe
build.go:121: GOOS=linux GOARCH=arm go build -tags release -o hk
build.go:153: git clone -b release https://github.com/kr/hk.git hk
...
build.go:153: git describe
build.go:121: GOOS=windows GOARCH=386 go build -tags release -o hk
build.go:153: git clone -b release https://github.com/kr/hk.git hk
...
build.go:153: git describe
build.go:121: GOOS=windows GOARCH=amd64 go build -tags release -o hk

It probably still needs some cleanup.

Of course, this requires that you have a cross-compiled Go build for each environment locally.

Do you think I should try doing this with CGO_ENABLED=0 for consistency across platforms?

[kr]

Oh hell yeah. I'll read this more thoroughly after dinner, but looks pretty good at first glance. :tada:

[bgentry]

One problem with this method is that each individual build re-clones the repo, and picks the latest release tag. That means that a single build process will not be guaranteed to build the same release version for every platform.

I've also wanted a way to force builds to happen on a specific tag as it would be useful for testing.

I'm thinking I should clone and pick the latest tag as a separate step before then building each binary sequentially.

Is there any extra cleanup I should do with the build command in that case, or should be compiler be smart enough to make sure there are no issues?

[bgentry]

@kr alright, I'm much happier with where this is at now. Please review :)

[bgentry]

New output:

$ hkdist build all
build.go:244: git clone -b release https://github.com/kr/hk.git hk-build
remote: Counting objects: 1013, done.
remote: Compressing objects: 100% (375/375), done.
remote: Total 1013 (delta 640), reused 994 (delta 624)
Receiving objects: 100% (1013/1013), 158.48 KiB | 110 KiB/s, done.
Resolving deltas: 100% (640/640), done.
build.go:244: git describe
build.go:128: Error building hk on 20130919 for darwin-386: already built: 20130919
build.go:128: Error building hk on 20130919 for darwin-amd64: already built: 20130919
build.go:128: Error building hk on 20130919 for freebsd-386: already built: 20130919
build.go:128: Error building hk on 20130919 for freebsd-amd64: already built: 20130919
build.go:128: Error building hk on 20130919 for freebsd-arm: already built: 20130919
build.go:128: Error building hk on 20130919 for linux-386: already built: 20130919
build.go:128: Error building hk on 20130919 for linux-amd64: already built: 20130919
build.go:128: Error building hk on 20130919 for linux-arm: already built: 20130919
build.go:128: Error building hk on 20130919 for windows-386: already built: 20130919
build.go:128: Error building hk on 20130919 for windows-amd64: already built: 20130919

And for an arbitrary release version:

$ hkdist build -v v20130918 all
build.go:70: Ignoring BUILDBRANCH and building on version v20130918 instead
build.go:244: git clone -b v20130918 https://github.com/kr/hk.git hk-build
remote: Counting objects: 1013, done.
remote: Compressing objects: 100% (375/375), done.
remote: Total 1013 (delta 640), reused 994 (delta 624)
Receiving objects: 100% (1013/1013), 158.48 KiB | 226 KiB/s, done.
Resolving deltas: 100% (640/640), done.
Note: checking out '1aec653a9c184ecbf8a5e91d2afc292622401419'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b new_branch_name

build.go:201: building release=20130918 os=darwin arch=386
build.go:210: GOOS=darwin GOARCH=386 go build -tags release -o hk
build.go:128: Error building hk on 20130918 for darwin-amd64: already built: 20130918
build.go:201: building release=20130918 os=freebsd arch=386
build.go:210: GOOS=freebsd GOARCH=386 go build -tags release -o hk
build.go:201: building release=20130918 os=freebsd arch=amd64
build.go:210: GOOS=freebsd GOARCH=amd64 go build -tags release -o hk
build.go:201: building release=20130918 os=freebsd arch=arm
build.go:210: GOOS=freebsd GOARCH=arm go build -tags release -o hk
build.go:201: building release=20130918 os=linux arch=386
build.go:210: GOOS=linux GOARCH=386 go build -tags release -o hk
build.go:201: building release=20130918 os=linux arch=amd64
build.go:210: GOOS=linux GOARCH=amd64 go build -tags release -o hk
build.go:201: building release=20130918 os=linux arch=arm
build.go:210: GOOS=linux GOARCH=arm go build -tags release -o hk
build.go:201: building release=20130918 os=windows arch=386
build.go:210: GOOS=windows GOARCH=386 go build -tags release -o hk.exe
build.go:128: Error building hk on 20130918 for windows-amd64: already built: 20130918

[bgentry]

Still not sure whether I should be running all builds with CGO_ENABLED=0. Currently, you get cgo & possibly some dynamic links on whatever OS you're compiling from, but not for the others.

[kr]

Still not sure whether I should be running all builds with CGO_ENABLED=0. Currently, you get cgo & possibly some dynamic links on whatever OS you're compiling from, but not for the others.

Yeah, I think we should do all builds the same way, so it doesn't matter what platform hkdist build runs on. If there's a reason to use cgo, it sort of means our whole strategy of cross compiling is screwed. By the contrapositive, if we're cross compiling at all, we might as well do it on all builds, consistently.

[kr]

This looks pretty good to me. I commented on a bunch of little and medium-size things but I like it overall. Thanks! :)

[bgentry]

@kr I think that should address all of your feedback. Thanks a bunch, all very helpful. Lmk if you see anything else to fix before I should squash + merge this in.

[bgentry]

@kr I've addressed all the feedback thus far. Since we've already had a couple solid rounds of review, I'm gonna go ahead and ship this. Hopefully we can pull all this hkdist stuff out soon enough anyway, possibly leveraging the work @ddollar has been doing :)

[kr]

Awesome. Thanks for staying on top of this.