tailscale / tailscale

The easiest, most secure way to use WireGuard and 2FA.
https://tailscale.com
BSD 3-Clause "New" or "Revised" License
19.39k stars 1.52k forks source link

version: come up with good default implementation #81

Closed bradfitz closed 2 years ago

bradfitz commented 4 years ago

When we do tailscale binary releases, we stamp version info into the binaries with the Go linker.

But for people just using "go get" the default way, we don't have that info.

Since Go modules, we at least see the summary of all our deps (and Go's version itself; I'm running a devel build) automatically stamped into the binaries:

$ go version -m ~/bin/tailscaled
/home/bradfitz/bin/tailscaled: devel +e7f9e17b79 Tue Jan 28 22:08:43 2020 +0000
        path    tailscale.com/cmd/tailscaled
        mod     tailscale.com   (devel)
        dep     github.com/apenwarr/fixconsole  v0.0.0-20191012055117-5a9f6489cc29      h1:muXWUcay7DDy1/hEQWrYlBy+g0EuwT70sBHg65SeUc4=            
        dep     github.com/golang/groupcache    v0.0.0-20200121045136-8c9f03a8e57e      h1:1r7pUrabqp18hOBcwBwiTsbnFeTZHV9eER/QT5JVZxY=    
        dep     github.com/google/go-cmp        v0.4.0  h1:xsAVV57WRhGj6kEIi8ReJzQlHHqcBYCElAvkovg3B/4=
        dep     github.com/klauspost/compress   v1.9.8  h1:VMAMUUOh+gaxKTMk+zqbjsSjsIcUcL/LF4o63i82QyA=
        dep     github.com/mdlayher/netlink     v1.1.0  h1:mpdLgm+brq10nI9zM1BpX1kpDbh3NLl3RSnVq6ZSkfg=
        dep     github.com/pborman/getopt       v0.0.0-20190409184431-ee0cd42419d3      h1:YtFkrqsMEj7YqpIhRteVxJxCeC3jJBieuLr0d4C4rSA=
        dep     github.com/tailscale/wireguard-go       v0.0.0-20200213180345-a7c4b7719b1d      h1:LVJovgZxbmPxtY6kJm4vwMtk0HpcNeI+vU2jB3T8M40=
        dep     golang.org/x/crypto     v0.0.0-20200210222208-86ce3cb69678      h1:wCWoJcFExDgyYx2m2hpHgwz8W3+FPdfldvIgzqDIhyg=
        dep     golang.org/x/net        v0.0.0-20200202094626-16171245cfb2      h1:CCH4IOTTfewWjGOlSp+zGcjutRKlBEZQ6wTn8ozI/nI=
        dep     golang.org/x/oauth2     v0.0.0-20200107190931-bf48bf16ab8d      h1:TzXSXBo42m9gQenoE3b9BGiEpg5IG2JkU5FkPIawgtw=
        dep     golang.org/x/sys        v0.0.0-20200217220822-9197077df867      h1:JoRuNIf+rpHl+VhScRQQvzbHed86tKkqwPMV34T8myw=
        dep     rsc.io/goversion        v1.2.0  h1:SPn+NLTiAG7w30IRK/DKp1BjvpWabYgxlLp/+kx5J8w=

We can get that at runtime (using https://godoc.org/rsc.io/goversion/version) but for better or worse, that doesn't include that the Git hash of the top-level module, only the deps.

So one thing we could do is add a fake dep to another module (like github.com/tailscale/version-horizon) that we have a bot auto-advance every $INTERVAL, so at least we have a rough date range of what binary people are running when they're running unofficial binaries. Or, if we're already having a bot do this, avoid that indirection and just have the bot auto-commit the latest date to version/version.go every $INTERVAL instead.

Low priority, but we'll probably want better data in the future.

/cc @apenwarr @crawshaw @danderson

bradfitz commented 4 years ago

If others want to see what's available, you can try:

diff --git a/version/version.go b/version/version.go
index 1bc61d6..3a2562e 100644
--- a/version/version.go
+++ b/version/version.go
@@ -7,5 +7,24 @@
 // Package version provides the version that the binary was built at.
 package version

-const LONG = "LONGVER-TODO"
-const SHORT = "SHORTVER-TODO"
+import (
+       "fmt"
+       "os"
+
+       "rsc.io/goversion/version"
+)
+
+var LONG = "VERSION-UNKNOWN"
+var SHORT = "VERSION-UNKNOWN"
+
+func init() {
+       n, err := os.Executable()
+       if err != nil {
+               return
+       }
+       v, err := version.ReadExe(n)
+       if err != nil {
+               println(err.Error())
+       }
+       println(fmt.Sprintf("Version: %+v\n", v))
+}

Then go run ./cmd/tailscaled -h.

danderson commented 4 years ago

I'm getting an increasing number of inquiries about this in the context of OSS packaging in distros. It's a problem that we can't ship sensible version info if you're building from just the OSS repo.

I'm going to take a swing at fixing this ahead of the Go compiler providing us more useful information, at least well enough that distro packagers can embed version info as they build, while we can continue to provide corp version info for the non-free programs.

bradfitz commented 4 years ago

What's your plan?

danderson commented 4 years ago

Rough plan, not tested out yet:

Works in the following use cases:

That's the general shower thoughts I had this morning. The context is that I reviewed the initial Nix/NixOS derivations for Tailscale last night, which are going to ship to NixOS users with the crappy fallbackversion info. Separately this morning, an Arch user pointed out that some of my AUR packages violate AUR convention by shipping binaries, and I need to switch to building from source - which would force me to implement some "version for OSS" hack anyway. Given those, I want to solve that problem ONCE AND FOR ALL upstream, so I can give downstream distro packagers uniform instructions for how to build binaries with good version data.

bradfitz commented 4 years ago

So, another bespoke Makefile / make.bash / make.go for oss. (https://github.com/golang/go/issues/37475)

That's fine for now, for people who want to use it.

I have a crazier idea for the go get users: look up the version from a network service at runtime based on the program taking a signature of its own binary (using debug/elf, etc). The network service would serve effectively a map[Signature]struct{GitCommit, CommitTime} that would be populated by a daemon repeatedly building each commit with a dozen latest/most populate versions of the compiler. I might even just build that as a non-Tailscale service on Cloud Run that anybody could use for free.

danderson commented 4 years ago

Pretty much, yeah. It's definitely not what I want to do, but the third-party distros need something to embed version information we can reason about for debugging.

A versioning service sounds interesting, though I question whether you can actually account for the variety of weirdery out there (e.g. all Nix binaries have altered library load paths and whatnots, so the compilers are likely to produce slightly different output). I think it's possible, but it'd be fiddly.

Foxboron commented 4 years ago

Yo,

This issue has been silent since March and assigned a low priority. But with the release of version 1.0 I think you need to reconsider.

Today if I want to download tailscale there are a few binaries I can select. But no signatures. That isn't a problem in theory, go binaries are reproducible. However there is no clear way to figure out how they are built. Looking at the built version

λ tailscale_1.0.5_amd64 » ./tailscale --version          
1.0.5-g31b5dec0a

I can checkout the 1.0.5 tag, and try build it with -trimpath. But the commit g31b5dec0a is not present in this repository. Even injecting the correct version into the binary gives me a different checksum of the binaries.

What are the major roadblocks for embedding version information, and streamlining the build process? It would help reproducible builds for your project and distro packagers (semi-relevant as I'm interested distributing tailscale in the [community] repo of Arch).

bradfitz commented 4 years ago

This got blocked on https://github.com/golang/go/issues/37475 which is accepted but not yet implemented in upstream Go.

Our -g31b5dec0a suffix means "git commit" 31b5dec0a, but that's a git commit of our parent repo that has all our other repos (e.g. https://github.com/tailscale/tailscale-android, the closed-source iOS app) as git submodules.

I would suggest solutions but I'm not sure your goal. You mentioned several somewhat separate concerns.

Our builds don't contain anything closed-source mixed in from the parent repo other than the version string, so if you set that, use -trimpath, and build with the same version of Go that we do (https://github.com/tailscale/go, the commit of which we print out at start-up), then you should get the same output bytes.

For Arch builds, I'd just make the version be "1.0.5-g<git commit of tailscale/tailscale repo>"

Foxboron commented 4 years ago

I would suggest solutions but I'm not sure your goal. You mentioned several somewhat separate concerns.

My goal is to have tailscale reproducible. This is important both when distributing binaries with no signatures, and for packagers to have a streamlined build process. My impression was that this versioning problem was the reason why I can't have a make build in the project. But it seems like I was mistaken.

Our builds don't contain anything closed-source mixed in from the parent repo other than the version string, so if you set that, use -trimpath, and build with the same version of Go that we do (https://github.com/tailscale/go, the commit of which we print out at start-up), then you should get the same output bytes.

So I tried. I fetched https://github.com/tailscale/go/releases/tag/build-56db76510f9640d2ad652f206ae6e41c1a5d63ca which is what 1.0.5-g31b5dec0a should be built with (output of running tailscale). Ran ./src/make.bash in the go source. Tried building tailscale by inserting the correct version and running the following compilation options which is purely guessed as it's not dynamically compiled.

CGO_ENABLED=0 GOOS=linux GOARCH=amd64 ~/Downloads/go/bin/go build -trimpath ./cmd/tailscale

λ ~ » sha256sum Downloads/tailscale_1.0.5_amd64/tailscale 
05ce75560631d6723a73fdac7a6711f8d5e142dad8bae7c2b14f8c13c61077b6  Downloads/tailscale_1.0.5_amd64/tailscale
λ ~ » sha256sum Git/prosjekter/Go/tailscale/tailscale 
4791c43c5f1f24f3177731cd93bbd0f56d764b03a27fea919a0465acfa9a6326  Git/prosjekter/Go/tailscale/tailscale
λ ~ » ./Git/prosjekter/Go/tailscale/tailscale --version
1.0.5-g31b5dec0a
λ ~ » ./Downloads/tailscale_1.0.5_amd64/tailscale --version
1.0.5-g31b5dec0a

I don't know how the bootstrapping process affects the results, and I'm unsure if it's feasibly without you redistributing the compiler itself (what guarantees does the go compiler have?). Please do point out if I have made any mistakes with this, I'm just guessing my way around at this point.

I think this is a problem, but I'm unsure if it's related to this issue specifically. Currently there doesn't seem to be a good way to prove how the distributed tailscale is being built.

apenwarr commented 4 years ago

I think we supply some specific build tags too nowadays, right?

bradfitz commented 4 years ago

Oh, we also set the tailscale_go build tag in our builds.

Relevant bits from our build system:

redo  out/x86_64-linux/oss/cmd/tailscale/tailscale (resumed)
#!/bin/sh -e
# Per-platform settings for the go compiler.
# Auto-generated by go.od. Do not edit!
S='/home/bradfitz/src/tailscale.io'
export GO='/home/bradfitz/.cache/tailscale-go/bin/go'
export CC='cc'
export CFLAGS=''
export GO_LDFLAGS='-extldflags ""'
export GOOS=''
export GOARCH=''
export GOARM=''
export CGO_ENABLED='0'

# Necessary because CI forces GOROOT and we need
# to override it back.
export GOROOT=
redo    (../../../../../oss/version/all)                                                                                                                                            
redo    (../../../../../oss/version/version.h)
redo    oss/version/describe.txt
redo    oss/version/describe.txt (done)                                                                                                                                             

redo  out/x86_64-linux/oss/cmd/tailscale/tailscale (done)                                                                                                                           

....

        '$GO' build \\
            -v -trimpath -tags=redo,tailscale_go \\
            -ldflags "\$GO_LDFLAGS" \\
            -o "\$t" \\
            "\$pkg"
bradfitz commented 4 years ago

(This is all pretty off topic for this issue, though.)

Foxboron commented 4 years ago

(Thanks,!I'm aware it's offtopic. I'm happy to take up this discussion wherever appropriate.)

bradfitz commented 4 years ago

If you like Slack, see https://github.com/tailscale/tailscale/wiki/Community for a link. Otherwise if you prefer GitHub, just create a new issue here? Perhaps titled "Document how to generate the same binaries Tailscale distributes"

DentonGentry commented 3 years ago

I think the main intent of this issue has been addressed by https://github.com/tailscale/tailscale/blob/main/build_dist.sh Specific questions about reproducibility split out into #779.

DentonGentry commented 3 years ago

https://go-review.googlesource.com/c/go/+/353930 will embed the git hash into generated binaries. I think that will be sufficient to cover the remaining case of concern in this issue, of people building from source.

bradfitz commented 2 years ago

Waiting on Go 1.18 in couple weeks.