Support tip? - Githubissues

markphelps commented 4 years ago

I looked through the installer.js script and it looked to me like only tags from https://github.com/golang/go/tags are supported as valid versions to install.

I would imagine that some users would also want to test their Go programs against tip to ensure that upcoming language changes/additions don't break.

Go even provides a way to install tip via https://godoc.org/golang.org/dl/gotip (once you have a single Go version installed that is).

What do you think?

mvdan commented 4 years ago

tip has a slight problem that the rest of the tags don't - it would incur building Go, which usually takes about a minute on regular hardware. This should be fine for most people, but we shouldn't recommend that people test on tip by default.

Supporting it still makes sense, though. Given that some Go versions are already installed by default, the support could simply be:

git clone --depth=1 https://go.googlesource.com/go /somewhere/gotip
cd /somewhere/gotip/src
GOROOT_BOOTSTRAP=/path/to/stable/goroot ./make.bash
export PATH="/somewhere/gotip/bin:$PATH"

tsatke commented 4 years ago

I would like being able to use it in go_version: [1.11, 1.12, 1.13, tip]

zikaeroh commented 4 years ago

For now, I've done the following:

- name: Install Go
  if: matrix.go != 'tip'
  uses: actions/setup-go@v2
  with:
    go-version: ${{ matrix.go }}

- name: Install gotip
  if: matrix.go == 'tip'
  run: |
    git clone --depth=1 https://go.googlesource.com/go $HOME/gotip
    cd $HOME/gotip/src
    ./make.bash
    echo "GOROOT=$HOME/gotip" >> $GITHUB_ENV
    echo "$HOME/gotip/bin:$PATH" >> $GITHUB_PATH

~~Because I'm lazy I used gotip rather than cloning directly~~ (EDIT: Updated to just clone and build as @mvdan suggested; it's faster), but in addition to modifying $PATH, $GOROOT has to get set for some reason to get the correct $GOTOOLDIR (even though it should be inferred; probably some existing environment variable). No need to run setup-go to bootstrap this stage, since the builders come with a Go version already installed. Takes about 3 minutes to build.

I haven't tested this with anything but a Linux host. It probably needs a shell option to say it's a bash script. Probably could be simplified as well.

(EDIT: updated 2021-10-17 for new versions and GHA changes :smile:)

zikaeroh commented 4 years ago

I've updated my comment above to just build from a clone. It's better than effectively cloning and building two repos to save a line. :slightly_smiling_face:

tie commented 4 years ago

@zikaeroh, I’d suggest downloading zip archive from ~~GitHub~~ Gitiles.

- name: Install Go
  if: matrix.go != 'tip'
  uses: actions/setup-go@master
  with:
    go-version: ${{ matrix.go }}

- name: Install Go
  if: matrix.go == 'tip'
  run: |
    export GOROOT_BOOTSTRAP=`go env GOROOT`
    export GOROOT_FINAL=/go
    export GOROOT=$HOME/gotip
    mkdir $HOME/gotip
    cd $HOME/gotip

    curl -s 'https://go.googlesource.com/go/+/refs/heads/master?format=JSON' | awk '/"commit"/{print substr($2,2,40);exit}' >HEAD
    awk '{printf("gotip-%s",substr($0,0,7))}' <HEAD >VERSION

    curl -s -o go.tar.gz https://go.googlesource.com/go/+archive/`cat HEAD`.tar.gz
    tar xfz go.tar.gz

    cd src
    bash make.bash

    echo "::set-env name=GOROOT::$GOROOT"
    echo "::add-path::$GOROOT/bin"

zikaeroh commented 4 years ago

Go is actually hosted on googlesource/gerrit, where changes are made; the Github repo is a mirror that is automatically updated by gopherbot (really, something in x/build). I'd rather get a clone or archive directly from the original source than the mirror. I've seen cases where things have broken and the Github mirror is slightly behind.

tie commented 4 years ago

@zikaeroh OK, updated the comment to use .tar.gz archive from googlesource.

See https://gerrit-review.googlesource.com/c/gitiles/+/47141/

Edit: Download speed from Google’s server seems to be faster than from GitHub.

zikaeroh commented 4 years ago

Using an archive doesn't work, actually. Since Go builds encode a version, they expect to be in a git clone. Downloading the linked tarball:

$ ./make.bash 
Building Go cmd/dist using /usr/lib/go. (go1.13.5 linux/amd64)
go tool dist: FAILED: not a Git repo; must put a VERSION file in $GOROOT

I think a --depth=1 clone is fine, but grabbing an archive was a good idea.

tie commented 4 years ago

@zikaeroh oops, forgot about this file. Updated to write VERSION file. Now with a bit of awk wizardry.

~~The only downside is that it’s always one commit behind because /+/refs/heads/master?format=TEXT does not return hash for the latest commit. Looking for a way to improve this.~~

Edit: updated to use format=JSON to fetch the latest commit.

zikaeroh commented 4 years ago

One big asterisk on running tip is the fact that GitHub actions do not support allowed failures. There is no current way to allow a job to fail entirely and still have the rest of the workflow pass.

This really defeats the idea of having jobs that test against development releases of languages, as you can't really have them in your normal workflow without complicating things. This is a big bummer for me; I like to have tip running in my pipelines so I can report issues upstream. So far, I can't do this without a lot of extra work (and I may just stick with Travis until there's feature/performance parity).

tie commented 4 years ago

@zikaeroh isn’t it the jobs.<job_id>.steps.continue-on-error option?

Prevents a job from failing when a step fails. Set to true to allow a job to pass when this step fails.

zikaeroh commented 4 years ago

Not equivalent; that just makes the step skipped and the job continue moving on as though nothing had broken. If compilation or something were to fail and I had that option enabled, then:

The next steps would probably fall apart, because the previous step broke.
If things don't break, the GitHub UI would report "all good!", and not provide any indication that things had broken.

Travis does this right, and displays that a job is allowed to fail (even moving them to their own section). GitHub even displays Travis's info.

For example, before I fixed a bug in another library that was found in tip's new checkptr option, a build may have looked like:

tie commented 4 years ago

@zikaeroh then jobs.<job_id>.strategy.fail-fast is what you are looking for.

When set to true, GitHub cancels all in-progress jobs if any matrix job fails. Default: true

Cheers.

zikaeroh commented 4 years ago

That is also not the same thing. Fail-fast set to false prevents GitHub from not immediately cancelling every currently running job when one of them fails. It doesn't mean that dependent jobs (or the entire workflow) don't fail. I already use this so I don't lose the output of every other build I run concurrently when one of them fails, as that's what happens. I'm looking for allow_failures.

tie commented 4 years ago

Ah, I get what you want now. That doesn’t seem hard to implement, have you tried contacting GitHub support?

zikaeroh commented 4 years ago

There is feedback on the GitHub Actions community page, but submitting/upvoting things there feels comparable to firing feedback into deep space. https://github.community/t5/GitHub-Actions/continue-on-error-allow-failure-UI-indication/m-p/37033

Either way, it's not relevant for tip support in setup-go other than as a warning for people trying it out after working with Travis. I don't want to clog up this issue with unrelated feedback.

maxatome commented 3 years ago

Have a look at https://github.com/maxatome/install-go to support any go version including tip.

AlekSi commented 2 years ago

One can avoid re-building Go tip (and other branches like in-development fuzzing support) every time by using pre-built versions from https://github.com/AlekSi/golang-tip.

tie commented 2 years ago

@AlekSi, I think it’d be better to pre-build tip in https://github.com/actions/go-versions.

AlekSi commented 2 years ago

I dunno; that repository does not have issues enabled, so we discuss it there :)

Sergey-Murtazin commented 2 years ago

Hi @markphelps ! Sorry for the late response! Could you please clarify if the issue is still actual for you? Thanks!

zikaeroh commented 2 years ago

I am not the original poster here, but after about two years, I'm still wanting this. I recently moved all of my go setup to a composite action in my repo, but without conditionals, I can't use the workaround I put in my comment as I have to explicitly skip setup-go in favor of it when go-version is tip: https://github.com/actions/setup-go/issues/21#issuecomment-565704236

Of course, I'm also still waiting for an allow-failures equivalent to make tip testing to be viable, too...

maxatome commented 2 years ago

@zikaeroh see my workaround: https://github.com/maxatome/go-testdeep/blob/master/.github/workflows/ci.yml tip build success is optional and you are sure the last patch version of each go version is used (it is not the case with this action).

zikaeroh commented 2 years ago

That's a helpful script, but it sidesteps the niceness of GHA toolchain caching (setup-go takes ~0s to run), and I'm a bit wary of curl-ing a script from the internet directly into an interpreter in my CI (compared to an officially supported action), so I still want setup-go to support tip.

Regarding allow-failures, I'd go follow https://github.com/actions/runner/issues/2347; continue-on-failure leaves much to be desired as compared to Travis's allow-failures system (e.g., if tip fails my tests, the build fails but does not impede other steps; this is different than allowing specific steps in a job to succeed, and then allow the whole job to succeed without any feedback that actually, a step failed).

maxatome commented 2 years ago

@zikaeroh you are completely right, one can never be too careful (even if you can compute a hash of the script you first checked, then for each download in your CI, check it matches the same hash before going forward). I developed this script because setup-go is not always up-to-date regarding the latest go releases and this issue is opened for 2 years now. Note that when the image already contains the required go version, the script doesn't install anything but uses the installed version, so ~0s to run also.

I agree with you for allow-failures vs continue-on-failure. Here again, it is a last resort solution until something moves in this area too.

mmcloughlin commented 2 years ago

I haven't fully thought this through but I wonder if you could skip a build by downloading a snapshot from the Go build system. For example, this is a build for the most recent commit:

https://storage.googleapis.com/go-build-snap/go/linux-amd64/5d6d9f5610066584374c2dfe7624fa9f251089a0.tar.gz

Alternatively, you can also use the go install golang.org/dl/gotip@latest tool. Under the hood, it's still doing the clone and build, but arguably that's cleaner than doing it yourself.

maxatome commented 2 years ago

@mmcloughlin getting already built tip is very interesting :)

but it seems only linux builds are available this way, right? Do you know where are these URLs documented (to be sure it is stable over time and to know if other OS/arch are supported)?

mmcloughlin commented 2 years ago

@maxatome I suspect this isn't officially supported but in practice it's been stable for a long time. You'd need to combine this with a "soft fail" mechanism, but you should be doing that for tip builds anyway.

There are some libraries to help with this. The buildenv package has a SnapshotURL that gives you the URL to download:

https://pkg.go.dev/golang.org/x/build/buildenv#Environment.SnapshotURL

See an example of using it here:

https://github.com/mmcloughlin/goperf/blob/c9cfaf6957aa068c925aff1b7d7f818e67f3c85f/pkg/runner/toolchain.go#L165

Note from that interface that the linux-amd64 in the URL is a builder type. You can get a list of builders from the dashboard package. So you can lookup a builder for a GOOS/GOARCH combination like this:

https://github.com/mmcloughlin/goperf/blob/c9cfaf6957aa068c925aff1b7d7f818e67f3c85f/pkg/runner/toolchain.go#L182-L196

I used this as part of a Go benchmarking system that I never finished, but I suspect you could use these functions to write a faster version of the gotip tool that installs from a snapshot.

maxatome commented 2 years ago

I did some tests with SnapshotURL() and dashboard.Builders. linux-amd64 exists, but for darwin, freebsd or windows a version is always suffixed: darwin-amd64-10_14, freebsd-amd64-12_2, windows-arm64-10 for example. I'll try to see how this version can be guessed at best from the OS running the script. Be that as it may, thanks for the information @mmcloughlin

List of builder types

``` aix-ppc64 android-386-emu android-amd64-emu android-arm-corellium android-arm64-corellium darwin-amd64-10_12 darwin-amd64-10_14 darwin-amd64-10_15 darwin-amd64-11_0 darwin-amd64-nocgo darwin-amd64-race darwin-arm64-11_0-toothrot dragonfly-amd64 freebsd-386-11_2 freebsd-386-11_4 freebsd-386-12_2 freebsd-amd64-11_2 freebsd-amd64-11_4 freebsd-amd64-12_2 freebsd-amd64-race freebsd-arm-paulzhol freebsd-arm64-dmgk illumos-amd64 ios-arm64-corellium js-wasm linux-386 linux-386-buster linux-386-clang linux-386-jessie linux-386-longtest linux-386-sid linux-386-softfloat linux-386-stretch linux-amd64 linux-amd64-androidemu linux-amd64-bullseye linux-amd64-buster linux-amd64-clang linux-amd64-fedora linux-amd64-jessie linux-amd64-localdev linux-amd64-longtest linux-amd64-nocgo linux-amd64-noopt linux-amd64-perf linux-amd64-race linux-amd64-racecompile linux-amd64-sid linux-amd64-ssacheck linux-amd64-staticlockranking linux-amd64-stretch linux-amd64-unified linux-amd64-vmx linux-amd64-wsl linux-arm-aws linux-arm64-aws linux-arm64-packet linux-loong64-3a5000 linux-mips-rtrk linux-mips64-rtrk linux-mips64le-mengzhuo linux-mips64le-rtrk linux-mipsle-rtrk linux-ppc64-buildlet linux-ppc64le-buildlet linux-ppc64le-power9osu linux-riscv64-jsing linux-riscv64-unleashed linux-riscv64-unmatched linux-s390x-crosscompile linux-s390x-ibm misc-compile-darwinarm64 misc-compile-freebsd misc-compile-mac-win misc-compile-mips misc-compile-mipsle misc-compile-netbsd misc-compile-netbsd-arm misc-compile-openbsd misc-compile-openbsd-arm misc-compile-other-1 misc-compile-other-2 misc-compile-plan9 misc-compile-ppc netbsd-386-9_0 netbsd-amd64-9_0 netbsd-arm-bsiegert netbsd-arm64-bsiegert openbsd-386-68 openbsd-amd64-68 openbsd-arm-jsing openbsd-arm64-jsing openbsd-mips64-jsing plan9-386 plan9-386-0intro plan9-amd64-0intro plan9-arm solaris-amd64-oraclerel windows-386-2008 windows-amd64-2008 windows-amd64-2012 windows-amd64-2016 windows-amd64-longtest windows-amd64-race windows-arm-zx2c4 windows-arm64-10 ```

mmcloughlin commented 2 years ago

I was assuming that you could pick any of the builder types for a given GOOS/GOARCH pair. The code I sent you picks the shortest builder name. Here's the list I get, with the GOOS-GOARCH printed too.

package main

import (
    "fmt"

    "golang.org/x/build/dashboard"
)

func main() {
    for name, conf := range dashboard.Builders {
        fmt.Printf("%s-%s\t%s\n", conf.GOOS(), conf.GOARCH(), name)
    }
}

List of builder types

``` aix-ppc64 aix-ppc64 android-386 android-386-emu android-amd64 android-amd64-emu android-arm64 android-arm64-corellium android-arm android-arm-corellium darwin-amd64 darwin-amd64-10_12 darwin-amd64 darwin-amd64-10_14 darwin-amd64 darwin-amd64-10_15 darwin-amd64 darwin-amd64-11_0 darwin-amd64 darwin-amd64-nocgo darwin-amd64 darwin-amd64-race darwin-arm64 darwin-arm64-11_0-toothrot dragonfly-amd64 dragonfly-amd64 freebsd-386 freebsd-386-11_2 freebsd-386 freebsd-386-11_4 freebsd-386 freebsd-386-12_2 freebsd-amd64 freebsd-amd64-11_2 freebsd-amd64 freebsd-amd64-11_4 freebsd-amd64 freebsd-amd64-12_2 freebsd-amd64 freebsd-amd64-race freebsd-arm64 freebsd-arm64-dmgk freebsd-arm freebsd-arm-paulzhol illumos-amd64 illumos-amd64 ios-arm64 ios-arm64-corellium js-wasm js-wasm linux-386 linux-386 linux-386 linux-386-buster linux-386 linux-386-clang linux-386 linux-386-jessie linux-386 linux-386-longtest linux-386 linux-386-sid linux-386 linux-386-softfloat linux-386 linux-386-stretch linux-amd64 linux-amd64 linux-amd64 linux-amd64-androidemu linux-amd64 linux-amd64-bullseye linux-amd64 linux-amd64-buster linux-amd64 linux-amd64-clang linux-amd64 linux-amd64-fedora linux-amd64 linux-amd64-jessie linux-amd64 linux-amd64-localdev linux-amd64 linux-amd64-longtest linux-amd64 linux-amd64-nocgo linux-amd64 linux-amd64-noopt linux-amd64 linux-amd64-perf linux-amd64 linux-amd64-race linux-amd64 linux-amd64-racecompile linux-amd64 linux-amd64-sid linux-amd64 linux-amd64-ssacheck linux-amd64 linux-amd64-staticlockranking linux-amd64 linux-amd64-stretch linux-amd64 linux-amd64-unified linux-amd64 linux-amd64-vmx linux-amd64 linux-amd64-wsl linux-arm64 linux-arm64-aws linux-arm64 linux-arm64-packet linux-arm linux-arm-aws linux-loong64 linux-loong64-3a5000 linux-mips64le linux-mips64le-mengzhuo linux-mips64le linux-mips64le-rtrk linux-mips64 linux-mips64-rtrk linux-mipsle linux-mipsle-rtrk linux-mips linux-mips-rtrk linux-ppc64le linux-ppc64le-buildlet linux-ppc64le linux-ppc64le-power9osu linux-ppc64 linux-ppc64-buildlet linux-riscv64 linux-riscv64-jsing linux-riscv64 linux-riscv64-unleashed linux-riscv64 linux-riscv64-unmatched linux-s390x linux-s390x-crosscompile linux-s390x linux-s390x-ibm misc-compile misc-compile-darwinarm64 misc-compile misc-compile-freebsd misc-compile misc-compile-mac-win misc-compile misc-compile-mips misc-compile misc-compile-mipsle misc-compile misc-compile-netbsd misc-compile misc-compile-netbsd-arm misc-compile misc-compile-openbsd misc-compile misc-compile-openbsd-arm misc-compile misc-compile-other-1 misc-compile misc-compile-other-2 misc-compile misc-compile-plan9 misc-compile misc-compile-ppc netbsd-386 netbsd-386-9_0 netbsd-amd64 netbsd-amd64-9_0 netbsd-arm64 netbsd-arm64-bsiegert netbsd-arm netbsd-arm-bsiegert openbsd-386 openbsd-386-68 openbsd-amd64 openbsd-amd64-68 openbsd-arm64 openbsd-arm64-jsing openbsd-arm openbsd-arm-jsing openbsd-mips64 openbsd-mips64-jsing plan9-386 plan9-386 plan9-386 plan9-386-0intro plan9-amd64 plan9-amd64-0intro plan9-arm plan9-arm solaris-amd64 solaris-amd64-oraclerel windows-386 windows-386-2008 windows-amd64 windows-amd64-2008 windows-amd64 windows-amd64-2012 windows-amd64 windows-amd64-2016 windows-amd64 windows-amd64-longtest windows-amd64 windows-amd64-race windows-arm64 windows-arm64-10 windows-arm windows-arm-zx2c4 ```

mmcloughlin commented 2 years ago

Still, seems like a decent number of them don't always have snapshots available. Updated the script a bit and used curl -I to do HEAD requests.

package main

import (
    "fmt"

    "golang.org/x/build/buildenv"
    "golang.org/x/build/dashboard"
)

const rev = "5d6d9f5610066584374c2dfe7624fa9f251089a0"

func main() {
    for name, conf := range dashboard.Builders {
        if conf.IsRace() || conf.SkipSnapshot || conf.KnownIssue != 0 {
            continue
        }
        url := buildenv.Production.SnapshotURL(name, rev)
        fmt.Printf("%s-%s\t%s\t%s\n", conf.GOOS(), conf.GOARCH(), name, url)
    }
}

$ go run . | sort | while read osarch name url; do echo -n "$name    "; curl -Is $url | grep HTTP; done
aix-ppc64    HTTP/2 200 
android-386-emu    HTTP/2 200 
android-amd64-emu    HTTP/2 200 
darwin-amd64-10_12    HTTP/2 403 
darwin-amd64-10_14    HTTP/2 200 
darwin-amd64-10_15    HTTP/2 200 
darwin-amd64-11_0    HTTP/2 200 
darwin-amd64-nocgo    HTTP/2 200 
darwin-arm64-11_0-toothrot    HTTP/2 200 
freebsd-386-11_2    HTTP/2 403 
freebsd-386-11_4    HTTP/2 200 
freebsd-386-12_2    HTTP/2 200 
freebsd-amd64-11_2    HTTP/2 403 
freebsd-amd64-11_4    HTTP/2 200 
freebsd-amd64-12_2    HTTP/2 200 
freebsd-arm64-dmgk    HTTP/2 200 
illumos-amd64    HTTP/2 200 
js-wasm    HTTP/2 200 
linux-386-buster    HTTP/2 200 
linux-386-clang    HTTP/2 200 
linux-386    HTTP/2 200 
linux-386-jessie    HTTP/2 200 
linux-386-longtest    HTTP/2 200 
linux-386-sid    HTTP/2 200 
linux-386-softfloat    HTTP/2 200 
linux-386-stretch    HTTP/2 200 
linux-amd64-androidemu    HTTP/2 200 
linux-amd64-bullseye    HTTP/2 200 
linux-amd64-buster    HTTP/2 200 
linux-amd64-clang    HTTP/2 200 
linux-amd64-fedora    HTTP/2 200 
linux-amd64    HTTP/2 200 
linux-amd64-jessie    HTTP/2 200 
linux-amd64-localdev    HTTP/2 403 
linux-amd64-longtest    HTTP/2 200 
linux-amd64-nocgo    HTTP/2 200 
linux-amd64-noopt    HTTP/2 200 
linux-amd64-sid    HTTP/2 200 
linux-amd64-ssacheck    HTTP/2 200 
linux-amd64-staticlockranking    HTTP/2 200 
linux-amd64-stretch    HTTP/2 403 
linux-amd64-unified    HTTP/2 200 
linux-amd64-vmx    HTTP/2 403 
linux-arm64-aws    HTTP/2 200 
linux-arm64-packet    HTTP/2 200 
linux-arm-aws    HTTP/2 200 
linux-ppc64le-buildlet    HTTP/2 200 
linux-ppc64le-power9osu    HTTP/2 200 
linux-ppc64-buildlet    HTTP/2 200 
linux-riscv64-unmatched    HTTP/2 200 
linux-s390x-crosscompile    HTTP/2 403 
linux-s390x-ibm    HTTP/2 200 
misc-compile-darwinarm64    HTTP/2 403 
misc-compile-freebsd    HTTP/2 403 
misc-compile-mac-win    HTTP/2 403 
misc-compile-mips    HTTP/2 403 
misc-compile-mipsle    HTTP/2 403 
misc-compile-netbsd-arm    HTTP/2 403 
misc-compile-netbsd    HTTP/2 403 
misc-compile-openbsd-arm    HTTP/2 403 
misc-compile-openbsd    HTTP/2 403 
misc-compile-other-1    HTTP/2 403 
misc-compile-other-2    HTTP/2 403 
misc-compile-plan9    HTTP/2 403 
misc-compile-ppc    HTTP/2 403 
netbsd-386-9_0    HTTP/2 200 
netbsd-amd64-9_0    HTTP/2 200 
netbsd-arm64-bsiegert    HTTP/2 200 
netbsd-arm-bsiegert    HTTP/2 200 
openbsd-386-68    HTTP/2 200 
openbsd-amd64-68    HTTP/2 200 
plan9-386-0intro    HTTP/2 403 
plan9-386    HTTP/2 403 
plan9-amd64-0intro    HTTP/2 403 
plan9-arm    HTTP/2 200 
solaris-amd64-oraclerel    HTTP/2 200 
windows-386-2008    HTTP/2 200 
windows-amd64-2008    HTTP/2 200 
windows-amd64-2012    HTTP/2 200 
windows-amd64-2016    HTTP/2 200 
windows-amd64-longtest    HTTP/2 200 
windows-arm64-10    HTTP/2 200 
windows-arm-zx2c4    HTTP/2 403

maxatome commented 2 years ago

@mmcloughlin done, you can now use:

curl -sL https://raw.githubusercontent.com/maxatome/install-go/v3.0/install-go.pl |
    perl - tip [DEST_DIR]

on my last tip test on go-testdeep it took 5 secs to setup go vs more than 3 mins before:

before: Setup go → 3m1s
after: Setup go → 5s

Example of use: https://github.com/maxatome/go-testdeep/blob/master/.github/workflows/ci.yml#L24-L25

Thanks for your help!

mmcloughlin commented 2 years ago

@maxatome Nice!

I'm curious why you didn't choose Go for the installer? I was thinking you could fork this:

https://github.com/golang/dl/blob/master/gotip/main.go https://github.com/golang/dl/tree/master/internal/version

You could make a gosnap tool that installs from the latest snapshot it can find.

Anyway, if it's Go you can use the dashboard and buildenv packages directly rather than parsing out the source code.

maxatome commented 2 years ago

@mmcloughlin perl executable is available almost everywhere. A go compiler is not. Downloading go to go run to download go reminds me chicken and egg problem. If we use an already compiled executable, it won't work for several OSes, so we need several ones, plus people can have doubt about the content of these executables, as we read above. Here the code is auditable and a sum can be computed to check that the downloaded script matches the audited one. Parsing the source code is certainly a weakness as it can change (it didn't for last 2 years), but it is not a blocker as it falls back on building tip as before.

mmcloughlin commented 2 years ago

@maxatome Got it, yeah Perl is ubiquitous.

Don't the Github Actions runners have preinstalled versions of Go? It says they have the last three versions cached but I don't know if go is guaranteed to be on the PATH, or in a predictable location.

https://github.com/actions/virtual-environments/blob/main/images/linux/Ubuntu1804-README.md#go

zikaeroh commented 2 years ago

Go is definitely on PATH, that's what my https://github.com/actions/setup-go/issues/21#issuecomment-565704236 uses to avoid running setup-go.

Perl is less ubiquitous when it comes to Windows, but GHA images have perl too. IMO there's really no difference between using the two at that point, if GHA images are guaranteed to have Go anyway. You can fairly easily just do go run github.com/whatever/gosnap@latest and not even worry about GOBIN either.

Best case, this code is stuck into setup-go and supported, if the Go team is actually okay with a mass of people grabbing these snapshots when they were intended for the Go builders. They could get cached, but that's going to be one huge cache.

mmcloughlin commented 2 years ago

@zikaeroh That's good, yeah I was thinking it would be fine if you could just use the pre-installed Go. It doesn't need to be a specific version, just not ancient.

Right, I should say I definitely don't speak for the Go team. I just discovered these snapshots once when I saw a post from Brad Fitzpatrick. He used the snapshots to produce a graph of the tarball size over time.

If people on here want to use it for their own projects that's probably fine. If we actually want this in setup-go we'd need to ask on golang-dev. Or maybe @mvdan has an idea.

Also would the cache need to be huge? Installs would only need the latest snapshot, you could expire the older ones.

zikaeroh commented 2 years ago

Also would the cache need to be huge? Installs would only need the latest snapshot, you could expire the older ones.

I guess that depends on the definition of huge. I was thinking that big list of potential snapshots cached for a few versions by the big size of the download, but I guess that's not "huge" for the likes of GitHub or Google; they've probably forgotten how to count that low. 😃

mvdan commented 2 years ago

If people on here want to use it for their own projects that's probably fine. If we actually want this in setup-go we'd need to ask on golang-dev. Or maybe @mvdan has an idea.

I think asking golang-dev sounds right. Assuming they are OK with it, I think the logic should be to try downloading a prebuilt archive, and fall back to a source download and build.

vearutop commented 2 years ago

Thank you for helpful discussion. I came up with this snippet as a workaround, it works reasonably fast, installing Go tip takes about 5 seconds.

    strategy:
      matrix:
        go-version: [ 1.16.x, 1.17.x, tip ]
    runs-on: ubuntu-latest
    steps:
      - name: Install Go stable
        if: matrix.go-version != 'tip'
        uses: actions/setup-go@master
        with:
          go-version: ${{ matrix.go-version }}
      - name: Install Go tip
        if: matrix.go-version == 'tip'
        run: |
          curl -sL https://storage.googleapis.com/go-build-snap/go/linux-amd64/$(git ls-remote https://github.com/golang/go.git HEAD | awk '{print $1;}').tar.gz -o gotip.tar.gz
          ls -lah gotip.tar.gz
          mkdir -p ~/sdk/gotip
          tar -C ~/sdk/gotip -xzf gotip.tar.gz
          ~/sdk/gotip/bin/go version
          echo "PATH=$HOME/go/bin:$HOME/sdk/gotip/bin/:$PATH" >> $GITHUB_ENV

dsame commented 11 months ago

Building the go from source impacts the action performance while not being useful for most of users. Since this advanced use case has a workaround with the custom build step the request it rejected.

mvdan commented 11 months ago

I'm sorry but that doesn't make sense. Building Go is quick - it takes a minute on a slim laptop, and it can be cached on your side easily since the build is fully reproducible.

This is not a niche or advanced use case either. Testing on Go pre-releases and development versions is crucial to spot any upstream issues or incompatibilities early.

powerman commented 11 months ago

@dsame In addition to @mvdan's reply I wanna say you don't even have to build tip from source, it will be helpful even if setup-go will implement same thing as in https://github.com/actions/setup-go/issues/21#issuecomment-997208686 and thus became a complete solution which doesn't need extra steps/workarounds to run tests on tip.

andig commented 7 months ago

Active discussion in https://github.com/actions/setup-go/issues/439. Suggesting to upvote there.

yurishkuro commented 3 weeks ago

Active discussion in #439. Suggesting to upvote there.

That link is dead.

I've been using the manual download solution from https://github.com/actions/setup-go/issues/21#issuecomment-997208686, but recently it started failing on the latest commit. It does not seem like https://storage.googleapis.com/go-build-snap has that snapshot anymore. Does anyone know of an alternate location to download gotip bundle?

mmcloughlin commented 3 weeks ago

I've been using the manual download solution from #21 (comment), but recently it started failing on the latest commit. It does not seem like https://storage.googleapis.com/go-build-snap has that snapshot anymore. Does anyone know of an alternate location to download gotip bundle?

I wonder if it's related to golang/go#63471.

actions / setup-go

Support tip? #21