search

chainguard-dev / gobump

Go tool to declaratively bump dependencies.
Apache License 2.0
7 stars 11 forks source link

go/bump package downloading order causing failure #24

Closed jamonation closed 8 months ago

jamonation commented 8 months ago

@jjm8759 discovered an issue with gobump and the order that it uses to download packages when running go get.

Here are two examples of the same pull request and the same gobump invocation showing different results. Running the failing GitHub actions job repeatedly results in success if the order of downloaded packages doesn't present any conflicts.

# We use the --tidy flag to run go mod tidy before and after in some cases (if old versions of go are used, we need to update the go.mod format)
gobump --packages "go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.46.1 google.golang.org/grpc@v1.58.3 golang.org/x/net@v0.17.0 k8s.io/apiserver@v0.28.0 go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc@v1.21.0 go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc@v0.46.1 golang.org/x/crypto@v0.17.0" --replaces "" --tidy=true --show-diff=false --go-version= --compat=

if [ -d "./vendor" ]; then
  go mod vendor
fi

exit 0
⚠️  aarch64   | 2024/02/06 21:19:39 Running go mod tidy with go version '1.20' ...
⚠️  aarch64   | 2024/02/06 21:19:41 Update package: go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc
⚠️  aarch64   | 2024/02/06 21:19:41 Running go get ...
⚠️  aarch64   | 2024/02/06 21:19:42 Update package: go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc
⚠️  aarch64   | 2024/02/06 21:19:42 Running go mod edit -droprequire ...
⚠️  aarch64   | 2024/02/06 21:19:42 Running go get ...
⚠️  aarch64   | 2024/02/06 21:19:44 Update package: golang.org/x/crypto
⚠️  aarch64   | 2024/02/06 21:19:44 Running go mod edit -droprequire ...
⚠️  aarch64   | 2024/02/06 21:19:44 Running go get ...
⚠️  aarch64   | 2024/02/06 21:19:44 Update package: go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp
⚠️  aarch64   | 2024/02/06 21:19:44 Running go mod edit -droprequire ...
⚠️  aarch64   | 2024/02/06 21:19:44 Running go get ...
⚠️  aarch64   | 2024/02/06 21:19:45 Update package: google.golang.org/grpc
⚠️  aarch64   | 2024/02/06 21:19:45 Running go mod edit -droprequire ...
⚠️  aarch64   | 2024/02/06 21:19:45 Running go get ...
⚠️  aarch64   | 2024/02/06 21:19:46 Update package: golang.org/x/net
⚠️  aarch64   | 2024/02/06 21:19:46 Running go mod edit -droprequire ...
⚠️  aarch64   | 2024/02/06 21:19:46 Running go get ...
⚠️  aarch64   | 2024/02/06 21:19:46 Update package: k8s.io/apiserver
⚠️  aarch64   | 2024/02/06 21:19:46 Running go mod edit -droprequire ...
⚠️  aarch64   | 2024/02/06 21:19:46 Running go get ...
⚠️  aarch64   | 2024/02/06 21:19:47 Running go mod tidy with go version '1.20' ...
⚠️  aarch64   | Error: Failed to running update. Error: package go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc with v1.19.0 is less than the desired version v1.21.0
⚠️  aarch64   | Usage:
⚠️  aarch64   |   gobump [flags]
64   |   gobump [command]
⚠️  aarch64   | 
⚠️  aarch64   | Available Commands:
⚠️  aarch64   |   completion  Generate the autocompletion script for the specified shell
⚠️  aarch64   |   help        Help about any command
⚠️  aarch64   |   version     Prints the version
⚠️  aarch64   | 
⚠️  aarch64   | Flags:
⚠️  aarch64   |       --compat string       set the go version for which the tidied go.mod and go.sum files should be compatible
⚠️  aarch64   |       --go-version string   set the go-version for go-mod-tidy
⚠️  aarch64   |   -h, --help                help for gobump
⚠️  aarch64   |       --modroot string      path to the go.mod root
⚠️  aarch64   |       --packages string     A space-separated list of packages to update
⚠️  aarch64   |       --replaces string     A space-separated list of packages to replace
⚠️  aarch64   |       --show-diff           Show the difference between the original and 'go.mod' files
⚠️  aarch64   |       --tidy                Run 'go mod tidy' command
⚠️  aarch64   | 
64   | Use "gobump [command] --help" for more information about a command.
⚠️  aarch64   | 
⚠️  aarch64   | 2024/02/06 21:19:48 Failed to running update. Error: package go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc with v1.19.0 is less than the desired version v1.21.0
❌  aarch64   | ERROR: failed to build package. the build environment has been preserved:
ℹ️  aarch64   |   workspace dir: /tmp/melange-workspace-60501173
ℹ️  aarch64   |   guest dir: /tmp/melange-guest-3060968994
ℹ️            | error during command execution: failed to build package: unable to run pipeline: exit status 1

Running the same job again a few times and it succeeds. The order of the dependencies is different this time, but the gobump command is the same:

# We use the --tidy flag to run go mod tidy before and after in some cases (if old versions of go are used, we need to update the go.mod format)
gobump --packages "go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.46.1 google.golang.org/grpc@v1.58.3 golang.org/x/net@v0.17.0 k8s.io/apiserver@v0.28.0 go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc@v1.21.0 go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc@v0.46.1 golang.org/x/crypto@v0.17.0" --replaces "" --tidy=true --show-diff=false --go-version= --compat=

if [ -d "./vendor" ]; then
  go mod vendor
fi

exit 0
⚠️  aarch64   | 2024/02/06 22:02:21 Running go mod tidy with go version '1.20' ...
⚠️  aarch64   | 2024/02/06 22:02:25 Update package: go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp
⚠️  aarch64   | 2024/02/06 22:02:25 Running go mod edit -droprequire ...
⚠️  aarch64   | 2024/02/06 22:02:25 Running go get ...
⚠️  aarch64   | 2024/02/06 22:02:27 Update package: google.golang.org/grpc
⚠️  aarch64   | 2024/02/06 22:02:27 Running go mod edit -droprequire ...
⚠️  aarch64   | 2024/02/06 22:02:27 Running go get ...
⚠️  aarch64   | 2024/02/06 22:02:28 Update package: golang.org/x/net
⚠️  aarch64   | 2024/02/06 22:02:28 Running go mod edit -droprequire ...
⚠️  aarch64   | 2024/02/06 22:02:28 Running go get ...
⚠️  aarch64   | 2024/02/06 22:02:28 Update package: k8s.io/apiserver
⚠️  aarch64   | 2024/02/06 22:02:28 Running go mod edit -droprequire ...
⚠️  aarch64   | 2024/02/06 22:02:28 Running go get ...
⚠️  aarch64   | 2024/02/06 22:02:29 Update package: go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc
⚠️  aarch64   | 2024/02/06 22:02:29 Running go get ...
⚠️  aarch64   | 2024/02/06 22:02:31 Update package: go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc
⚠️  aarch64   | 2024/02/06 22:02:31 Running go mod edit -droprequire ...
⚠️  aarch64   | 2024/02/06 22:02:31 Running go get ...
⚠️  aarch64   | 2024/02/06 22:02:32 Update package: golang.org/x/crypto
⚠️  aarch64   | 2024/02/06 22:02:32 Running go mod edit -droprequire ...
⚠️  aarch64   | 2024/02/06 22:02:32 Running go get ...
⚠️  aarch64   | 2024/02/06 22:02:33 Running go mod tidy with go version '1.20' ...
ℹ️            | executing: bwrap --bind /tmp/melange-guest-4253256722 / --bind /tmp/melange-workspace-3312508085 /home/build --bind /etc/resolv.conf /etc/resolv.conf --bind /__w/enterprise-packages/.melangecache /var/cache/melange --unshare-pid --dev /dev --proc /proc --chdir /home/build --clearenv --new-session --setenv LDFLAGS -Wl,--as-needed,-O1,--sort-common -Wl,-z,relro,-z,now --setenv GOFLAGS -buildmode=pie --setenv HOME /home/build --setenv GOPATH /home/build/.cache/go --setenv CPPFLAGS -O2 -Wall -fomit-frame-pointer -march=armv8-a -mtune=neoverse-n1 -Wp,-D_FORTIFY_SOURCE=3 --setenv CXXFLAGS -O2 -Wall -fomit-frame-pointer -march=armv8-a -mtune=neoverse-n1 -Wp,-D_FORTIFY_SOURCE=3 --setenv GOMODCACHE /var/cache/melange/gomodcache --setenv CFLAGS -O2 -Wall -fomit-frame-pointer -march=armv8-a -mtune=neoverse-n1 -Wp,-D_FORTIFY_SOURCE=3 --setenv SOURCE_DATE_EPOCH 1707254203 /bin/sh -c set -e 
export PATH='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'

The behaviour is the same locally and in GitHub actions CI. Is there a way to force the order of packages, or would it be better to run individual go get commands for complicated sets of dependencies?

hectorj2f commented 8 months ago

@jamonation Thanks for opening the issue. We could enforce the order of the dependencies to be the same than in the input.

hectorj2f commented 8 months ago

related to this issue: https://chainguard-dev.slack.com/archives/C0636FTRFED/p1709678145047489?thread_ts=1709672760.605789&cid=C0636FTRFED