gazelle issues with sum violations

[hsyed]

What version of gazelle are you using?

0.18.2

What version of rules_go are you using?

0.19.4

What version of Bazel are you using?

1.1.0

Does this issue reproduce with the latest releases of all the above?

no

What operating system and processor architecture are you using?

Darwin catalina

Problem

This dependency is throwing a go sum db error

➜  esqimo-platform git:(master) ✗ go get -u github.com/friendsofgo/errors
verifying github.com/friendsofgo/errors@v0.9.1: checksum mismatch
    downloaded: h1:ksGIdtObXKKxxMq9VgsjKecdLCOlXwsE6XypldOuDi0=
    sum.golang.org: h1:VnyCwEgi1BgGL4wAfCxBOLsSNyJ+iAwJOxh4Gbq7aI4=

SECURITY ERROR
This download does NOT match the one reported by the checksum server.
The bits may have been replaced on the origin server, or an attacker may
have intercepted the download attempt.

For more information, see 'go help module-auth'.

it's a transitive dependency for sqlboiler

gazelle also errors when run on our mono repo:

gazelle: finding module path for import github.com/volatiletech/null: exit status 1: verifying github.com/friendsofgo/errors@v0.9.1: checksum mismatch
    downloaded: h1:ksGIdtObXKKxxMq9VgsjKecdLCOlXwsE6XypldOuDi0=
    sum.golang.org: h1:VnyCwEgi1BgGL4wAfCxBOLsSNyJ+iAwJOxh4Gbq7aI4=

SECURITY ERROR
This download does NOT match the one reported by the checksum server.
The bits may have been replaced on the origin server, or an attacker may
have intercepted the download attempt.

For more information, see 'go help module-auth'.
gazelle: finding module path for import github.com/volatiletech/null: exit status 1: verifying github.com/friendsofgo/errors@v0.9.1: checksum mismatch
    downloaded: h1:ksGIdtObXKKxxMq9VgsjKecdLCOlXwsE6XypldOuDi0=
    sum.golang.org: h1:VnyCwEgi1BgGL4wAfCxBOLsSNyJ+iAwJOxh4Gbq7aI4=

SECURITY ERROR
This download does NOT match the one reported by the checksum server.
The bits may have been replaced on the origin server, or an attacker may
have intercepted the download attempt.

For more information, see 'go help module-auth'.
gazelle: finding module path for import github.com/volatiletech/sqlboiler/queries/qm: exit status 1: verifying github.com/friendsofgo/errors@v0.9.1: checksum mismatch
    downloaded: h1:ksGIdtObXKKxxMq9VgsjKecdLCOlXwsE6XypldOuDi0=
    sum.golang.org: h1:VnyCwEgi1BgGL4wAfCxBOLsSNyJ+iAwJOxh4Gbq7aI4=

SECURITY ERROR
This download does NOT match the one reported by the checksum server.
The bits may have been replaced on the origin server, or an attacker may
have intercepted the download attempt.

For more information, see 'go help module-auth'.
gazelle: finding module path for import github.com/volatiletech/sqlboiler/queries/qm: exit status 1: verifying github.com/friendsofgo/errors@v0.9.1: checksum mismatch
    downloaded: h1:ksGIdtObXKKxxMq9VgsjKecdLCOlXwsE6XypldOuDi0=
    sum.golang.org: h1:VnyCwEgi1BgGL4wAfCxBOLsSNyJ+iAwJOxh4Gbq7aI4=

Observed issues

Gazelle doesn't hard error out, it proceeds to generate the build files. But any external dep that transitively depends on github.com/friendsofgo/errors just gets removed from the existing build files.

Workarround ?

Our build is currently broken so any work around would be appreciated. How do we GOSUMDB=off gazelle ?

[hsyed]

I have no idea how this problem showed up all of a sudden github.com/friendsofgo/errors has seen no activity recently so what caused this ?

see: https://github.com/friendsofgo/errors/issues/2

[hsyed]

I think it is a bug that gazelle continues and silently drops pkgs with sum-violating packages in the transitive closure. The error is logged out so it should just continue business as usual I think ?

[hsyed]

The offending package is not listed in our go_repositories() macro it is fetched by a transitive dep.

[hsyed]

updated our make target to :

build:
    GOSUMDB=off bazel run //:gazelle -- update-repos -prune=true -from_file=go.mod -to_macro=build/go_repositories.bzl%go_repositories
    GOSUMDB=off bazel run //:gazelle
    GOSUMDB=off bazel build //:platform

Also a bazel clean --expunge has to be done if the a fetch with a sum violation has been cached.

I don't see another work around. Declaring the offending package in the workspace doesn't solve the problem as it is used by many other external packages and these are not go modules. We also have no way to control what version of the package they chose to fetch.

[aperezg]

This was a bug in our side, we've fixed that publishing a new release, so now all must be working again. Truly sorry!

[jayconrod]

Sorry for taking so long to respond on this. The repeated error messages are certainly not a good experience.

When Gazelle resolves an import path to a dependency outside of the current prefix, it tries to resolve the path against known go_repository rules in WORKSPACE. You can declare additional rules with # gazelle:repository or functions full of rules with # gazelle:repository_macro. If no repository is found, Gazelle runs go list -find -f {{.Module.Path}} -- $importPath. Any errors from that command need to be reported.

I'm marking this as a bug and leaving it open though because we probably shouldn't be calling go list at all anymore. If something can't be resolved from known repositories, we could run go list -m -json all once and use that to resolve packages to modules.

[jayconrod]

Note to anyone reading, checksum errors are serious and you shouldn't workaround them without understanding why they happened. Usually, this is caused by a module author rewriting a version tag. They should change the tag back to the original commit. Users should upgrade or downgrade to a working version.

You can look up the "canonical" sum with:

curl https://sum.golang.org/lookup/<module>@<version>

[loeffel-io]

to keep it easy you can create a simple function for this

# .functions
function golangSum(){
    curl https://sum.golang.org/lookup/$1@$2
}