Closed piotrkubisa closed 2 years ago
I have no idea where to even begin to debug something like that.
I tried running the script in your gist on 3 different networks (home, cellular, coffeeshop) to try and reproduce.
First, I tried with no other dependencies besides berglas (removed "github.com/sirupsen/logrus"). On all three networks, I go no diff.
Next, I tried adding "github.com/sirupsen/logrus" back in, but got the same result.
My suspicion is that your proxy, network, ISP, or something higher up the chain is modifying the response.
Logrus has nothing to do with the issue. It presents a completely other dependency that does not seem to have different checksum, whether the GOPRIVATE=*
is set or not. It also takes part in verification if my network setup has some proxy or doesn't. Because, the checksum is exact for both scenarios, I think I don't have any unknown proxy of my ISP.
Also, because we start blaming someone's other network, which likely has nothing to do with my problem, I have prepared you two GitHub workflows, one with GOPRIVATE=*
and other without. I find GitHub-managed workflow runners as a neutral ground and good place for more investigation. Here are results, which confirms my thesis - I am not only one, who can be haunted by this problem:
GOPRIVATE
github.com/GoogleCloudPlatform/berglas v0.6.2 h1:mgcdWF53ltZA0hBGWrpBAjuKh5VnQPxHRlFRoHd5tpA=
github.com/GoogleCloudPlatform/berglas v0.6.2/go.mod h1:LEKpytS5wf+P8OGenFlsVmi/uwqcjkFQTH8wZABaCgI=
GOPRIVATE=*
github.com/GoogleCloudPlatform/berglas v0.6.2 h1:GeTSHX7B/eIONuwnVuh92I3f0yAmdSz6RdAYP5/efQI=
github.com/GoogleCloudPlatform/berglas v0.6.2/go.mod h1:LEKpytS5wf+P8OGenFlsVmi/uwqcjkFQTH8wZABaCgI=
Alright, I reworked your example a bit to eliminate as many variables as possible and was still able to reproduce it (workflow).
I downloaded the cached binary from https://proxy.golang.org/github.com/%21google%21cloud%21platform/berglas/@v/v0.6.2.zip, unzipped it, and compared it to the v0.6.2 tag. It looks like none of the examples/
are included:
HEAD detached at v0.6.2
Changes not staged for commit:
(use "git add/rm <file>..." to update what will be committed)
(use "git restore <file>..." to discard changes in working directory)
deleted: examples/appengine/go/.gcloudignore
deleted: examples/appengine/go/.gitignore
deleted: examples/appengine/go/README.md
deleted: examples/appengine/go/app.yaml
deleted: examples/appengine/go/go.mod
deleted: examples/appengine/go/go.sum
deleted: examples/appengine/go/main.go
deleted: examples/appengineflex/go/.gitignore
deleted: examples/appengineflex/go/Dockerfile
deleted: examples/appengineflex/go/README.md
deleted: examples/appengineflex/go/app.yaml
deleted: examples/appengineflex/go/go.mod
deleted: examples/appengineflex/go/go.sum
deleted: examples/appengineflex/go/main.go
deleted: examples/cloudfunctions/go/README.md
deleted: examples/cloudfunctions/go/fn.go
deleted: examples/cloudfunctions/go/go.mod
deleted: examples/cloudfunctions/go/go.sum
deleted: examples/cloudrun/go/Dockerfile
deleted: examples/cloudrun/go/README.md
deleted: examples/cloudrun/go/go.mod
deleted: examples/cloudrun/go/go.sum
deleted: examples/cloudrun/go/main.go
deleted: examples/kubernetes/README.md
deleted: examples/kubernetes/deploy/sample.yaml
deleted: examples/kubernetes/deploy/webhook.yaml
deleted: examples/kubernetes/go.mod
deleted: examples/kubernetes/go.sum
deleted: examples/kubernetes/main.go
This makes me think that the proxy doesn't include that directory, either because it declares its own go.sum/mod files, or because it is named examples
(which seems weird).
Nonetheless, I don't see anything malicious here. If you believe the Go module proxy is modifying packages in ways that it shouldn't, I would recommend using the contact information at the bottom of https://proxy.golang.org/.
Okay, that sounds great!
I have strong suspicion towards examples/
directory, which you have named as possible root of the problem. I have done two little changes to the berglas
repository in my fork:
rm -rf examples/
completely. The go.sum
file started to look exactly the same in both scenarios.examples
dir (mv examples renamed-examples-dir
). The workflow says, the checksum is identical in both scenarios too.$ export GOPRIVATE=*
$ go mod download -json -x github.com/piotrkubisa/berglas@v0.6.4
{
"Path": "github.com/piotrkubisa/berglas",
"Version": "v0.6.4",
"Info": "/home/piotr/go/pkg/mod/cache/download/github.com/piotrkubisa/berglas/@v/v0.6.4.info",
"GoMod": "/home/piotr/go/pkg/mod/cache/download/github.com/piotrkubisa/berglas/@v/v0.6.4.mod",
"Zip": "/home/piotr/go/pkg/mod/cache/download/github.com/piotrkubisa/berglas/@v/v0.6.4.zip",
"Dir": "/home/piotr/go/pkg/mod/github.com/piotrkubisa/berglas@v0.6.4",
"Sum": "h1:d6xGda/YPfMBD3q6Qbpgqi5slRlQ4ULJ5w/8nPW4fK4=",
"GoModSum": "h1:LEKpytS5wf+P8OGenFlsVmi/uwqcjkFQTH8wZABaCgI="
}
$ unset GOPRIVATE
$ go mod download -json -x github.com/piotrkubisa/berglas@v0.6.4
{
"Path": "github.com/piotrkubisa/berglas",
"Version": "v0.6.4",
"Info": "/home/piotr/go/pkg/mod/cache/download/github.com/piotrkubisa/berglas/@v/v0.6.4.info",
"GoMod": "/home/piotr/go/pkg/mod/cache/download/github.com/piotrkubisa/berglas/@v/v0.6.4.mod",
"Zip": "/home/piotr/go/pkg/mod/cache/download/github.com/piotrkubisa/berglas/@v/v0.6.4.zip",
"Dir": "/home/piotr/go/pkg/mod/github.com/piotrkubisa/berglas@v0.6.4",
"Sum": "h1:d6xGda/YPfMBD3q6Qbpgqi5slRlQ4ULJ5w/8nPW4fK4=",
"GoModSum": "h1:LEKpytS5wf+P8OGenFlsVmi/uwqcjkFQTH8wZABaCgI="
}
Would the rename from examples/
to demo/
will fix this problem?
This issue is stale because it has been open for 14 days with no activity. It will automatically close after 7 more days of inactivity.
Would the rename from examples/ to demo/ will fix this problem?
It's possible, but I "examples" is a pretty common name. I really think a bug should be opened against the module registry for this.
References
What?
Whether
GOPRIVATE=*
environment has been set, the checksum ingo.sum
forberglas
dependency is different than if I would not setGOPRIVATE
(standard Go installation). This issue makes unable to have defined customGOPRIVATE
across the team and/or CI environments, because Go compiler will deny any further operations.While, I rely on numerous go-modules (including various Gcloud SDKs) I have only encountered this problem in
berglas
. The fact that theberglas
manages secrets, this issue might be a security-related problem, because defaultsum.golang.org
offers different version ofberglas
than actually is distributed overgithub.com
.How to reproduce?
In a secret gist I have prepared a sample project that imports
github.com/GoogleCloudPlatform/berglas
and prints out thego.sum
, depending whetherGOPRIVATE=*
was or wasn't defined.In the
diff(1)
output presented below, I compare the computedgo.sum
file withGOPRIVATE=*
set and without respectively.If you would like to test it on your machine, please take a look in my gist - https://gist.github.com/piotrkubisa/4dbba8d36bf3748944489dc523eefead. I hope example presenting sample code will be more clear for you.
main.go
Dockerfile