Open sluongng opened 2 weeks ago
Just a note: Even if we implement this, we still wouldn't benefit from the repository cache in its current form as we can't provide a hash for the .zip
file itself, only for its contents.
Right. The current hash in go.sum
is the hash of the extracted archive directory using https://pkg.go.dev/golang.org/x/mod/sumdb/dirhash#HashDir and not of the .zip
archive itself.
I wonder if there is something we can do in Bazel to support rctx.download_directory
that is mapped to Remote Asset API FetchDirectory
rpc.
Currently in go_repository, unless the
url
attribute is set specifically, a go module is fetched via callingcmd/fetch_repo
https://github.com/bazelbuild/bazel-gazelle/blob/d16fc424cfcfdbd8893a267e500ec5d04ec0bbf0/internal/go_repository.bzl#L147
https://github.com/bazelbuild/bazel-gazelle/blob/d16fc424cfcfdbd8893a267e500ec5d04ec0bbf0/cmd/fetch_repo/main.go#L55
fetch_repo
will in turn callgo mod download
to (a) download, (b) extract and (c) verify the content using GOSUMDB.Because we are not using Bazel's native downloader via
rctx.download
orrctx.download_and_extract
, the downloaded go modules are not cached via Bazel's--repository_cache=
or does it benefit from--experimental_remote_downloader=
.We should be able to replicate (a) and (b) of
go mod download
by reconstructing the download URLs in starlark and feed it torctx.download_and_extract
to expand the zip. The URLs could be a mix of proxy download and direct download to replicate GOPROXY config.For (c), we can create a custom binary called
cmd/verify_zip
to validate the downloaded content against GOSUMDB and GONOSUMDB config.