The filer package already supports FromGitURL which fetches the contents of a remote Git repo, but this doesn't cover the case of fetching and analyzing a Go module's source, wherever it may be. Go modules can be fetched from Mercurial, Subversion, etc., and might have import path aliases in place that make FromGitURL tricky.
I'd like to suggest adding filer.FromGoModule that would address this by fetching the module's source from the Go module proxy (proxy.golang.org by default) as a zip file, and analyzing it as FromZip does today -- possibly using a shared internal method that takes a zip.Reader instead of having to write to disk.
For example, filer.FromGoModule("knative.dev/serving@v0.26.0") would fetch and process https://proxy.golang.org/knative.dev/serving/@v/v0.26.0.zip. Note that this import path has an alias configured, and the user didn't have to perform a lookup to find that knative.dev/serving aliases https://github.com/knative/serving, then detect that it's GitHub and use Git, or GitHub's zip archive, etc. If knative.dev/serving moves to Mercurial or Subversion or non-GitHub, the caller wouldn't have to be aware.
If the module isn't found in the proxy, return an error. Users can set the GOPROXY env var to point to a different (e.g., internal corporate) module proxy, and filer.FromGoModule should support that. I'd suggest that filer.FromGoModule shouldn't support unversioned importpaths (e.g., knative.dev/serving) since looking up the latest correct version adds complexity, and I imagine in most cases users will be processing a go.mod file where versions are available anyway.
edit: a quick sketch of the code is here: https://github.com/go-enry/go-license-detector/compare/master...imjasonh:go-module -- it buffers the zip in memory on first read; it could also just write to a temp directory and filer.FromZIP it. It also doesn't take into account GOPROXY yet, still WIP. Let me know if I should keep working on this and send a PR.
The
filer
package already supportsFromGitURL
which fetches the contents of a remote Git repo, but this doesn't cover the case of fetching and analyzing a Go module's source, wherever it may be. Go modules can be fetched from Mercurial, Subversion, etc., and might have import path aliases in place that makeFromGitURL
tricky.I'd like to suggest adding
filer.FromGoModule
that would address this by fetching the module's source from the Go module proxy (proxy.golang.org by default) as a zip file, and analyzing it asFromZip
does today -- possibly using a shared internal method that takes azip.Reader
instead of having to write to disk.For example,
filer.FromGoModule("knative.dev/serving@v0.26.0")
would fetch and processhttps://proxy.golang.org/knative.dev/serving/@v/v0.26.0.zip
. Note that this import path has an alias configured, and the user didn't have to perform a lookup to find thatknative.dev/serving
aliases https://github.com/knative/serving, then detect that it's GitHub and use Git, or GitHub's zip archive, etc. Ifknative.dev/serving
moves to Mercurial or Subversion or non-GitHub, the caller wouldn't have to be aware.If the module isn't found in the proxy, return an error. Users can set the
GOPROXY
env var to point to a different (e.g., internal corporate) module proxy, andfiler.FromGoModule
should support that. I'd suggest thatfiler.FromGoModule
shouldn't support unversioned importpaths (e.g.,knative.dev/serving
) since looking up the latest correct version adds complexity, and I imagine in most cases users will be processing a go.mod file where versions are available anyway.More info about the Go module proxy protocol:
edit: a quick sketch of the code is here: https://github.com/go-enry/go-license-detector/compare/master...imjasonh:go-module -- it buffers the zip in memory on first read; it could also just write to a temp directory and
filer.FromZIP
it. It also doesn't take into accountGOPROXY
yet, still WIP. Let me know if I should keep working on this and send a PR.