Including internal, closed source repos causes lookup failures

[everactivejack]

We have a private gitlab where we host our own go repos, and it seems, even with overrides or exceptions, I can't prevent the sum.golang.org lookup, which causes a failure -- thus preventing us from using this in CI. To be clear, our gitlab instance isn't accessible except on internal networks, so this lookup fails. See snippet for more details (I anonymized the URL, but I think you'll get the point).

$ (lichen -c lichen.yaml ./${PROJECT_NAME} | tee license_report.txt) || true
2021/07/22 15:04:44 failed to evaluate licenses: failed to fetch: exit status 1 (output: 
{
    "Path": "github.com/hashicorp/hcl",
    "Version": "v1.0.0",
    "Info": "/go/pkg/mod/cache/download/github.com/hashicorp/hcl/@v/v1.0.0.info",
    "GoMod": "/go/pkg/mod/cache/download/github.com/hashicorp/hcl/@v/v1.0.0.mod",
    "Zip": "/go/pkg/mod/cache/download/github.com/hashicorp/hcl/@v/v1.0.0.zip",
    "Dir": "/go/pkg/mod/github.com/hashicorp/hcl@v1.0.0",
    "Sum": "h1:0Anlzjpi4vEasTeNFn2mLJgTSwt0+6sfsiTG8qcWGx4=",
    "GoModSum": "h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ="
}
{
    "Path": "github.com/spf13/afero",
    "Version": "v1.3.4",
    "Info": "/go/pkg/mod/cache/download/github.com/spf13/afero/@v/v1.3.4.info",
    "GoMod": "/go/pkg/mod/cache/download/github.com/spf13/afero/@v/v1.3.4.mod",
    "Zip": "/go/pkg/mod/cache/download/github.com/spf13/afero/@v/v1.3.4.zip",
    "Dir": "/go/pkg/mod/github.com/spf13/afero@v1.3.4",
    "Sum": "h1:8q6vk3hthlpb2SouZcnBVKboxWQWMDNF38bwholZrJc=",
    "GoModSum": "h1:Ai8FlHk4v/PARR026UzYexafAt9roJ7LcLMAmO6Z93I="
}
# ...
# truncated ...
# ...
{
    "Path": "gitlab.ourcompany.com/software/whatever/thing",
    "Version": "v0.0.0-20210610205813-eb9a997ab73f",
    "Error": "gitlab.ourcompany.com/software/whatever/thing@v0.0.0-20210610205813-eb9a997ab73f: verifying go.mod: gitlab.ourcompany.com/software/whatever/thing@v0.0.0-20210610205813-eb9a997ab73f/go.mod: reading https://sum.golang.org/lookup/gitlab.ourcompany.com/software/whatever/thing@v0.0.0-20210610205813-eb9a997ab73f: 410 Gone\n\tserver response: not found: gitlab.ourcompany.com/software/whatever/thing@v0.0.0-20210610205813-eb9a997ab73f: unrecognized import path \"gitlab.ourcompany.com/software/whatever/thing\": https fetch: Get \"https://gitlab.ourcompany.com/software/whatever/thing?go-get=1\": dial tcp: lookup gitlab.everactive.com on 8.8.8.8:53: no such host",
    "Info": "/go/pkg/mod/cache/download/gitlab.ourcompany.com/software/whatever/thing/@v/v0.0.0-20210610205813-eb9a997ab73f.info"
}

Not sure if this is really a bug or a feature request, so I'll leave that up to you, but I would imagine maybe adding some sort of config option to ignore modules that start with a certain URL?

ignoreURL: gitlab.ourcompany.com

Or making it so that override directives don't query sum.golang.org at all.

[nick-jones]

Thanks for the detailed report. Just to check a couple of things:

• Is the private repo accessible from the CI build on a general basis, or is it entirely inaccessible from that context?
• If it is accessible, is GOPRIVATE set? That would in theory prevent requests to sum.golang.org

[everactivejack]

It's definitely accessible from CI (it's part of that gitlab's CI), but we don't have the GOPRIVATE set in CI, so I'll give that a shot!

[everactivejack]

That did it! Thank you! Might be worth adding a section to the README for future private repo users. Either way, great tool.

[nick-jones]

I've just added a note in the README :+1: Apologies for being slow getting back to this.

[everactivejack]

Thanks!