Closed qequ closed 11 months ago
@fviernau Could we change the analyzer not to crash but simple continue the analyzer and throw an issue?
Hi @qequ,
Can you please provide a minimal example for reproducing this? What exactly I'd like to receive I've outlined already in a previous bug you've been involved in too, see https://github.com/oss-review-toolkit/ort/issues/6615#issuecomment-1455735479.
Ping @qequ.
Closing this as there was no feedback from the OP in months. Feel free to reopen if new information becomes available.
@sschuberth I believe that ORT should require the definition files to be in sync with what tidy
would output. If we would agree this make sense, we could implement a check for that and provide a more meaningful analyzer issue, which asks the user to run tidy
. What do you think? (I'm no a Go expert, so it'd need to be confirmed whether such requirement would be feasible)
What do you think?
Wouldn't it make more sense then to simply run tidy
beforehand ourselves? Assuming it's a tool that's always available when go
itself is.
Ping @qequ.
Pinging him myself. He will come back to this, sorry about the delay.
@sschuberth @fviernau the go.mod that produces the crash
module github.com/path/repo
go 1.18
require (
github.com/BurntSushi/toml v1.2.0
github.com/PuerkitoBio/goquery v1.8.0
github.com/SebastiaanKlippert/go-wkhtmltopdf v1.7.2
github.com/agiledragon/gomonkey v2.0.2+incompatible
github.com/cnkei/gospline v0.0.0-20191204052713-d67fac29a294
github.com/gin-gonic/gin v1.8.1
github.com/go-kit/kit v0.12.0
github.com/go-kit/log v0.2.1
github.com/gogo/protobuf v1.3.2
github.com/golang/protobuf v1.5.2
github.com/grpc-ecosystem/grpc-gateway v1.16.0
github.com/kkyr/fig v0.3.0
github.com/koding/multiconfig v0.0.0-20171124222453-69c27309b2d7
github.com/krzysztofreczek/go-structurizr v0.1.32
github.com/nicksnyder/go-i18n/v2 v2.2.1
github.com/remeh/sizedwaitgroup v1.0.0
github.com/rs/zerolog v1.28.0
github.com/stretchr/testify v1.8.1
github.com/wcharczuk/go-chart v2.0.2-0.20190219185203-9852fce5a172+incompatible
golang.org/x/exp v0.0.0-20230105000112-eab7a2c85304
golang.org/x/image v0.0.0-20220902085622-e7cb96979f69
golang.org/x/net v0.0.0-20221004154528-8021a29435af
golang.org/x/oauth2 v0.0.0-20221006150949-b44042a4b9c1
golang.org/x/text v0.4.0
google.golang.org/genproto v0.0.0-20221010155953-15ba04fc1c0e
google.golang.org/grpc v1.50.0
gopkg.in/h2non/gock.v1 v1.1.2
)
require (
github.com/andybalholm/cascadia v1.3.1 // indirect
github.com/blend/go-sdk v2.0.0+incompatible // indirect
github.com/cnf/structhash v0.0.0-20201127153200-e1b16c1ebc08 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/fatih/camelcase v1.0.0 // indirect
github.com/fatih/structs v1.1.0 // indirect
github.com/gin-contrib/sse v0.1.0 // indirect
github.com/go-logfmt/logfmt v0.5.1 // indirect
github.com/go-playground/locales v0.14.0 // indirect
github.com/go-playground/universal-translator v0.18.0 // indirect
github.com/go-playground/validator/v10 v10.10.0 // indirect
github.com/goccy/go-json v0.9.7 // indirect
github.com/golang/freetype v0.0.0-20170609003504-e2365dfdc4a0 // indirect
github.com/h2non/parth v0.0.0-20190131123155-b4df798d6542 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/leodido/go-urn v1.2.1 // indirect
github.com/mattn/go-colorable v0.1.12 // indirect
github.com/mattn/go-isatty v0.0.14 // indirect
github.com/mitchellh/mapstructure v1.4.2 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/pelletier/go-toml v1.9.3 // indirect
github.com/pelletier/go-toml/v2 v2.0.1 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/stretchr/objx v0.5.0 // indirect
github.com/ugorji/go/codec v1.2.7 // indirect
golang.org/x/crypto v0.0.0-20220214200702-86341886e292 // indirect
golang.org/x/sys v0.1.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/protobuf v1.28.1 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)
Wouldn't it make more sense then to simply run
tidy
beforehand ourselves? Assuming it's a tool that's always available whengo
itself is.
In order to check whether the files are in line with tidy
we would need to run tidy
anyway. However, the reason why I proposed to flag an issue was, that I believe it's better to treat this as a problem the user needs to fix. So, that when GoMod
is run, it produces the outputs corresponding the the actual build script, not to modified uncommitted version of it.
@sschuberth @fviernau the go.mod that produces the crash
@dgutson could you attach the output of ORT as well?
@sschuberth @fviernau the go.mod that produces the crash
@dgutson could you attach the output of ORT as well?
@qequ ? :)
@sschuberth
when running analyzer
on repo;
12:20:08.197 [DefaultDispatcher-worker-1] ERROR org.ossreviewtoolkit.analyzer.PackageManager - GoMod failed to resolve dependencies for path 'go.mod': IOException: Running 'go list -m -json -buildvcs=false all' in '/home/alvaro/repo' failed with exit code 1:
go: github.com/blend/go-sdk@v2.0.0+incompatible: invalid version: unknown revision v2.0.0
go: unrecognized import path "gopkg.in/h2non/gock.v1": reading https://gopkg.in/h2non/gock.v1?go-get=1: 502 Bad Gateway
server response: Cannot obtain refs from GitHub: cannot talk to GitHub: Get https://github.com/h2non/gock.git/info/refs?service=git-upload-pack: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
go: unrecognized import path "gopkg.in/yaml.v2": reading https://gopkg.in/yaml.v2?go-get=1: 502 Bad Gateway
server response: Cannot obtain refs from GitHub: cannot talk to GitHub: Get https://github.com/go-yaml/yaml.git/info/refs?service=git-upload-pack: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
go: github.com/blend/go-sdk@v2.0.0+incompatible: reading github.com/blend/go-sdk/go.mod at revision v2.0.0: unknown revision v2.0.0
running with source, latest from main,
./cli/build/install/ort/bin/ort analyze -i ~/repo/ -o ~/repo/
@qequ - The log says that one cannot run go list -m -json -buildvcs=false all
, which is required by ORT. So, the bug is not within ORT but within the build files of the project. It can be fixed automatically by just using tidy
, right?
@sschuberth I ran go mod tidy
and keeps crashing with the same issue;
11:26:54.127 [DefaultDispatcher-worker-1] ERROR org.ossreviewtoolkit.analyzer.PackageManager - GoMod failed to resolve dependencies for path 'go.mod': IOException: Running 'go list -m -json -buildvcs=false all' in '/home/alvaro/project' failed with exit code 1:
go: github.com/blend/go-sdk@v2.0.0+incompatible: invalid version: unknown revision v2.0.0
go: github.com/blend/go-sdk@v2.0.0+incompatible: reading github.com/blend/go-sdk/go.mod at revision v2.0.0: unknown revision v2.0.0
I can reproduce the issue above with GOPROXY=direct
which is the setting GoMod
in ORT uses.
Without GOPROXY
set, the issue does not reproduce.
ORT uses GOPROXY=direct
, because this ensures that VCS info is available for all resolved dependencies.
If proxies are used, this is not guaranteed. So, ORT relies on that.
The reason why this issue happens is because go list
tries to find the git tag v2.0.0
matching the version specified in go.mod
. That git tag does not exist anymore , so it must have been deleted. So, using the proxy only works because that proxy has the module cached. If the proxy cleared it cache, it would also not find the release. Note that the tag has been renamed to legacy-v2.0.0
.
As a work around you can just alter the entry in go.mod (mind update the sum file):
github.com/blend/go-sdk legacy-v2.0.0
Go tooling will alter the entry to the following, which probably would be the right one to use in the first place:
require github.com/blend/go-sdk v0.0.0-20190401002810-24dca5bdfc9a
Closing this as won't fix, since the problem was caused by release tag deletion. So, ORT is actually right in not being able to find the release.
@qequ Mind trying the above work around and leaving some feedback below?
@fviernau tested it and updated the dependency with
require github.com/blend/go-sdk v0.0.0-20190401002810-24dca5bdfc9a // indirect
and the analyzer produced the output correctly :+1:
@dgutson @arieltorti FYI
Running the analyzer in a go.mod with this specific dependency
makes ort crashes. Further investigation showed that is because the dependency format is incompatible with Go's current standard. A workaround applied was to update
go.sum
andgo.mod
by runninggo mod tidy
CC: @arieltorti @dgutson