Closed rogpeppe closed 5 years ago
I had this issue with a message about finding github.com/google/martian
(an indirect dependency via cloud.google.com/go/storage.test
) every time, I assume because of some weirdness with its tag situation. Forcing a newer revision via go get github.com/google/martian@master
fixed it.
I think there's something to do with indirect dependencies not providing packages at the chosen module version, but that's just a wild guess.
github.com/google/martian
is weird: it has two valid prerelease version tags (v2.0.0-beta.2
and v2.0.0-beta
) from three years ago, one invalid tag (v1.0
), and no valid release-version tags, and in the three years since the prerelease tag it has gotten a lot of new packages.
Azure
is weird too, but I don't understand why yet. (See #27627.)
As another example of an "unstable" go mod tidy
(by "unstable" I mean that for two consecutive calls to go mod tidy
, all things being equal, the second should produce no output, else it is "unstable"):
https://gist.github.com/myitcv/65994e8083770e0a518c30cf815db47d
The first gist file shows that go mod tidy
is "unstable" on a v0.13.0
checkout of Buffalo.
The second gist file shows that go mod tidy
is "stable" where that same version of Buffalo is a dependency of another module.
What's interesting however is the contents of the go.mod
in that second gist:
module example.com/blah
require (
github.com/cockroachdb/apd v1.1.0 // indirect
github.com/cockroachdb/cockroach-go v0.0.0-20181001143604-e0a95dfd547c // indirect
github.com/gobuffalo/buffalo v0.13.0
github.com/gobuffalo/fizz v1.0.12 // indirect
github.com/jackc/fake v0.0.0-20150926172116-812a484cc733 // indirect
github.com/jackc/pgx v3.2.0+incompatible // indirect
github.com/jmoiron/sqlx v1.2.0 // indirect
github.com/satori/go.uuid v1.2.0 // indirect
github.com/shopspring/decimal v0.0.0-20180709203117-cd690d0c9e24 // indirect
)
Given Buffalo is already a module, I would only have expected a single require
directive, namely:
github.com/gobuffalo/buffalo v0.13.0
(that said, the initial go mod tidy
in the first gist suggests that go.mod
has not been fully tidied, so this expectation doesn't hold 100%. And that's before any issues that might be influencing what's going on here)
Perhaps the other contents of the go.mod
give some sort of clue as to what's going wrong here?
Here's some more oddness demonstrated with the buffalo example. If I create a dumb wrapper package as follows:
package main
import _ "github.com/gobuffalo/buffalo"
func main() {
}
Then:
go mod init example.com/foo
go mod tidy
go get github.com/gobuffalo/buffalo@v0.13.0
This created a go.mod
file with more entries than expected, as mentioned by @myitcv above:
module example.com/foo
require (
github.com/cockroachdb/apd v1.1.0 // indirect
github.com/cockroachdb/cockroach-go v0.0.0-20181001143604-e0a95dfd547c // indirect
github.com/gobuffalo/buffalo v0.13.0
github.com/gobuffalo/fizz v1.0.12 // indirect
github.com/jackc/fake v0.0.0-20150926172116-812a484cc733 // indirect
github.com/jackc/pgx v3.2.0+incompatible // indirect
github.com/satori/go.uuid v1.2.0 // indirect
github.com/shopspring/decimal v0.0.0-20180709203117-cd690d0c9e24 // indirect
)
Then I checked out github.com/gobuffalo/buffalo@v0.13.0 and went into that directory.
After running go mod tidy
there, one extra dependency was added (github.com/jmoiron/sqlx@v1.2.0) but even after that, some of the dependencies mentioned in the dumb wrapper weren't mentioned.
The results of go list -m all
that I'd expect to be very similar, were quite different. Here's the diff:
1c1
< example.com/foo
---
> github.com/gobuffalo/buffalo
5,6d4
< github.com/cockroachdb/apd v1.1.0
< github.com/cockroachdb/cockroach-go v0.0.0-20181001143604-e0a95dfd547c
14d11
< github.com/gobuffalo/buffalo v0.13.0
19d15
< github.com/gobuffalo/fizz v1.0.12
51,53c47
< github.com/jackc/fake v0.0.0-20150926172116-812a484cc733
< github.com/jackc/pgx v3.2.0+incompatible
< github.com/jmoiron/sqlx v0.0.0-20180614180643-0dae4fefe7c0
---
> github.com/jmoiron/sqlx v1.2.0
55d48
< github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51
82d74
< github.com/satori/go.uuid v1.2.0
85d76
< github.com/shopspring/decimal v0.0.0-20180709203117-cd690d0c9e24
109d99
< google.golang.org/appengine v1.2.0
To take one example, the github.com/gobuffalo/fizz
package seems to be included in the dumb wrapper package but not in the github.com/gobuffalo/buffalo
module, which is quite odd.
Further anomalies ensue:
% go list -m
github.com/gobuffalo/buffalo
% go mod why -m github.com/gobuffalo/fizz
# github.com/gobuffalo/fizz
(main module does not need module github.com/gobuffalo/fizz)
% go mod why github.com/gobuffalo/fizz
# github.com/gobuffalo/fizz
github.com/gobuffalo/buffalo
github.com/gobuffalo/pop
github.com/gobuffalo/fizz
go mod graph
includes this entry:
github.com/gobuffalo/buffalo-pop@v1.0.5 github.com/gobuffalo/pop@v4.8.3+incompatible
but then no further entries for the dependencies of github.com/gobuffalo/pop@v4.8.3+incompatible
, which does have several dependencies, including on github.com/gobuffalo/fizz
.
when we looked at something similar today it seems tidy is doing something similar to go get -u
which means if you're using go get -u=patch
go mod tidy
will break your repo.
@stevenh
it seems tidy is doing something similar to go get -u
Do you have a repro that shows go mod tidy
behaving like go get -u
?
Just linking to another go mod tidy
issue for reference: https://github.com/golang/go/issues/27868
Our reproduction case was quite simple:
github.com/influxdata/influxdb
go get -u=patch
you'll see it updates from 1.4.0 to 1.4.3go mod tidy
you'll see it updates to 1.6.4I'm trying to create a simple repo case as there may have been something else a play.
@stevenh I'm not seeing that:
$ export GOPATH=$(mktemp -d)
$ export PATH=$GOPATH/bin:$PATH
$ cd $(mktemp -d)
$ go mod init example.com/hello
go: creating new go.mod: module example.com/hello
$ cat <<EOD >main.go
package main
import _ "github.com/influxdata/influxdb"
func main() {}
EOD
$ go get github.com/influxdata/influxdb@v1.4.0
go: finding github.com/influxdata/influxdb v1.4.0
go: downloading github.com/influxdata/influxdb v1.4.0
$ go list -m all
example.com/hello
github.com/influxdata/influxdb v1.4.0
$ go get -u=patch
go: finding github.com/influxdata/influxdb v1.4.3
go: downloading github.com/influxdata/influxdb v1.4.3
$ go list -m all
example.com/hello
github.com/influxdata/influxdb v1.4.3
$ go mod tidy
$ go list -m all
example.com/hello
github.com/influxdata/influxdb v1.4.3
Do you have another dependency here which requires v1.6.3
? As a bit of a long shot, there's a chance you hadn't done anything that required that dependency (the one that requires v1.6.3
) and that go mod tidy
was the first time MVS actually ran across all dependencies. What does:
$ go mod graph | grep influx
give you?
Ok here's a set of repo steps from a blank directory:
mkdir wibble
cat <<EOT >wibble/test.go
package main
import (
"fmt"
"github.com/influxdata/influxdb/query"
)
func main() {
fmt.Println("go mod test")
}
EOT
go mod init
go get -u github.com/influxdata/influxdb@v1.3.9
grep influxdb go.mod
go get -u=patch
grep influxdb go.mod
go mod tidy
grep influxdb go.mod
The final grep here will show:
github.com/influxdata/influxdb v1.6.4
go mod graph | grep influx
github.com/multiplay/gomod github.com/influxdata/influxdb@v1.6.4
github.com/multiplay/gomod github.com/influxdata/influxql@v0.0.0-20180925231337-1cbfca8e56b6```
@stevenh that's happening because github.com/influxdata/influxdb/query
only came into existence in >= v1.4.0
. So the go
tool resorts to getting the latest
Interesting, so why doesn't the go get -u=patch
detect that and update the dependency?
I'll keep playing as the original issue we had on our main repo which it at 1.4.0, go get -u=patch
updated it to 1.4.3 but then something else forced it to 1.6.4
Thanks for investigating that, most appreciated.
@rogpeppe, can you still reproduce this? I notice that github.com/google/martian
added a well-formed tag (v2.1.0
), although I still haven't been able to dig into the Azure
issue.
@bcmills Yes, I just tried with the latest release and it still happens. Example:
% go mod tidy
go: finding github.com/masterzen/azure-sdk-for-go/core/http latest
go: finding github.com/masterzen/azure-sdk-for-go/core/tls latest
go: finding github.com/masterzen/azure-sdk-for-go/core latest
go: finding github.com/Azure/azure-sdk-for-go/arm/disk latest
go: finding github.com/Azure/azure-sdk-for-go/arm/network latest
go: finding github.com/Azure/azure-sdk-for-go/arm/storage latest
go: finding github.com/Azure/azure-sdk-for-go/arm/resources/resources latest
go: finding github.com/Azure/azure-sdk-for-go/arm/compute latest
go: finding github.com/Azure/azure-sdk-for-go/arm/authorization latest
go: finding github.com/Azure/azure-sdk-for-go/arm/resources/subscriptions latest
go: finding github.com/lxc/lxd latest
go: finding github.com/Azure/azure-sdk-for-go/arm latest
go: finding github.com/Azure/azure-sdk-for-go/arm/resources latest
I can't reproduce this from the HEAD
revision of github.com/juju/charmstore-client
because it seems to depend on a private repo, but I do see it at the v2.3.0
revision.
The Azure
thing is just some combination of #27063 and #27102. github.com/Azure/azure-sdk-for-go
removed the arm
subdirectory starting at v14.0.0
.
Same for github.com/lxc/lxd
: the top-level package in that repo was removed at lxd-2.17
, and for some reason they didn't consider that a big enough change to warrant a major version...
Same for go.opencensus.io/resource
. The resource
package was added on the master
branch, but is not present in the latest tagged version (v0.18.0
). (See also https://github.com/census-instrumentation/opencensus-go/issues/998.)
After running
go get -m go.opencensus.io@master github.com/lxc/lxd@lxd-2.16 github.com/Azure/azure-sdk-for-go@v12.5.0-beta
go mod tidy
is stable in github.com/juju/charmstore-client
at v2.3.0
.
Closing as a dup of #27063 and/or #27102. Thanks for the report!
go version devel +4e1b11e2c9 Tue Aug 21 14:08:55 2018 +0000 linux/amd64
While I expect log messages the first time running a command, I don't expect them to be printed the second time, after all information has already been found. However in some cases,
go mod tidy
prints somefinding ... latest
messages every time it's run.For example: