Closed jayschwa closed 5 years ago
Have you tried go mod why
?
The underlying cause here is almost certainly the same as https://github.com/golang/go/issues/26208, although per the description, not obviously so. Not least because time (and releases) have passed since that issue was reported.
There are a number of issues/factors at play here:
github.com/docker/docker
does not follow semver so we need to do a separate go get
for the latest version of thatgithub.com/docker/docker@v17.05.0-ce
(for example) is not a modulegithub.com/docker/docker@v17.05.0-ce
's dependencies are managed via vendor.conf
and are sufficiently esoteric (dependencies not using semver, etc) that we need to go mod init
and "merge" those dependencies into our main module's go.mod
github.com/Sirupsen/logrus
vs github.com/sirupsen/logrus
is a long standing issue. GitHub redirects github.com/Sirupsen/logrus
to github.com/sirupsen/logrus
github.com/sirupsen/logrus
/github.com/Sirupsen/logrus
is now a module, github.com/sirupsen/logrus
. Hence in module mode:$ go get github.com/Sirupsen/logrus
go: github.com/Sirupsen/logrus@v1.1.1: parsing go.mod: unexpected module path "github.com/sirupsen/logrus"
github.com/Sirupsen/logrus
is used as such consistently in github.com/docker/docker@v17.05.0-ce
and all its dependencies. This means that any use of github.com/sirupsen/logrus
in module mode is a problem (case-insensitive import collision: "github.com/sirupsen/logrus" and "github.com/Sirupsen/logrus"
) but also that we need to use github.com/Sirupsen/logrus
(sic) prior to v1.1.0 when it became the module github.com/sirupsen/logrus
Hence we end up needing to do something like the following (notice we get the pre v1.1.0 version of
github.com/Sirupsen/logrus
/github.com/sirupsen/logrus
via the gomodmerge
):
https://gist.github.com/myitcv/f7270ab81ab45aa286f496264f034b56
To my mind, the v1.1.0 module-enabled release of github.com/Sirupsen/logrus
/github.com/sirupsen/logrus
is a breaking change; because an import path of github.com/Sirupsen/logrus
now no longer works when in module mode (the irony). Hence I think the module release of github.com/Sirupsen/logrus
/github.com/sirupsen/logrus
should in fact have been a v2 release.
So one potential way forward here is for github.com/Sirupsen/logrus
/github.com/sirupsen/logrus
to create a new v1 release that reverts the conversion to Go modules and instead release github.com/sirupsen/logrus/v2
as a module.
cc @bcmills and @rsc for any additional thoughts here.
Have you tried
go mod why
?
It produces the same error with no additional info.
github.com/docker/docker
does not follow semver so we need to do a separatego get
for the latest version of that
I should note that I am using a very recent pseudo-version of docker in my example, and the problem still occurs: github.com/docker/docker v0.7.3-0.20181027010111-b8e87cfdad8d
Ideally, docker should start using go.mod
and tag its repo correctly, but in the meantime, it would be nice to get more details from the error message so these sorts of problems are easier to diagnose. Nothing in the error message indicates that the conflict is coming from docker or one of its dependencies.
I should note that I am using a very recent pseudo-version of docker in my example, and the problem still occurs: github.com/docker/docker v0.7.3-0.20181027010111-b8e87cfdad8d
Indeed, but the issues I listed above are independent of the version of Docker (unless, at some point recently or in the future, Docker and all its dependencies switch to using github.com/sirupsen/logrus
as the import path).
Ideally, docker should start using go.mod and tag its repo correctly,
This will help certainly with the "dance" required to get sensible versions of Docker's dependencies, yes.
But that leaves the github.com/Sirupsen/logrus
/github.com/sirupsen/logrus
issue:
it would be nice to get more details from the error message so these sorts of problems are easier to diagnose
I think this is a particularly subtle edge case. Whilst in general I absolutely agree with the sentiment, I don't know if there's much we can do to generally improve what is likely to be an extremely unlikely confluence of events. But that's just my two cents.
To my mind, the v1.1.0 module-enabled release of
github.com/Sirupsen/logrus
/github.com/sirupsen/logrus
is a breaking change; because an import path ofgithub.com/Sirupsen/logrus
now no longer works when in module mode (the irony).
If you believe that github.com/Sirupsen/logrus
was ever a valid import path, then that is correct. 🙂
(At some point we should consider whether we can make import paths case-insensitive, but that's definitely not happening in 1.12.)
A fix for https://github.com/golang/go/issues/26904#issuecomment-411873481 would allow you to at least replace github.com/Sirupsen/logrus => github.com/sirupsen/logrus
, but that opens a whole other can of case-sensitivity worms.
So one potential way forward here is for
github.com/Sirupsen/logrus
/github.com/sirupsen/logrus
to create a new v1 release that reverts the conversion to Go modules and instead releasegithub.com/sirupsen/logrus/v2
as a module.
I think it's actually fine to keep the change in v1
. Code that uses the “wrong” import path for v1
in is already broken in general anyway: if you try to import the same module with two different cases, you'll get a case-sensitive import collision (https://github.com/golang/go/issues/26208#issuecomment-411955266), so if we allow modules that depend on each path individually build successfully, they still can't be combined: they aren't really “modular”.
It looks like people are working hard to get this fixed correctly. Until then, is there a TL;DR workaround I can use?
@maguro, a TL;DR workaround to what? The issue reported here is a less-than-helpful error message, but presumably the thing you want to work around is some error itself⸮
@maguro in my case I rename all "Sirupsen" to "sirupsen" from my dependencies
I think the problem here is that we're emitting the error in the wrong place.
If we fetch a module with some casing of its path, and the module declaration indicates a different casing, that's not a defect in the module — it's a defect in whatever package is trying to import from that module, and should be diagnosed within that package.
@bcmills
If we fetch a module with some casing of its path, and the module declaration indicates a different casing, that's not a defect in the module — it's a defect in whatever package is trying to import from that module, and should be diagnosed within that package.
I think a better error would go a long way.
I think this is the same issue with docker/libcompose
, where go mod tidy
and go mod why
were both erroring out prior to saying where the conflict came from. In other words, this error (from using docker/libcompose
) doesn't tell you the real culprit:
go: github.com/Sirupsen/logrus@v1.2.0: parsing go.mod: unexpected module path "github.com/sirupsen/logrus"
It seems like there is already a reproduction here, but here is a simple one showing go mod tidy
fail for a libcompose
client:
CC @vito-c
In https://github.com/golang/go/issues/28489#issuecomment-434616301 above, @myitcv has a very nice explanation of the somewhat common issue one can hit with docker with a uppercase/lowercase mismatch between github.com/Sirupsen/logrus
vs. github.com/sirupsen/logrus
, and he also has a worked example there.
The starting point for that worked example above happens to be a docker version from ~18 months ago (in part because of the way docker has been tagging their releases and moving repos around, as touched upon above), and the worked example above ends up with github.com/Sirupsen/logrus
(the uppercase version) because that worked example shows finding a consistent set of older versions.
Here is a worked example using a more recent docker version, and ends up with github.com/sirupsen/logrus
(the lowercase version), using a simple docker/libcompose
client that initially fails with:
go: github.com/Sirupsen/logrus@v1.2.0: parsing go.mod:
unexpected module path "github.com/sirupsen/logrus"
The following FAQ covers the technique in a bit more detail:
First, we clone libcompose
in order to run go mod init
to convert from the vendor.conf
from libcompose
to a temporary go.mod
, and then use those results to populate the require
directives for our client. Here is a worked example of doing that (with a 2018 version of libcompose and docker):
There is some discussion above about making the error friendlier, including to output the parent of the improperly specified dependency (in order to reduce need to then hunt with go mod graph
, or similar).
In addition to that, the current error likely could do a better job describing the issue.
As an example, the repo history of docker is a bit confusing, but if you look at github/moby/moby
for example, there is a deliberately added "import comment" to enforce people always use a single, canonical import starting with github/docker/docker
(and not moby/moby
). Sample example here:
https://github.com/moby/moby/blame/master/client/client.go#L42
Here is current / pre-modules error you get if you try to go get github.com/moby/moby/client
:
$ mkdir -p $GOPATH/example.com/scratchpad/mobytest
$ cd $GOPATH/example.com/scratchpad/mobytest
$ go get github.com/moby/moby/client
package github.com/moby/moby/client: code in directory
...\go\src\github.com\moby\moby\client expects import "github.com/docker/docker/client"
(That's the "old" error triggered by the enforcement of the import comment).
Here's a "simulated" error message IF they had enforced the same thing with a go.mod
:
go: github.com/moby/moby@latest: parsing go.mod:
unexpected module path "github.com/docker/docker"
I think the old error is simpler to understand, including the fact that it states what the code "expects" (probably making it simpler to understand than something that is just "unexpected" in the newer error).
Change https://golang.org/cl/158477 mentions this issue: cmd/go: update unexpected module import error to be more actionable
I just raised the above CL to hopefully improve this error message. I'm looking to become one of the maintainers of logrus
, so improving this is of personal interest to me because of the repo migration we're going to try and do to accomplish this[1].
[1] https://github.com/sirupsen/logrus/issues/799#issuecomment-454910992
@bcmills to your earlier comment about what the real issue may be[1], I don't think my CL is addressing that as the fix. It's only focused on the value of the error string, and not the location it's being emitted.
I marked my CL as a fix for this issue, only because to me it felt like reworking where the error was emitted was maybe a separate (larger) task. This could be a way to improve the error for 1.12, and we could file a separate issue (or rename this one) to speak of a more complete fix.
[1] https://github.com/golang/go/issues/28489#issuecomment-449420638
I marked my CL as a fix for this issue
@theckman, I think your CL makes the error message clearer, and I hope it gets merged. However, discovering where the mismatch is happening in the dependency graph was kind of the crux of this issue when I created it. I don't think it would be accurate to call the CL a fix in its current form. It's also not clear to me what is gained by re-purposing this issue to fit the CL, closing it, and then creating a new issue for the same thing.
@jayschwa ta; adjusted commit to align with that.
@jayconrod Is the thinking that your WIP https://golang.org/cl/166984 and #30661 would make the error message here much more actionable by making it easy to see what piece of code is importing via the "wrong" import path (e.g., who is importing github.com/Sirupsen/logrus
when the logrus go.mod
declares itself to be github.com/sirupsen/logrus
, or who is misusing github.com/golang/lint
vs. golang.org/x/lint
, etc.)?
Side note: when you have something functional, I suspect your colleagues would be very interested in trying it out and giving you early feedback as they work through these types of issues. (CC @dmitshur @broady @jadekler).
CL 166984 should help with build and list commands. It produces output like this (from the test):
go: example.com/badchain/a@v1.0.0
-> example.com/badchain/b@v1.0.0
-> example.com/badchain/c@v1.0.0: parsing go.mod: unexpected module path "example.com/badchain/wrong"
That at least tells you which module has the unexpected import.
Unfortunately, this doesn't produce intelligible output for go get -u
because we don't preserve the relationships between modules when upgrading the build list. More refactoring needed there.
I spent much of this morning being confused by this. I was eventually untangled by #modules on slack, but I think my experience represents an underlying problem in the complexity and required knowledge to use modules. (And huge shout outs to @thepudds and @johanbrandhorst for their patience and help)
I maintain https://github.com/kolide/launcher/ it's a mid sized project that has some dependancies that ultimately require [sS]irupsen/logrus
. Parts of the dependency chain hadn't been updated in a while, some some of the underlying case stuff had been resolved, but not in my go.mod. I also have several internal projects that use launcher. https://github.com/directionless/go-issue-28489 is a fine example.
When I went to update launcher, I used go get -u github.com/kolide/launcher
. This resulted in an error about
go: github.com/Sirupsen/logrus@v1.4.1: parsing go.mod: unexpected module path "github.com/sirupsen/logrus"
But I could not tell where that dependency came from. From that example repo, it's not in the go mod why
output, and the go mod graph
output seems misleading WRT indirect dependancies.
After a fair bit of frustrating, and coaxing from slack I discovered the following:
go mod why
will show the dependency chaingo get github.com/kolide/launcher
would have avoided all of this.I feel like there are several issues.
The most obvious one is that I didn't need the -u
. I can upgrade my dependency with go get github.com/kolide/launcher
. I'm not sure why I internalized that -u
is needed. Probably because I think of u for upgrade. Possible akin to https://github.com/golang/go/issues/26581 there's a big footprint of tooling to learn.
The second one, is that if I'm running a command that tells me about an issue with a module, I expect to be able to understand why that module is in play. In retrospect, it's obviously because of the -u
, but I didn't realize that. If the error message could explicitly mention that it was being upgraded because of -u
that would be a big hint.
The third one, is that unexpected module path
isn't really clear. I did notice the case difference, but it took some googling before I realized it was significant. And my initial attempts at fixing it with go get
or editing the go.mod file didn't help.
For general searching, here's a somewhat simulated transcript:
$ go get -u github.com/kolide/launcher
go: finding github.com/kolide/launcher latest
go: finding github.com/hailocab/go-hostpool latest
go: finding github.com/bmizerany/assert latest
go: finding github.com/bitly/go-hostpool latest
go: finding github.com/Shopify/logrus-bugsnag latest
go: finding github.com/kolide/osquery-go latest
go: finding github.com/kolide/updater latest
go: finding github.com/jinzhu/inflection latest
go: finding github.com/mixer/clock latest
go: finding github.com/kolide/kit latest
go: finding golang.org/x/sync latest
go: finding golang.org/x/time latest
go: finding golang.org/x/image latest
go: finding golang.org/x/sys latest
go: finding github.com/alexkohler/nakedret latest
go: finding github.com/WatchBeam/clock latest
go: finding github.com/cloudflare/cfssl latest
go: finding github.com/kr/logfmt latest
go: finding github.com/agl/ed25519 latest
go: github.com/Sirupsen/logrus@v1.4.1: parsing go.mod: unexpected module path "github.com/sirupsen/logrus"
go: finding github.com/serenize/snaker latest
go: finding github.com/groob/plist latest
go: finding golang.org/x/net latest
go get: error loading module requirements
# where is this coming from?
$ go mod why github.com/Sirupsen/logrus
# github.com/Sirupsen/logrus
(main module does not need package github.com/Sirupsen/logrus)
# where is this coming from?
$ go mod why github.com/sirupsen/logrus
# github.com/sirupsen/logrus
(main module does not need package github.com/sirupsen/logrus)
# WTF? Why is this problem?
# get slack help
$ go get github.com/kolide/launcher
go: finding github.com/kolide/launcher latest
(Along with an associated go get github.com/docker/distribution
in my launcher repo to fix the underlying dependency)
Related, as I work through the golang/lint rename (https://github.com/golang/go/issues/30831) I feel similarly.
I want to be able to run go get -u
and have it update stuff. But, that results in complaints about lint. So I'm chasing down individual dependancies.
Agree that the error message should be more clear and tell where it originates from. In my case, I get:
go: github.com/dedis/protobuf@v1.0.6: parsing go.mod: unexpected module path "go.dedis.ch/protobuf"
But when I do go mod graph
, I get:
As you can see, the mentioned module isn't even a dependency... Sure, I played around with that module somewhere else but not in combination with that repo at all. (It doesn't show up on go list -u -m all
either). It leaves me with no clue what I can do about the error and why it shows up.
ref: https://github.com/tendermint/tendermint/pull/3613
(go version go1.12.5 darwin/amd64)
@Liamsi There has been a change in Go tip to make the output better, see https://github.com/golang/go/issues/28489#issuecomment-473297974 above. You can try Go tip via go get golang.org/dl/gotip
. (h/t to @FiloSottile for making it!)
Awesome, will check that out! Thank you! Update: That change actually resolved the issue I had. Also, thanks for the reminder for gotip.
@dmitshur it seems that when one of the module is in the private github repo, gotip
still tries to fetch some info from sum.golang.org
with a result of "410 Gone" error:
https://sum.golang.org/lookup/github.com/<PRIVATE>/<MODULE>@v0.0.0-xxxxxxxxxxxxxx-xxxxxxxxx: 410 Gone
GOPROXY
is unset.
@inliquid That sounds related to a recent change on tip (see https://groups.google.com/d/msg/golang-dev/4Kw_OfGa7cc/rTapejrQBQAJ) and not this issue. That post has information on how to disable the mirror and checksum database for non-public modules on tip.
@dmitshur thanks!
I have a very similar issue with GO 1.12.5 I get: go: github.[enterprise server name]/[my team name]/[my-module-name]@v1.0.0: parsing go.mod: unexpected module path "my-module-name" (The name of the module is with hyphens, don't know if this is relevant)
This happens when I run any of go get/go mod why/go mod tidy/go mod verify in the module that wants my-module-name as a dependency.
Is there any way to get more information on what went wrong and where the problem actually is?
@BorisKozo Is this portion of the error missing the hostname and full module path:
parsing go.mod: unexpected module path "my-module-name"
If not, the issue might be someone has an import statement like:
import "your-module-name"
when it perhaps it should be something more like
import "github.[enterprise server name]/[your team name]/[your-module-name]"
If it is not obvious where, you could try with the tip version of Go:
go get golang.org/dl/gotip && gotip download
gotip get <foo>
which in some cases will have a better error message.
@thepudds It wasn't the issue but your comment sent me in the right direction so thank you for that! The problem was that when doing "go mod init" I did
go mod init [my-module-name]
instead of
go mod init github.[enterprise server name]/[my team name]/[my-module-name]
which wrote the partial name into the go.mod file.
I wish the error would be clearer though.
Change https://golang.org/cl/185985 mentions this issue: cmd/go: clarify error text for module path mismatch
Change https://golang.org/cl/186377 mentions this issue: cmd/go: tweak wording of module path mismatch error message
if anyone encounters this issue again, a simple solution is to add the following to your go.mod
file:
replace github.com/Sirupsen/logrus v1.4.2 => github.com/sirupsen/logrus v1.4.2
Replace the version with whatever you need to use.
I just hit this issue, but the above solution did not work for me. What did work was the solution in this very helpful comment by @kolyshkin: https://github.com/moby/moby/issues/39302#issuecomment-504146736
For me, that meant the following:
main.go (or whatever you're using)
package main
import (
"github.com/docker/docker/api/types"
"github.com/docker/engine/client"
log "github.com/sirupsen/logrus"
)
...
go.mod
module <your-module>
go 1.12
require (
...
// This was auto-populated by running `go build`
github.com/docker/engine v0.0.0-20190822205725-ed20165a37b4
// Also auto-populated
github.com/sirupsen/logrus v1.4.2
...
)
// Put this in by hand, whatever version you want to use
replace github.com/docker/docker => github.com/docker/engine v19.03.0
// rewritten as follows after running `go build`
// replace github.com/docker/docker => github.com/docker/engine v0.0.0-20190717161051-705d9623b7c1
cc @leelynne and @lynncyrin
And I was just shown another way around this problem by a colleague (dropping this here for other poor souls who got stuck):
After this the problems disappeared.
Main issue were apparently old some old/unused deps causing the clash.
What version of Go are you using (
go version
)?Go 1.11.1
Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (
go env
)?Linux, amd64
Problem
Our
go.mod
has the following dependency, along with many other dependencies:When running
go mod vendor
, I get the following error:I suspect that some deeper indirect dependency is also using logrus, but with the incorrect (old) capital S. However, the error message isn't giving me enough detail to track it down. It would be nice if the tool gave me a list of the modules or packages that are using the conflicting import path.
Example
https://github.com/jayschwa/go28489