search

golang / mock

GoMock is a mocking framework for the Go programming language.
Apache License 2.0
9.3k stars 611 forks source link

go generate fails to mockgen on go 1.14 #415

Closed Callisto13 closed 4 years ago

Callisto13 commented 4 years ago

Actual behavior.

On go 1.14.

Running go generate ./... when the vendor dir is present fails with:

package github.com/golang/mock/mockgen: cannot find package "." in:
        /Users/callisto/workspace/cretaceous/vendor/github.com/golang/mock/mockgen
mocks/doc.go:1: running "go": exit status 1

(I am guessing this is because mockgen depends on some runtime thing being there which would not be included when vendoring.)

We saw this appear after bumping to go 1.14. On go 1.13.x go generate ran fine.

Expected behavior

My mocks should be generated and no error should occur.

To Reproduce

I have set up a dumb example repo which mimics how we generate mocks in our codebase.

Silly interfaces found in dinosaur/dinosaur.go. Mocks should be generated in mocks/mock_dino.go.

To create mocks we keep a doc.go file in mocks/ and set the mocks to be generated in that package using the format:

//go:generate go run github.com/golang/mock/mockgen -package mocks -destination=./mock_dinos.go -source=../dinosaur/dinosaur.go
  1. Ensure you are running go 1.14
  2. git clone https://github.com/Callisto13/cretaceous && cd cretaceous
  3. Note that the vendor dir is not present.
  4. Observe that go generate ./... works happily.
  5. go mod vendor
  6. Observe that go generate ./... fails with the error noted above.
  7. Remove the vendor dir and run through again for lolz.
  8. Do the exact same with go 1.13.x and see that everything works fine with the vendor dir present.

The behaviour is seen on all my team's machines.

Additional Information

Triage Notes for the Maintainers

codyoss commented 4 years ago

Please update your generate command to look like this: //go:generate go run github.com/golang/mock/mockgen --build_flags=--mod=vendor -package mocks -destination=./mock_dinos.go -source=../dinosaur/dinosaur.go

Notable added: --build_flags=--mod=vendor.

That did the trick for me /w the latest mockgen and 1.14 :smile:

Callisto13 commented 4 years ago

Afraid that doesn't work for me @codyoss, running go generate ./... (after go mod vendor) still results in:

package github.com/golang/mock/mockgen: cannot find package "." in:
        /Users/callisto/workspace/cretaceous/vendor/github.com/golang/mock/mockgen
mocks/doc.go:1: running "go": exit status 1

sorry, I should have mentioned that --build_flags=--mod=vendor was the first thing I tried.

maelvls commented 4 years ago

Thank you for the minimal working example @Callisto13!! Helped me a lot when I tried to reproduce the bug.

TL;DR: Go 1.13 and below would still pick up from the pkg/mod cache even when vendor/ was there. With Go 1.14, any Go command automatically uses vendor/. And since github.com/golang/mock/mockgen is not imported anywhere, go mod vendor doesn't vendor it and go generate ./... fails 😔 A workaround is to add this somewhere:

import (
    _ "github.com/golang/mock/mockgen/model"
)

After some investigation, I think that what happens is that when you run go mod vendor it doesn't vendor github.com/golang/mock/mockgen since this import path isn't imported anywhere.

Let's see step by step

  1. git clone https://github.com/Callisto13/cretaceous && cd cretaceous

  2. go generate ./... succeeds

  3. now let's vendor stuff

    % go mod vendor
% git status
Untracked files:
 (use "git add <file>..." to include in what will be committed)
    vendor/github.com/golang/mock/AUTHORS
    vendor/github.com/golang/mock/CONTRIBUTORS
    vendor/github.com/golang/mock/LICENSE
    vendor/github.com/golang/mock/gomock/call.go
    vendor/github.com/golang/mock/gomock/callset.go
    vendor/github.com/golang/mock/gomock/controller.go
    vendor/github.com/golang/mock/gomock/matchers.go
    vendor/modules.txt

  4. at this point go generate ./... fails:

    % go generate ./...
package github.com/golang/mock/mockgen: cannot find package "." in:
     /Users/mvalais/code/ori/cretaceous/vendor/github.com/golang/mock/mockgen
mocks/doc.go:1: running "go": exit status 1

  5. now let's add a fake import so that github.com/golang/mock/mockgen gets vendored:

    diff --git a/mocks/doc.go b/mocks/doc.go
index ac383c5..c30949b 100644
--- a/mocks/doc.go
+++ b/mocks/doc.go
@@ -1,3 +1,7 @@
//go:generate go run github.com/golang/mock/mockgen -package mocks -destination=./mock_dinos.go -source=../dinosaur/dinosaur.go

package mocks
+
+import (
+    _ "github.com/golang/mock/mockgen"
+)

  6. let's vendor and see what gets added:

    % go mod vendor
% git status
Untracked files:
 (use "git add <file>..." to include in what will be committed)
    vendor/github.com/golang/mock/AUTHORS
    vendor/github.com/golang/mock/CONTRIBUTORS
    vendor/github.com/golang/mock/LICENSE
    vendor/github.com/golang/mock/gomock/call.go
    vendor/github.com/golang/mock/gomock/callset.go
    vendor/github.com/golang/mock/gomock/controller.go
    vendor/github.com/golang/mock/gomock/matchers.go
    vendor/github.com/golang/mock/mockgen/mockgen.go
    vendor/github.com/golang/mock/mockgen/model/model.go
    vendor/github.com/golang/mock/mockgen/parse.go
    vendor/github.com/golang/mock/mockgen/reflect.go
    vendor/github.com/golang/mock/mockgen/version.1.11.go
    vendor/github.com/golang/mock/mockgen/version.1.12.go
    vendor/golang.org/x/tools/AUTHORS
    vendor/golang.org/x/tools/CONTRIBUTORS
    vendor/golang.org/x/tools/LICENSE
    vendor/golang.org/x/tools/PATENTS
    vendor/golang.org/x/tools/go/gcexportdata/gcexportdata.go
    vendor/golang.org/x/tools/go/gcexportdata/importer.go
    vendor/golang.org/x/tools/go/internal/gcimporter/bexport.go
    vendor/golang.org/x/tools/go/internal/gcimporter/bimport.go
    vendor/golang.org/x/tools/go/internal/gcimporter/exportdata.go
    vendor/golang.org/x/tools/go/internal/gcimporter/gcimporter.go
    vendor/golang.org/x/tools/go/internal/gcimporter/iexport.go
    vendor/golang.org/x/tools/go/internal/gcimporter/iimport.go
    vendor/golang.org/x/tools/go/internal/gcimporter/newInterface10.go
    vendor/golang.org/x/tools/go/internal/gcimporter/newInterface11.go
    vendor/golang.org/x/tools/go/internal/packagesdriver/sizes.go
    vendor/golang.org/x/tools/go/packages/doc.go
    vendor/golang.org/x/tools/go/packages/external.go
    vendor/golang.org/x/tools/go/packages/golist.go
    vendor/golang.org/x/tools/go/packages/golist_overlay.go
    vendor/golang.org/x/tools/go/packages/packages.go
    vendor/golang.org/x/tools/go/packages/visit.go
    vendor/golang.org/x/tools/internal/fastwalk/fastwalk.go
    vendor/golang.org/x/tools/internal/fastwalk/fastwalk_dirent_fileno.go
    vendor/golang.org/x/tools/internal/fastwalk/fastwalk_dirent_ino.go
    vendor/golang.org/x/tools/internal/fastwalk/fastwalk_dirent_namlen_bsd.go
    vendor/golang.org/x/tools/internal/fastwalk/fastwalk_dirent_namlen_linux.go
    vendor/golang.org/x/tools/internal/fastwalk/fastwalk_portable.go
    vendor/golang.org/x/tools/internal/fastwalk/fastwalk_unix.go
    vendor/golang.org/x/tools/internal/gopathwalk/walk.go
    vendor/golang.org/x/tools/internal/semver/semver.go
    vendor/modules.txt

  7. now it works but compiling fails (see next comment from @Callisto13)

    % go generate ./...
# Works!
% go test ./...
mocks/doc.go:6:2: import "github.com/golang/mock/mockgen" is a program, not an importable package

  8. as @Callisto13 proposed in the next comment, we can import mockgen/model instead (which is the only non-main module!)

    import (
_ "github.com/golang/mock/mockgen/model"
)

Quite an unpleasant workaround I must admit 🙄

Callisto13 commented 4 years ago

Thanks @maelvls!

I experimented with this last week and came to almost the same conclusion.

Unfortunately importing _ "github.com/golang/mock/mockgen" leads to a compilation failure:

import "github.com/golang/mock/mockgen" is a program, not an importable package

Having any fake import is a rather unpleasant hack so I am hoping the folks at gomock can suggest something less awful.

codyoss commented 4 years ago

I am not sure I have a great solution for this. To get the 1.13 behavior I think you have use a -mod=mod but I have not had a chance to test this: https://golang.org/doc/go1.14#go-command

You have to pick if you want the go tooling to properly respect the vendor folder, only on partially. I am thinking the former is what most people will want.

With modules I think people are going to find having something like a tool.go file, where there are some underscore imports quite helpful, for reproducible builds. I don't think this is a bad pattern to use. This is nice to make sure things like goimports is in your CI system. Here is an example from another project I work on: https://github.com/googleapis/google-cloud-go/blob/master/tools.go

maelvls commented 4 years ago

I just tried the -mod=mod solution, it works! For anyone to try,

  1. git clone https://github.com/Callisto13/cretaceous && cd cretaceous
  2. apply this diff 
    diff --git a/mocks/doc.go b/mocks/doc.go
index ac383c5..204c82e 100644
--- a/mocks/doc.go
+++ b/mocks/doc.go
@@ -1,3 +1,3 @@
-//go:generate go run github.com/golang/mock/mockgen -package mocks -destination=./mock_dinos.go -source=../dinosaur/dinosaur.go
+//go:generate go run -mod=mod github.com/golang/mock/mockgen -package mocks -destination=./mock_dinos.go -source=../dinosaur/dinosaur.go -build_flags=-mod=mod

    Note that there are two -mod=mod laying around: one for the go run, one passed with -build_flags that is used for subsequent go calls that mockgen does

  3. go mod vendor
  4. go generate ./... should work!
Callisto13 commented 4 years ago

🎉 thanks @codyoss and @maelvls