Go 1.11 Modules (vgo) vs dep

[MaerF0x0]

@sdboyer Thank you for your talk Jul 18 at the GoSF meetup!

Could we get an official tldr; about how to treat golang/dep in the mean time as go 1.11 modules becomes production worthy? Should new repos standardize on dep or vgo for example? What is your best advice you'd give to a new person who is just starting out in golang?

As well any links to a video of the talk?

[sdboyer]

hi, thank you! Unfortunately, the talk video got lost 😭 i'm going to be re-recording it as a screencast, though, including some epiphanies sparked by discussions after the talk. i think it'll be even better 😄

i realize that guidance on these sorts of questions has been in short supply in the recent months. There actually IS more coming now, though. To that end, my TODO list looks something like this:

1. Release v0.5.0
2. Rerecord screencast
3. Big Thing 1
4. Maybe Big Thing 2

Items 3 and 4 will make guidance clearer on questions of what to use, i think. And, frustrating as it is, it'll be clear in retrospect why we needed to get our ducks in a row. Right now, i can say that dep will absolutely be adapting to the new world, and it won't hurt you to continue using it.

[rsc]

This advice is at best misleading.

Once Go 1.11 is out, it really makes very little sense for a new project not already using dep to start using it. Dep has many, many problems - some of which Sam acknowledged in the talk - and you avoid all of them, avoid taking the time to master a system that is going away, and help make Go modules better by simply using Go modules from the start.

Kubernetes, for example, is planning to jump straight from godep to Go modules, skipping dep entirely. They actually spent quite a lot of effort trying to figure out how to migrate to dep and could not quite make it work.

If you're already using dep, then it makes sense to keep using it until you're ready to make the switch - no one likes forced upgrades after all - but I think the vast majority of dep users will be very happy with the conversion to Go modules, especially since everything is integrated more smoothly into the go command.

$ cd /tmp
$ git clone https://github.com/golang/dep
Cloning into 'dep'...
remote: Counting objects: 20554, done.    
...    
$ go mod -init
go: creating new go.mod: module github.com/golang/dep
go: copying requirements from Gopkg.lock
$ rm -rf vendor Gopkg.* # look ma no hands
$ go build ./...
$ go list -m all
github.com/golang/dep
github.com/Masterminds/semver v0.0.0-20180403015459-24642bd05731
github.com/Masterminds/vcs v1.11.1
github.com/armon/go-radix v0.0.0-20160115234725-4239b77079c7
github.com/boltdb/bolt v1.3.1
github.com/golang/protobuf v1.0.0
github.com/jmank88/nuts v0.3.0
github.com/nightlyone/lockfile v0.0.0-20170707060451-e83dc5e7bba0
github.com/pelletier/go-toml v1.2.0
github.com/pkg/errors v0.8.0
github.com/sdboyer/constext v0.0.0-20170321163424-836a14457353
golang.org/x/net v0.0.0-20170828231752-66aacef3dd8a
golang.org/x/sync v0.0.0-20170517211232-f52d1811a629
golang.org/x/sys v0.0.0-20170830134202-bb24a47a89ea
gopkg.in/yaml.v2 v2.0.0
$ go test -short ./...
ok      github.com/golang/dep   0.983s
ok      github.com/golang/dep/cmd/dep   4.461s
ok      github.com/golang/dep/gps   1.208s
?       github.com/golang/dep/gps/internal/pb   [no test files]
ok      github.com/golang/dep/gps/paths 0.025s
ok      github.com/golang/dep/gps/pkgtree   0.062s
ok      github.com/golang/dep/gps/verify    0.073s
ok      github.com/golang/dep/internal/feedback 0.074s
ok      github.com/golang/dep/internal/fs   0.059s
?       github.com/golang/dep/internal/importers    [no test files]
ok      github.com/golang/dep/internal/importers/base   13.199s
ok      github.com/golang/dep/internal/importers/glide  1.370s
ok      github.com/golang/dep/internal/importers/glock  0.675s
ok      github.com/golang/dep/internal/importers/godep  0.925s
ok      github.com/golang/dep/internal/importers/govend 0.683s
ok      github.com/golang/dep/internal/importers/govendor   1.083s
ok      github.com/golang/dep/internal/importers/gvt    2.253s
?       github.com/golang/dep/internal/importers/importertest   [no test files]
ok      github.com/golang/dep/internal/importers/vndr   0.671s
?       github.com/golang/dep/internal/test [no test files]
?       github.com/golang/dep/internal/test/integration [no test files]
$ 

[MaerF0x0]

@rsc Thanks for the update. I've been reading your blog posts and watched your SG keynote to try and get up to speed. I suppose August is quite close so one could reasonably go with tip but as you said yourself:

but for compatibility we will keep the ability to read today's go.mod 
files, just as vgo can also read requirement information from 
GLOCKFILE, Godeps/Godeps.json, Gopkg.lock, dependencies.tsv, 
glide.lock, vendor.conf, vendor.yml, vendor/manifest, and 
vendor/vendor.json files.

Couldn't an organization just choose the package file style they prefer? That is, with Gopkg.toml being the only encouraged alternative, the rest deprecated and slated for removal. Or is there a roadmap in which these other file types would also be removed from go mod?

Without knowing the intricacies this seems to be a sort of yarn vs npm style scenario.

[trashhalo]

Once Go 1.11 is out, it really makes very little sense for a new project not already using dep to start using it

but I think the vast majority of dep users will be very happy with the conversion to Go modules

@rsc I just went through this and its been a bit of a mess. Yes, yes I know experimental software. But go 1.11 // July 31 is coming very soon and go modules are still are still shaky. my team is considering going back to dep and revisiting in 6 months.

Issues we hit along the way -

1. We corrupted our go.mods https://github.com/golang/go/issues/26375
2. Proxy support broke due to incompatible protocol changes https://github.com/jfrog/jfrog-cli-go/issues/194 https://github.com/golang/go/issues/26503
3. We couldn't use go modules on linux https://github.com/golang/go/issues/26525 https://github.com/golang/go/issues/26501
4. Debugging these problems has been frustrating due to lack of helpful logging https://github.com/golang/go/issues/26501#issuecomment-406655517
5. Documentation is spread all over the place. When we were ramping up we were pieces together blog posts, github gists and official docs
6. The attitude on some of the go team is this is all okay because its marked as experimental. And as far as I know it maintains that label in go 1.11 https://github.com/golang/go/issues/26503#issuecomment-406615458 https://github.com/golang/go/issues/26503#issuecomment-406630413
7. It also doesnt inspire confidence in go modules when the team ships world breaking bug like go get not working on ubuntu 16 the fix doesn't include a test change to prevent this in the future. https://github.com/golang/go/issues/26501#issuecomment-407431340

2 was especially annoying because we just go get vgo in our ci pipeline. so one day builds started failing because latest commit of vgo had the protocol change.

---

These aren't fundamental problems with go modules. These are signs of a product that are not ready for launch and I would advise any team to stick with what you got till things firm up.

[dimroc]

@trashhalo Have you tried go modules recently? How has the released 1.11 version held up?

[rugwirobaker]

Are there more subcommands coming to go mod or the api has already stabilized??

[mvdan]

Dep was officially deprecated earlier this year, and the proposal to archive this repository was accepted. As such, I'm closing outstanding issues before archiving the repository. For any further comments, please use the proposal thread on the Go issue tracker. Thanks!