Support dep

[bits01]

Just like it has an option for govendor and godeps, it should also support dep which is now considered production ready. Thank you.

[dmitshur]

This is a reasonable feature request. Thanks. /cc @sdboyer FYI.

I'll look into implementing support for dep.

@bits01, do you have an example repository that uses dep that I could use to test the functionality? Actually, never mind, I think I'll be able to use @bradleyfalzon's gopherci project (after https://github.com/bradleyfalzon/gopherci/pull/102).

[bradleyfalzon]

Nice! Also https://github.com/bradleyfalzon/graphql-mergeable commits the vendor folder if you needed that case.

[sdboyer]

ahh, yes! this was one of the many conversations i never quite got to finishing at gophercon 😄

i'm gonna cc @darkowlzz here, maintainer of dep status. while there are likely a number of possible ways to attack this problem, at least one should be through the information dep status ought to provide. that's something we're just getting around to working on now.

the other obvious option is for this project to rely on dep/gps directly. unfortunately, that isn't an option until we get around to re-exporting gps, which currently lives under an internal directory. and even then, it may be a bit rough going, as the API is still quite unstable. plus, doing that would likely be pretty duplicative of work already done here, inasmuch as this project would probably have to define its own cache area for repositories, separate from dep's.

because dep has a strong opinion about what "upgrade" means - and one that's almost certainly different from what this repo defines - that's probably a reasonable place to start discussions.

dep has four different version "types" - branches, semver tags, non-semver tags, and revisions. these map onto all four of the underlying vcs. each type has its own meaning of "upgrade":

1. branches are expected to move, and dep generally looks on any branch movement as constituting an "upgrade," even if was a move backwards chronologically or topologically (to use git concepts).
2. semver, having the property of defining a partial order across valid semver numbers, has a clear definition for "upgrades." however, a distinction needs to be made between there being an upgrade available, and an upgrade that's available within current Gopkg.toml constraints.
3. plain tags are tags that don't conform to semver. they have no defined relationship with any other tag, and as such cannot define any upgrade path.
4. revisions, in this regard, have the same properties as plain tags.

[dmitshur]

Thanks for the relevant information @sdboyer!

My initial thoughts/plan for which updates to show is those that are within current Gopkg.toml constraints.

That seems like a good, easy, and useful first milestone to hit.

[sdboyer]

My initial thoughts/plan for which updates to show is those that are within current Gopkg.toml constraints.

cool, i definitely agree this is simplest (for the user), and therefore the place to start.

[dmitshur]

Made some progress on this, here's what it's looking like.

I've implemented a naive version as a starting point. It loads the Gopkg.lock file and uses that as initial dependency versions. Implementing gps.Updater interface is quite trivial by deferring to the following command:

dep ensure -update {Root}

However, I've realized I will need to make additional changes to Go Package Store to fully support dep.

Namely, right now Go Package Store supports figuring out the current package versions from your GOPATH, from a Godeps.json and govendor.json files, etc. However, from there onwards, it takes on the work of figuring out the latest versions by querying the repository remotes.

This will not work for dep because I need to let dep figure out the latest available updates that are within Gopkg.toml constraints. Otherwise, it ends up displaying all updates possible while ignoring all the constraints in Gopkg.toml. I obviously don't want to reimplement all the constraint checking myself.

My current thinking is making the component that figures out latest available versions an interchangeable component with two implementations: the current remote-based one, and a dep-constraint-based one.

[dmitshur]

As an observation, in the future more advanced versions of this feature, it would be useful for Go Package Store users to be able to see both the updates that are available to them within the Gopkg.toml constraints, and those that are available if the constraints are loosened.

So a future implementation might want to implement constraint checking locally.

After looking more at dep source code, I see there are a lot of similarities in what dep does to figure out the available updates when running dep ensure -update, and what Go Package Store does internally to figure out available updates. I will want to be careful to avoid reimplementing same functionality, and in the future, ideally, work towards reducing overlap.

For now, my next step idea towards having correct behavior is to use dep status -json command (ideally with -old flag, but it doesn't seem to be implemented) to have dep figure out the latest versions within the constraints of Gopkg.toml, and simply display those. That means a large chunk of Go Package Store functionality (figuring out latest remote versions) will be unused as we defer to dep to do that work for us.

But, I can't (and shouldn't) easily use dep's code to do this because it's in an internal package (github.com/golang/dep/internal/dep). I'm sure its internal API will change pretty often, so using dep the binary API will be much easier for now. In the future, once things are more stable, using native Go code to replace shelling out to dep binary is very viable.

I'll have an initial version for people to play with on their dep-enabled projects soon.

[bits01]

Thanks for making progress on this! I agree that it is useful to be able to see what's available subject to the dep constraints but also without any dep constraints.

[dmitshur]

That is going to be left out as a future feature. For now, Go Package Store will only display updates within Gopkg.toml constraints. You can always temporarily modify your Gopkg.toml to lift any constraints and run Go Package Store to see unconstrained updates.

I will need to think about frontend and presentation changes to accommodate that additional information (and actions) well.

Created PR #82 that'll resolve this. Can someone who has a dep-enabled project try it out and let me know your experience? I've tried it on github.com/bradfitz/gopherci only so far.

[bradleyfalzon]

Can someone who has a dep-enabled project try it out and let me know your experience?

I've tried it successfully on bradleyfalzon/addgeoip/Gopkg.toml, bradleyfalzon/maintainer.me/Gopkg.toml and bradleyfalzon/graphql-mergeable/Gopkg.toml.

Behaved as expected.