dep: officially deprecate the "dep" experiment

[daenney]

This is a request to officially deprecate the "dep" package manager for Go, and would resolve #35894.

Rationale

The Go ecosystem has moved towards Go Modules, introduced in Go 1.11 and considered stable and production ready since 1.14. Many projects have started migrating to Go Modules, or already completed that migration since Go 1.12 was released.

Despite the fact that the Go project and Go ecosystem have largely embraced modules, dep remains in use by people. Unfortunately most if not all dep maintainers have moved on from the project. The net result of this is though people do still submit issues to the issue tracker and the occasional PR, no help is offered or review is received. This is not a good experience for any member of the Go community.

Changes

I would like to propose the following:

• The PR from @ALTree on https://github.com/golang/dep/pull/2233 is merged, or language to that same effect is added to the README of dep
• Any outstanding issues and PRs are closed with a "this project is no longer maintained, so we are not able to answer your question or accept new changes", or something like this
• The dep repository is archived on GitHub, resulting in the repository becoming read-only. This still allows anyone to explore and learn from the code, even fork it and build a new community around it should they want to, without creating the illusion that this is still a maintained tool by the Go project

[rsc]

Thanks for filing this proposal. We have been leaving Dep status alone and letting the users decide when they move. I saw @ALTree's PR and was waiting to see if the Dep maintainers had any comments. Using the proposal process seems like a good way to decide what to do with that PR.

Here are some recent numbers I gathered, not specifically for this issue. I built a list of ~all the public GitHub projects with ≥ 1000 stars and also downloaded some basic metadata about the default branch of the repo. This lets us understand a little about the current state of popular projects, including popular Go projects.

Out of 23,531 projects, 1,033 have a .go file in the root directory. Of those 640/1,033 (62%) also have a go.mod file. There are another 237 which have a go.mod but no .go sources, presumably because their Go sources are in subdirectories. If we include those, we have 877/1,270 (69%). For comparison, 73/1,033 (7.1%) have dep's Gopkg.toml, and 19/1,033 (2%) have godep's Godeps directory. There is a small amount of overlap: 11 have both go.mod and Gopkg.toml, so Dep-only is 62/1,033 (6.0%).

Limiting to projects for which the HEAD commit happened in the past six months, 588/742 (80%) or 812/966 (84%) use modules. Dep is 40/742 (5.4%). Dep-only is 32/742 (4.3%).

Limiting to the past three months, 515/629 (82%) or 727/841 (86%) use modules. Dep is 37/629 (5.9%). Dep-only is 29/629 (4.6%); full list below in the toggle.

So it looks like there are still a few active Dep users, but most of the Go user base has migrated at this point. Like I said, we'll put this through the proposal process and get this into the minutes for this week. Thanks.

---

The 29 dep-only GitHub projects with ≥ 1000 stars and commits in 2020

| project | branch | last commit | | --- | --- | --- | | GetStream/vg | master | 2020-01-16T14:39:29Z | | GoBelieveIO/im_service | master | 2020-01-02T11:53:05Z | | GoogleContainerTools/container-diff | master | 2020-02-13T15:34:58Z | | Originate/git-town | master | 2020-02-29T16:43:17Z | | boyter/scc | master | 2020-03-23T01:11:08Z | | centrifugal/centrifugo | master | 2020-03-12T16:06:21Z | | cloud66-oss/habitus | master | 2020-02-05T17:10:28Z | | cloudflare/redoctober | master | 2020-01-17T18:03:38Z | | convox/rack | master | 2020-02-27T16:44:09Z | | docker/app | master | 2020-01-07T16:27:28Z | | esimov/triangle | master | 2020-02-10T10:33:31Z | | fortio/fortio | master | 2020-03-11T22:31:16Z | | gravitational/teleport | master | 2020-03-21T00:57:05Z | | gruntwork-io/terragrunt | master | 2020-03-19T10:03:22Z | | influxdata/kapacitor | master | 2020-01-21T18:42:59Z | | inlets/inlets | master | 2020-03-10T14:42:51Z | | juju/juju | develop | 2020-03-23T13:14:32Z | | loadimpact/k6 | master | 2020-03-23T12:15:06Z | | minishift/minishift | master | 2020-03-12T14:33:09Z | | play-with-docker/play-with-docker | master | 2020-02-26T02:37:46Z | | requilence/integram | master | 2020-01-14T09:00:29Z | | rfjakob/gocryptfs | master | 2020-02-29T20:52:59Z | | ro31337/libretaxi | master | 2020-02-27T22:55:33Z | | shirou/gopsutil | master | 2020-03-14T13:36:25Z | | sosedoff/pgweb | master | 2020-03-16T01:54:17Z | | sourcegraph/go-langserver | master | 2020-03-03T00:36:35Z | | stripe/veneur | master | 2020-02-10T22:18:51Z | | twitchtv/twirp | master | 2020-01-22T18:01:08Z | | u-root/u-root | master | 2020-03-21T00:56:35Z |

[narqo]

@rsc could I point out, that, from personal experience, dependency management is generally more critical for (private) applications, than for (open source) libraries. I suspect, that by counting raw numbers of users it will make it look like 100 small and popular libraries, that "don't care what dependency management system they use", overweight 1 large application, that uses those libraries.

Clarifying the status of dep is very important for companies' so they can move towards to modules. Right now, because dep is part of golang's GitHub organisation and it isn't officially deprecated, it isn't clear for management and development if they must invest in changing what used to work or they can wait until dep will catch up.

[rsc]

Does anyone object to accepting this proposal?

[rsc]

Based on the (lack of) discussion above, this seems like a likely accept. I will also comment on golang/dep#2233.

[daenney]

I see golang/dep#2233 got merged, which is great. I'm wondering if we should add some stronger language though, stating that the project is no longer maintained and everyone is encouraged to migrate to modules?

[mvdan]

The top of the README also asks for funding, which is a bit weird for a deprecated experiment.

I think we should archive the github repository in the short term, to clarify that it's there for historical reasons, and won't see more development in the future as part of the Go project.

[flibustenet]

There is also https://golang.github.io/dep/ which doesn't mention modules at all.

[mvdan]

Good point, @flibustenet. It's the second result when one googles "go dep", so I think it should be fixed up too. The source seems to be under the same repo, at https://github.com/golang/dep/tree/master/website.

[daenney]

Alright, so follow-ups so far:

• [x] Add clearer deprecation warning to Dep README
• [x] Remove call for funding on dep
• [x] Close issues and PRs
• [x] Add deprecation warning and point to modules for golang.github.io/dep, or perhaps just redirect directly to https://blog.golang.org/using-go-modules
• [x] Archive dep

[rsc]

Because this is a significant issue, I'm going to leave this for an extra week of "likely accept" before moving it to accepted. I note that golang/dep#2233 has been merged, so what's being proposed here has become a smaller delta.

[rsc]

No change in consensus here, so accepting.

[mvdan]

Could we move forward with this? In the past week, I've seen two people get misled by the fact that the dep repository hasn't been archived and still has open and pinned issues which ask for funding, for example.

There's also a somewhat regular stream of new issues and pull requests on that repository, suggesting that many Go users still don't know that the tool is deprecated and the repository is effectively (but not actually) archived.

If the remaining action items approved simply require more hands, I'm happy to volunteer my time - I'm already in the github organization, so I'd just need to be given enough permissions in the dep repo.

[ALTree]

I agree with @mvdan and I also volunteer to help finish this if I'm given permissions to the dep repository. Who control the access to the single repositories in the golang organization, and can give out access? @andybons can you help with this?

[andybons]

Yep that’s me. Happy to help. Invited you both as maintainers. If you need admin access just let me know.

[mvdan]

Thank you! I'm getting started closing issues and PRs now. I'm linking back to this issue in each of the copied comments, so apologies for the cross-reference spam. I'll update this thread when I'm done.

[andybons]

Thank you very much @mvdan and @ALTree for volunteering to do this work.

[mvdan]

Alright, issue and PR cleanup done. No more cross-reference link spam :)

@andybons could you make me an admin? I do indeed need that permission to edit the repository settings, such as archiving it.

Also, I think we should amend the wording on the README and https://golang.github.io/dep/docs/introduction.html to clearly say that the project was an experiment that's now deprecated and archived, and point the reader towards the official modules documentation. I propose:

Please note that Dep was an official experiment to implement a package manager for Go. As of 2020, Dep is deprecated and archived in favor of Go modules, which have had official support since Go 1.11. For more details, see https://golang.org/ref/mod.

If that sounds good, I'll do a PR and shortly after we can archive the entire repo. Edit: I'd need to modify two branches to accomplish the above, so I'll just push directly to the branches if others agree with the wording.

[theckman]

Hold on. I was given access to this project so that I could work on some things to keep it running.

I'm starting to look at these issues and trying to work on things to improve them, so that I can continue using dep. You just ripped all of the PRs and issues out from under me without involving me.

That's not okay. Why was I never mentioned on this issue?

[mvdan]

Hold on. I was given access to this project so that I could work on some things to keep it running.

Where is that documented? I hadn't heard about that myself.

That's not okay. Why was I never mentioned on this issue?

This was very clearly outlined in early 2019 by @andybons and others, for example here: https://github.com/golang/go/issues/29639#issuecomment-454509924

Once modules is moved out of experimental (we are targeting Go 1.13), we will likely archive the dep repo shortly after to avoid confusion on which dependency management solution the Go project officially supports.

If you want to continue maintaining dep, doing so in a fork would be fine. It's unfortunate that GitHub doesn't have a way to inherit issues and PRs by forking, but as per the link above, the current repo is to be archived and the proposal to transfer ownership was declined.

[mvdan]

I should also clarify that this process has been slow and very public; this proposal was filed half a year ago, and follows another older issue about confusion around dep. It's true that we could have pinged the former dep maintainers for extra eyes, but this still went as planned and through the usual slow-paced proposal process.

[myitcv]

@theckman

That's not okay. Why was I never mentioned on this issue?

In https://github.com/golang/dep/pull/2233, specifically https://github.com/golang/dep/pull/2233#issuecomment-614168459, @rsc linked to this issue on 15 Apr 2020. @sdboyer then approved the PR.

[theckman]

For personal reasons I am not able to frequent the Go issue tracker, unless I am filing a bug report based on something discovered in the field. So I won't see any proposals or issues that come through unless I am tagged either via GitHub or Slack.

This honestly really hurts me on a personal level, because I was worried this exact thing would happen when I was given access to the dep project. I instead decided to go against my gut and assume nothing like this would happen and that @rsc would ensure that we'd have good communication.

I don't really know how to convey how this makes me feel emotionally, but with everything compounded in 2020 I kinda wanna cry after this.

@mvdan Can you please furnish the list of Issues and PRs closed so that I can make effort to recover that state?

Also, how do we make sure we aren't excluding folks in conversations that are extremely relevant to them?

[theckman]

@myitcv apologies if I missed it in your links, where was @theckman mentioned there? As I understand it, as was shared when I was added to dep, Sam was no longer involved.

[myitcv]

where was @theckman mentioned there?

You were not explicitly mentioned, but I'm not clear why there is an expectation that you, personally, should have been explicitly mentioned? Can you expand on that?

[mvdan]

I was worried this exact thing would happen when I was given access to the dep project

I'm still confused - when and where were you given access or custody of the project? I'm sorry that I've made you feel that way, but again, I was not aware and haven't seen any evience to make me think that I could have done better.

@rsc would ensure that we'd have good communication.

You were in multiple github threads where the plan to archive the repository was made clear, such as in the link I shared above with Andy's comment. As far as I can tell, you were made aware of the plan in at least that one place. If there is a newer thread that I've missed where it's agreed that the dep repo will remain active and in your custody, could you share a link?

@mvdan Can you please furnish the list of Issues and PRs closed so that I can make effort to recover that state?

You can filter by commenter:mvdan, because I'm pretty sure I never did anything on the dep project other than the bulk actions today. But please don't start reopening issues right away, because that would just further confuse the people in those threads, and this proposal is still accepted as of today.

[ALTree]

Let's not pretend that golang/dep was not unmaintained, please.

Excluding my PR to update the README, the latest commit is from August 2019, and the latest release is from July 2019. And unless I failed to properly search the issue tracker, the last time you commented on an issue or a PR was in November 2019. Nobody realized you were maintaining it because there hasn't been basically any activity on the repository in the last year.

[mvdan]

I'm going ahead with the paragraph from https://github.com/golang/go/issues/38158#issuecomment-687212823 on both the README and the github.io site, just to not leave the project in a half-deprecated state where the README still asks for funding. It got thumbs up from two people who are Gerrit approvers, so I reckon that's enough in terms of code review.

I won't archive the repo just yet, pending more discussion, but as far as I can tell that is still the short-term plan.

[aarzilli]

@mvdan Can you please furnish the list of Issues and PRs closed so that I can make effort to recover that state?

You can get that easily with github search feature: https://github.com/golang/dep/issues?q=is%3Aissue+is%3Aclosed+closed%3A%3E2020-08-01+ and scrape them using github's API, there's even a go package to do it; make sure to get v31.0.0 I've heard it's the finest one.

[andybons]

@mvdan @ALTree you are both admins now.

[mvdan]

Thanks @andybons! I've added a clear note to both the README and the first page of the github.io docs page; see https://github.com/golang/dep/pull/2253.

Out of respect for those being away from the computer on the weekend, and to give an extra few days for others to comment, I'll wait until Tuesday morning to archive the repository.

[theckman]

What about the possibility of a transfer, to avoid having to write some automation to figure out how to life-and-shift issues?

[mvdan]

That was proposed and rejected in https://github.com/golang/go/issues/29639.

[theckman]

41237

[mvdan]

It's already Tuesday evening, so I've archived the repository. As far as I can tell, all the items in this proposal have now been completed, so I'm closing it.

[rsc]

Thanks @mvdan. Sorry we didn't close the circle on this sooner.