Issues languishing unresolved for long periods of time

[dstufft]

A user has brought up some concerns that there are long outstanding issues, some of them simple or basic functionality pieces, that have been ignored for long periods of times (months or years) with no resolution or forward movement.

[ncoghlan]

Was this in relation to a specific project? If setuptools or one of its components, this is a combination of the lack of regression tests (making it fragile) and complexity of implementation (even I can barely figure out how pkg_resources actually works, since it has implicit hooks and triggers everywhere, so it's almost impossible to figure out what runs when). Again, one of the major reasons I split out the nascent import_resources project as something to work on post-3.4 feature freeze.

[qwcode]

pip has old issues like this. in my experience, most of them are fairly easy fixes once someone digs into it. to defend ourselves a bit, I've fixed a few 1-2 yr old bugs for 1.5, and have done similar in previous releases. just keep chippin away, and we should undo this perception eventually.

[dstufft]

FWIW I tried to generalize things more and tried to put down the issues whether I agreed with them or not. The project in question was pip, but I think it's been a general problem throughout all the packaging tools.

[ncoghlan]

/cc @jaraco

I think part of it has also been the toxic (non-)collaboration environment making all of us feel guilty about the idea of recruiting anyone else to contribute. There's a reason my early packaging related efforts were focused on empowering distutils-sig through changes to the PEP process rather than being focused directly on packaging itself. The culmination of those efforts was really the acceptance of PEP 453, which (in effect) formally delegated stewardship of the Python packaging ecosystem to the Python Packaging Authority rather than leaving that responsibility with the CPython core development team.

With the improvements in the working environment, do we think we're at a point where things are sufficiently coherent for us to start reaching out to groups like PyLadies and the Boston Python Workshop to suggest they consider pip and virtualenv as possible projects to help with?

I'm still wary of pointing new contributors in the direction of setuptools though - I acknowledge that setuptools solved, and still solves, a lot of problems for a lot of people, but "easy to maintain" and "contributor friendly" are not phrases I would use to describe that code base. At this point in time, I think we're probably better off trying to build setuptools-inspired parallel projects (e.g. pip as a replacement for easy_install, twine as a replacement for the PyPI upload components, import_resources as a replacement for pkg_resources, standardisation of the setup.py CLI as a replacement for the build component, distlib as a utility library for working with the packaging standards and infrastructure), rather than trying to fix setuptools itself (the risk of breaking backwards compatibility is incredibly high, whereas with pip taking over the installer role, it becomes easier to create partial replacements for specific components that people can migrate too if they meet their needs).

[qwcode]

making all of us feel guilty about the idea of recruiting anyone else to contribute

throughout, pip has had optimism around it, so I've been pretty shameless about asking people to help pip (e.g. I personally asked Donald, Daniel, and PaulM to become pip maintainers)

my hope is that when we finally announce a somewhat completed "Packaging User Guide" that presents a coherent picture and documents real unity and progress, that will create momentum, and people will naturally want to "pile on" and help.

acknowledge that setuptools solved, and still solves, a lot of problems for a lot of people, but "easy to maintain" and "contributor friendly" are not phrases I would use to describe that code base.

pip is a beast to maintain as well and changes to it involve risk. It has some pretty horrible code and needs a lot of refactor to be worthy.

we're probably better off trying to build setuptools-inspired parallel projects [...] rather than trying to fix setuptools itself.

I don't know. I think I feel more optimistic about everyone hunkering down and slowly/safely refactoring setuptools.

[ncoghlan]

I feel more optimistic about the build component of setuptools than I do the rest of it, particularly when pip is in the picture to change the defaults to something more desirable. My perspective on pkg_resources may change once I manage to take it apart and rebuild it from decoupled components in import_resources and then compare the behaviour of the two implementations.

[jaraco]

I encourage contribution to setuptools wherever possible. Since June, Setuptools has accepted 10 pull requests from 10 different contributors on bitbucket and 3 pull requests from another contributor on github. I strive to also maintain stability and predictability, hence the declined pull requests.

I believe the perception of non-maintenance is unwarranted (today) and historical, though I do believe there are some things we can do to improve the perception of languishing tickets. I intend to start closing languishing tickets as such. In particular, if a ticket is languishing for 6 months or more without a concrete patch or pull request or at least a comment indicating the next step, then it should just be closed as wontfix.

Of course, all volunteer-supported projects are subject to the same perception, so to some extent, we should also be clear that these projects will only be as good as the community that supports them, and encourage help from everybody at their capability.

[ncoghlan]

Yay, happy to find out I was wrong on that front - I may have an overly negative view after the challenges I encountered trying to resolve some of the issues I had filed against pkg_resources.

[pradyunsg]

pip is a beast to maintain as well and changes to it involve risk. It has some pretty horrible code and needs a lot of refactor to be worthy.

+1 even today. :)

[brainwane]

I wrote about this on distutils-sig:

Warehouse is attracting several newer contributors including people new to open source, which is great. As Warehouse matures, we have fewer and fewer easy small bugs in the Python side left. (So, we have more work for new frontend contributors, and less for Pythonists.) I'd love to refer these folks to other parts of the Python packaging and distribution ecosystem so we can improve the whole toolchain.

If more tools maintainers specifically mark "good first issues" (see my distutils-sig post for a template), we can share the love -- I have a list of dozens of volunteers I'd like to tell about how they can help related projects.

If maintainers actively put the word out that a project is seeking new volunteers, respond to new questions and patches within a few days, and comment on finished issues to say "great! want another?", volunteers work through a "good first issues" queue steadily and it needs regular replenishment. It is worth taking a fresh look at the queue every month or two to double-check whether any of the open issues labelled "good first issue" are harder than they first appeared, then remove the label with an explanatory comment.

(My further advice on stuff like this -- "How To Improve Bus Factor In Your Open Source Project", "How to Teach And Include Volunteers who Write Poor Patches", "Inclusive-Or: Hospitality in Bug Tracking", etc. -- are at my resources page https://changeset.nyc/resources.html .)

[pradyunsg]

There's quite a few "good first issues" over on pip. Those are all small simple changes tbh. After that it's probably gonna be an adventure. So, if someone's looking for an adventure, feel free to send them over to pip. ;P