[META] Incremental process improvements

[aaronjensen]

Hi all,

I've been really enjoying using spacemacs and am wanting to be able to contribute as much as I can. I know that we are all super busy and I'm very appreciative of all of the great curation, building and community fostering that @syl20bnr and the rest of the collaborators have done.

I've noticed that, when viewed as a system, there may be some room for improvement in the spacemacs project and I was hoping I could contribute in some way. Below are a few suggestions for how we might improve the overall process and start seeing a drop in issue count and pull requests. I'm new to the community, so please excuse me if these have been discussed and shot down before. My ideas only come from managing other projects with many stakeholders (not nearly as many as spacemacs!).

1. Close issues that are purely questions and redirect them to stackexchange or gitter. I recognize that this won't reduce overall work, but it opens things up to a wider audience and moves things to places that are better suited to Q&A.
2. Move discussions (like this!) to a discuss or a mailing list. Discussions could go on for a long time and there are better tools than github for it in my opinion. It is nice to have everything in one place, but it's also nice to have things in their own place.
3. Prune older issues. Rails does this with a bot, they'll leave a comment on older issues and if they are not responded to they will close them out. Alternatively, you could declare issue bankruptcy and close everything older than 3 months. This is a pretty classic and effective backlog technique. You start the slate clean and important things that people are interested in will reemerge. This may upset people, but it's probably no more upsetting than their issue sitting idle for months. At least they get closure :smile:
4. Add more pull request mergers with strict guidelines. If I was told, for example, to merge pull requests that only included small bug fixes or keybinding additions, etc, but to ignore anything larger, I and 10 other people could do that and the little pulls would start to disappear. The develop branch provides a safety net and the more experienced collaborators could monitor.
5. Add more people with the ability to close issues with guidelines like 1 and 2 above. The labels get added super quickly, so those people could also be more judicious in closing issues outright. Again, this may upset people, but I think, in the end, people would get better service on gitter and stackexchange.
6. Publish more conventions and guidelines so that it is easier to train people for 4 and 5.
7. Start to actively recruit and identify people that may be able to do more than just open issues and occasional pulls to give tasks like 4 and 5 with the guidance provided by 6.

Those are just some ideas off the top of my head, maybe there are some other ways that we could improve the overall process without increasing the existing collaborators' workloads.

Thanks again!

[TheBB]

I just want to say thanks for a well thought out and constructive post. This issue is raised in one form or the other every few months but this one brought a lot of good ideas to the table I think.

I concur with 1 and 2, I think I want to get rid of questions and discussions here so that we can focus on just the bugs and the feature requests. A mailing list would not be amiss.

I also am fine with 4. https://github.com/robbyrussell/oh-my-zsh is the classical example of a project that blew its infrastructure and I would hate to see Spacemacs go the same way. Almost every pull request from people like @justbur, @bmag, @cpaulik, @person808 (to mention some) are quality, and in spite of the occasional difference in opinion I'm sure they would stick to the guidelines as requested.

[AlejandroCatalina]

I think this is a very good idea. I agree upon what has been said here. A growing community like this one needs to improve and get more people to work.

There are really good contributors that may have a more involved role. While just a few PRs are core stuff or so, I think we can go more agile on the rest.

[joehillen]

3 is one of the biggest issues with most github projects. All PRs should have a reasonable SLA.

• If you don't want a change, just close it.
• If contributor isn't responding to questions or fixes, just close it. They can reopen it later if they wish.
• If the PR is good but not perfect, just merge it and fix it. (spacemacs is pretty good about this one.)

The worst thing you can do to a contributor is ignore them. Spacemacs is above average when it comes to PR management, but 150+ open PRs does indicate that things could be improved.

[StreakyCobra]

I agree, a reform of the collaboration system is needed from my point of view, but this is up to @syl20bnr as owner of the project. I made him some propositions in this sense recently, but he is busy these days so let's see when he will have time.

1. A simple "go" from @syl20bnr and I'll do this with pleasure.
2. A mailing list could be nice indeed. I don't have any problem with the current way though.
3. We organized such a cleanup in autumn: https://github.com/syl20bnr/spacemacs/issues/3549 I like the bankruptcy idea, it would be like a restart after an "issues overflow" (what is actually the case now).
4. I'm more or less in this category, allowed to deal with small bug fixes and documentation. A few more people could be nice indeed.
5. I think people you describe here should be the same that are in the 4th category :point_up:
6. The template I'm currently pointing too on badly described issues is now integrated in Spacemacs with a keybinding on the develop branch. The next step for me would be to make it mandatory.
7. I'm already keeping a list of names just in case :wink:

[peterhoeg]

One more thing to add to this (until the ML is up and running) is splitting things up between big hairy changes (like the layer reorg) and super simple low-impact fixes like #5025 (shameless plug, but a perfect example).

I know we have the release branches, but the current contributor guidelines say that everything should be done against develop which may be quite far from release.

One could define low-impact as:

• Does not change any keybindings, only adds them
• Does not require changes to any other layer
• Does not require changes to spacemacs itself apart from addition of new functions

If @TheBB or @StreakyCobra was up for it, it would make sense to have one of them (or someone else trusted by @syl20bnr) then be in charge of the most recently released version and assuming one other person does a :+1:, then the change could go straight into the next release - or some other variation of that flow.

Smaller fixes/improvements would hit the users faster and develop would then be for larger items.

[joehillen]

In regards to a questions system, i3 recently locked up their FAQ site and moved to reddit. Perhaps we could direct questions to the spacemacs subreddit and ask the mods there if they would add the core devs as mods.

As for discussions, reddit could work, but an old fashion mailing list might be better for people that don't want to sign up for reddit. Discourse is a nice middle ground between the two.

[syl20bnr]

In advance sorry to disappoint.

I voluntary keep the size of people being able to merge small to let the project grow slowly because I truly believe that speed =/= quality and we have no special speed agreement to give at all, you can just trust us to never stop merging PRs.

Now when I say slow it is relative, Spacemacs development is a fast project! Maybe a bit less fast now but that's OK from my point of view, sometimes it speeds up when needed, sometimes it slows down like now.

Also while I cannot prevent people from starting other discussion channels, I want as much things as possible on GitHub and I have reasons for doing this, one of these reasons is tags. Thanks to tags and the good tooling on top of them we will be able to have wonderful ways for the community to interact, discuss and help each other without the need for an army of people with complicated guidelines to merge things.

[syl20bnr]

Said another way: growing the size of mergers is the wrong answer to the current situation. Developing the right tools is the good answer.

By the right tools I can think of a voting system for PRs and most importantly a subscription mechanism based on tags. From tags we can even imagine being able to reconstruct a forum like interface by grouping discussion issues, there is a lot of possibilities with some imagination. By keeping all those things together we can leverage the cross-reference/mentions mechanisms of GitHub which are to my mind essential for a large project.

[peterhoeg]

Would you consider splitting up PRs between minor and major, so that the quick fixes could go in faster?

[robbyoconnor]

@peterhoeg that kinda already happens.

[peterhoeg]

@robbyoconnor, through cherry picking of PRs I presume then? Maybe it would be an idea to have something like a next branch that all smaller items could be targetting.

[robbyoconnor]

@peterhoeg, certain pull requests get priority over others.

[robbyoconnor]

and the cherry picking of commits preserves a linear history

[syl20bnr]

I like the idea of a bot to cleanup old issues, this is really a robot job.

[aaronjensen]

I thought this may help frame the discussion:

Created w/ https://github.com/MorrisJobke/github-issue-counter

[robbyoconnor]

@syl20bnr -- I propose auto-closing PRs against master

[cpaulik]

Has anybody ever used ZenHub on a bigger project like this. I've tested it recently but am not yet sure how useful it would be in this context.

It mainly introduces:

• Task boards
• Voting on PR's and Issues
• Todo Lists based on Issues and PR's

Big drawback is that one has to install a browser extension to see all these things.

[cpaulik]

I've also found a few interesting points of view in the discussion going on at https://github.com/eslint/eslint/issues/5205 . Mind you I am not suggesting a move away from Github.

To save @robbyoconnor time :wink: we could also use a bot for reminding people about the contribution guidelines.

[robbyoconnor]

Has anybody ever used ZenHub on a bigger project like this. I've tested it recently but am not yet sure how useful it would be in this context.

It mainly introduces:

Task boards Voting on PR's and Issues Todo Lists based on Issues and PR's Big drawback is that one has to install a browser extension to see all these things.

This is great, I installed it and it looks great. However is it free?

To save @robbyoconnor time :wink: we could also use a bot for reminding people about the contribution guidelines.

I think a bot will save us time. Closing invalid PR's for example those against master should always be closed. We can also evaluate that issues follow our template format -- and if they do not, recommend they edit it to follow it. It can also automatically apply labels, saving @StreakyCobra time too!

[robbyoconnor]

Zenhub seems to free for open source projects -- So let's use it. If anyone downloads it, you will see that I have +1'd some comments :)

[bmag]

@syl20bnr can you elaborate on the tooling part? I get it that issues have searchable labels, and the mentioning and cross-referencing sure is nice, but it still lacks many useful features (voting, bots, a forum, label-based subscriptions). Do you have any existing tools in mind, expect tools to appear in the future, expect us (active community members) to build these tools?

I've went through the list of GitHub-integrated collaboration tools. I haven't used any of them, but I found these tools seem to provide some of the features mentioned: ZenHub, Zube, Waffle.

However, it looks to me that these tools are aimed at team members, which as I understand it means only you and the collaborators (@TheBB , @StreakyCobra). The rest of us can't even label issues or use the "assign" button...

I should also mention that GitHub to responded yesterday to the Dear GitHub letter, and according to them we might see improvements to GitHub issues in the next weeks and also afterwards.

I understand your reluctance to add mergers. Can we help in other ways to handle issues and PRs? Here are some things off the top of my head:

• review PR quality
• test PRs locally
• answer question issues
• debug issues and submit fix PRs

[bmag]

@cpaulik interesting link, thank you

[Andre0991]

One problem is that sometimes it's hard to tell the exact situation of a PR - is it being considered? Is its idea approved? How to move on?

I think it would be good to have defined states that a given PR or issue can have. The idea is that for any given PR or issue, one can know exactly what's needed to move on. We already have some tags for that, my suggestion is to expand them and determine a flow that all PRs would follow. A bot could help with this too.

[AlejandroCatalina]

A basic kanbas workflow seems to work for this very case. Maybe assigning people to PRs can help too.

André Peric Tavares writes:

One problem is that sometimes it's hard to tell the exact situation of a PR - is it being considered? Is its idea approved? How to move on?

I think it would be good to have defined states that a given PR or issue can have. The idea is that for any given PR or issue, one can know exactly what's needed to move on. We already have some tags for that, my suggestion is to expand them and determine a flow that all PRs would follow. A bot could help with this too.

---

Reply to this email directly or view it on GitHub: https://github.com/syl20bnr/spacemacs/issues/5037#issuecomment-183683857

Un saludo, Alejandro Catalina Feliu

[robbyoconnor]

@Andre0991, I've learned to be patient I had one PR sit for like 2-3 months I think?

[Andre0991]

@robbyoconnor The biggest issue for me is not time but organization. It gets harder to manage a project as the number of PR/issues increases and we don't know clearly the state of each one.

I haven't studied kanban yet but I agree with the gist of @AlejandroCatalina's suggestion. It's probably better to use known & tested solutions for this kind of problem.

[AlejandroCatalina]

I agree with @Andre0991 , so many PRs without knowing where each one is could be a little confusing, for mergers overall.

I think the original suggestions made by @aaronjensen on the issues cleanup are actually needed asap. We'll think later about PRs, as they seem a more important matter to deal with, IMO.

[robbyoconnor]

waffle.io works.

[syl20bnr]

@cpaulik Thank you, I'm currently testing ZenHub and already see some benefits for our workflow.

I don't see the point to move to Gitlab or elsewhere for now, it would be a lot of work for little gain and most importantly it would upset all the users.

[syl20bnr]

I've added two columns merging and merged, everybody should see the events in the PR timeline when it is moved to one of this column.

Also the +1 can be used by everybody but unfortunately we cannot sort by +1 so the usefulness is limited.

I planned to merge 30 PRs tonight but I merged only half of them, the merging status can still be seen in the PR thread (for people with ZenHub installed), I will merge them tomorrow.

@bmag I'm currently working on a phoenix/elm stack for spacemacs.org, I'll let people know when the stack is ready. For a project like Spacemacs we won't find a project that fills all our needs, we will have to code our tools at some point. For the agile management of the project, no doubt we can use an existing tool, but for all the community related tooling, we will have to code our own. I know this is ambitious but Spacemacs is an ambitious project, I consider Spacemacs half-done if we don't have the right tools to have a vibrant community. To my mind the right tools can only be done by us.

[syl20bnr]

@aaronjensen Thank you for the plot, I see we are doing pretty good. The cleaning campaign seems to work for issues and PRs growth is linear which is expected, I would have been worried if the growth rate was not a constant, I'm confident with our capacity to handle PRs, especially since I have been quite busy during the last month. We can diminish the number of opened issues with an auto-close rule, would be helpful to have an histogram of issues classified by age to see what we can do.

[aaronjensen]

@syl20bnr no problem. Yes, a linear growth of pulls is OK if you have a plan for reversing it, of course. I'll see if I can put a histogram together of issues when I get a chance.

Re: zenhub, it looks cool. Might I propose some additional columns? I realize that some of these have labels already, but if we are going to use zenhub, it is nice if the columns clearly communicate status. These are just ideas, but as a contributor, I feel they would help me know what is going on with my pulls/issues and others.

• Incoming - Things here are issues/pulls that have not been looked at/tagged/etc by collaborators, it would be the first column. Items here would either be moved to an appropriate column or closed. Ideally this column would be emptied as soon as possible.
• Discussions & Questions - Issues/pulls w/ the discussion/question label.
• Waiting - Needing more info from original submitter, or a change to their pull request. Cards here, after a certain period of inactivity could probably be closed.
• Backlog - Issues that have been triaged by collaborators and should be worked on by collaborators or community.
• High Priority - A backlog of issues and pulls that are high priority, i.e. fix regressions, etc.
• Should we Merge - Collaborators can put pulls here if they look ready to merge, but merging them is questionable and may require discussion.
• Recommend to Merge - Collaborators can put cards here for @syl20bnr to consider if they think something is ready to merge.

[robbyoconnor]

For those interested this was an auto-generated issue: #5054

[aaronjensen]

This might be worth exploring: http://gitmagic.io/#home

[robbyoconnor]

I like it. We should consider something.

[d12frosted]

What I don't like about tooling like ZenHub - it works only with Chrome and Firefox, because it comes as extension. I would prefer separate service that works using GitHub access tokens, instead of injection.

I also wonder if there are any good options for GitHub in Trello (also cards and columns).

[aaronjensen]

@d12frosted https://github.com/integrations/zube and https://github.com/integrations/waffle would meet that need. I also like the fact that waffle (and maybe zube?) use labels instead of state stored on their servers (I think). It lets you do additional analysis w/ regular tools when you use labels as a backup for which column you are on. zenhub has some cool features, but the others may be better. Having it be off-site is bad, but having it be accessible to all browsers is good.

[cpaulik]

Github has added the possibility for Issue and PR templates. Details

[syl20bnr]

@cpaulik thank you, we are in the process to add it, should arrive with 0.105.10 which should be ready tonight (lots of should :-))

[syl20bnr]

@aaronjensen thank you for the proposition, @StreakyCobra has a great plan for refactoring our labels, he will work on it during the following weeks. The main idea is to have 3 top level tags which are bug tracker, forum and mailing list so we simulate different apps but gather everything under the same interface.

[d12frosted]

FYI, Issue and Pull Request templates.

[robbyoconnor]

Did anybody add

[ ] Pull request is against develop -- if we do this and they fail to check it -- The PR is invalid and close it. I wanna make it idiot-proof.

[d12frosted]

Having this checkbox is better than just a link to contribution rules. But still, it's far from perfect solution. It would be better to have an option to block master for any PRs. So people can only send PRs against develop. I believe it's not that hard to implement, so waiting for more improvements from GitHub :D

[StreakyCobra]

In the meantime this has been implemented on Github it could be a good solution.

[robbyoconnor]

I made one -- granted the wording was partially due my annoyance of people not paying attention. On 02/20/2016 12:53 PM, Fabien Dubosson wrote:

In the meantime this has been implemented on Github it could be a good solution.

— Reply to this email directly or view it on GitHub https://github.com/syl20bnr/spacemacs/issues/5037#issuecomment-186658555.

[LemmingAvalanche]

Add more pull request mergers with strict guidelines.

This isn’t the only way to solve the problem of a pull request backlog bottleneck. No repository is blessed in a technical sense, so everyone can pull from any public PR. There are a lot of possibilities for organizing things, and some of them might even be practical in a project like this (what do I know about that, really). One possibility might for other contributors to pick up incoming patches to this repository and bundle them into larger PRs—like for example bundling those simple PRs that you suggest.

That might sound like it would be complicated. But in any case; when you’re using a DVCS the number of people with write access to a repo doesn’t have to be a bottleneck.

(Sorry to ping everyone on an old thread.)

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Please let us know if this issue is still valid!