Create bot for surfacing stale issues and closing issues in the Icebox pipeline.

[nerrad]

We currently use the Icebox zenhub pipeline to contain issues that we feel are very low priority and unlikely to be tackled by the team. It's a way of helping prioritize our work and keep our backlog relevant. However, over time the icebox is essentially ignored and the issues just stagnate. As a funneling tool, it would have more value if periodically we reviewed issues in there to see if priorities have changed enough to warrant a second look at something placed in there. While this can be done manually, I think it'd be more helpful if we have a bot that helps surface things for us.

This bot will:

• likely be a github action (if we can have it act on a cron or a somewhat periodic frequency).
• work with issues in the Zenhub pipeline (so utilize the zenhub api).
• identify issues that are stale after x days, and add a stale label to them and cc configured repo team for triage.
• the team would have the opportunity to triage the stale issues to determine whether to move out of icebox or just remove the stale label.
• any issues in the zenhub pipeline will be automatically closed after x days since receiving the stale label.
• need some sort of config (.yml) that will be used to populate the periods and other dynamic values (pipeline id, label, etc).

[mikejolley]

GitHub Actions workflows can't be executed on this repository. Actions is not available for legacy per-repository billing plans. Please visit your billing settings to upgrade your account.

Might be a blocker..

[nerrad]

Yup, until pgbkb-6Oj-p2 happens at least. However, I could implement a custom probot app in the meantime.

[nerrad]

Mike if you end up doing this as a Github Action, it would be good to do it in the https://github.com/woocommerce/automations repo and then it can just be pulled in here via workflow. That would save you a bunch of boilerplate for action setup and the repo is already setup with a build process etc (I can do a quick walkthrough with you when you start this).

[mikejolley]

So some initial observations:

• ZenHub API lets you retrieve the pipeline of an issue, but it doesn't let you query by pipeline
• We could workaround with labels, but ZenHub workflows don't currently let you assign a label based on pipeline (e.g. Icebox label)
• Regardless of pipeline, it may be beneficial to expose issues (even in the backlog) of being stale. Priorities change.

I'm going to suggest that this automation does the following:

• Any open issue that has not been modified in X days (60 days), excluding those in certain pipelines ( "in progress" and "review/QA"), is labeled as stale.
• Stale issues get a comment that asks if the issue is still relevant; user can remove the label, close it immediately, or move it to the Icebox.
• After a further X days (10 days) action is taken based on the pipeline ID of each issue. In our case this would be to:
  • "backlog" - no action, leave issue open and labelled as stale
  • "icebox" - close the issue with a comment

The stale label can be a trigger for other events, for example. posting them to a Slack channel via baymax.

ZenHub API will need to be used to get pipeline data per issue, see https://github.com/ZenHubIO/API#get-issue-data

[mikejolley]

If we go with the suggestion above, we could actually just use https://github.com/actions/stale which is an existing stale issue handler. We would just set days-before-close to -1 (never close) and handle issue closing ourselves via a new automation.

So to clarify:

https://github.com/actions/stale marks issues and PRs stale, with a message, after X days in all pipelines. https://github.com/woocommerce/automations will include a new automation to handle stale issues based on pipeline.

The new automation will update an issue after X days, closing those only in the Icebox pipeline.

How does that sound @nerrad ?

Alternatively we could just close everything stale and ignore pipeline :D Then https://github.com/actions/stale would be all that is required. That is a little more aggressive though :p

[nerrad]

Thanks for your investigation here and suggestions!

ZenHub API lets you retrieve the pipeline of an issue, but it doesn't let you query by pipeline

Correct, but it is possible to retrieve a Zenhub board which is grouped by the pipelines and then issues in the board. The issue objects in the response don't have full data, but you could probably use it to get the initial list of issue numbers to do a regular Github query with - or match against results from a separate Github query. One thing we have to watch for is that Zenhub has a 100 requests per minute rate limit - so grabbing all the issues and then doing individual requests to find out what pipeline the issue belongs to in Zenhub is likely to hit the api rate limit at some point.

Alternatively we could just close everything stale and ignore pipeline :D Then https://github.com/actions/stale would be all that is required. That is a little more aggressive though :p

I'm leaning more towards this now. I'm regularly triaging (weekly) with Gary and David so we could prioritize any stale issues that pop up and either leave them to be closed, or remove the label to add back into the queue. We could use the pipeline ourselves to assist with triage. For example, if a stale issue in Backlog pops up and we're still not sure about closing, we'd move it to Icebox. When an Icebox issue surfaces to our triage we'd use that as a final check on whether to just close.

So essentially, the stale automation just helps us get an immediate signal on the age of the issue and handles automatically closing if we ignore.

Thoughts?

[mikejolley]

I'm leaning more towards this now. I'm regularly triaging (weekly) with Gary and David so we could prioritize any stale issues that pop up and either leave them to be closed

Makes sense to me, and avoids over-engineering something now that may not be needed. I'll setup the stale action in that case. If we set it up initially as 60 days stale, close in 10 days, that gives plenty of time to act on issues if needed. Numbers can be tweaked when the PR is up.