Issue management: discuss caine again

[indutny]

When things have just started with io.js, I proposed to use a "gentle buttler" bot caine for the purposes of tagging and assigning the issues. It has been awhile since then (a bit more than 1 year), and I think it is a right time to discuss it again.

This doesn't mean that the bot should be used as it is, though. I just want to see a possibility of off-loading issue tagging and mentions of github groups to some automated system (which is sadly not provided by github itself).

What are your thoughts on this @nodejs/ctc and @nodejs/collaborators ? What should be changed in caine in order to make it useful for us?

Thank you, Fedor.

---

P.S.

A bit of personal reasoning for this: I was watching the joyent/node repo years ago, but the number of email notifications was just killing my productivity and trashing my inbox. So I decided to unwatch it, and pick the issues where people mention me, or rarely pick them manually from the issue tracker (assuming someone has tagged them with crypto, tls or anything else). Having automated system which will notify core devs about new issues that are relevant to their scope will definitely be much more useful to me, and hopefully to others too.

[indutny]

cc @chrisdickinson and @piscisaureus

[indutny]

also @mscdex and @TheAlphaNerd

[mcollina]

:+1: that would be pretty helpful, even something that just automatically notifies when a relevant tag is applied.

[jasnell]

+1

[MylesBorins]

I'm very into this and up to contribute in any way I can help

[mscdex]

In general I think I am for something like this, but I'm not sure about the auto-issue labeling since sometimes an issue may need to be labeled more than once if it affects multiple parts/subsystems (e.g. something crypto-related that affects both crypto and tls). Also, sometimes the subsystem prefix in the commit isn't always the best choice the first time around, so users could end up getting erroneous notifications.

I do think however that notifications on manual label assignment might be good.

[mscdex]

I am also curious as to how the notifications would work and how you would subscribe/unsubscribe to specific labels.

[indutny]

@mscdex the way caine works right now:

1. Post a comment in the issue/PR, asking user to answer to a list of questions (subsystem, version, stuff like that)
2. Tag issue accordingly
3. Mention github user/team according to selected tags

So it is all messages-based.

[indutny]

@mscdex all of this is configurable through CONTRIBUTING.md, which may look like this: https://github.com/indutny/caine/blob/master/test/fixtures/CONTRIBUTING.md#caines-requirements

(Updated link)

[tellnes]

+1 from me.

I'm not an active contributor at the moment, but I'm trying to keep updated by reading some issues. If I can be mentioned more often in issues that I care about and less in issues which is not relevant that would be great.

I think one of the problems last time was that it started by commenting in all open issues which resulted in a lot of full inboxes.

[indutny]

@tellnes if we are talking about the same thing - it was just a bug that was quickly fixed. It will comment only on new issues if we will decide to use it.

[tellnes]

@indutny Yes. But I'm guessing it was one of the reasons it did not have enough support last time. But please correct me, it is some time ago and I was not involved in the decisions then. I just remember that bug was a pain.

[indutny]

Do we want to discuss it on CTC call tomorrow @nodejs/ctc @rvagg ?

[joshgav]

Perhaps we can abstract up a level and discuss issue management in general. It's difficult for veterans and newcomers alike to follow issues and comments across the nodejs org's many repos, not to mention other orgs; perhaps we can consolidate all to one tracker with tags, milestones, etc.?

Doesn't have to be GitHub, but ideally should integrate into GH and provide some of the conveniences GH does, like # references. Most likely there are companies trying to tackle this problem already - the Dear GitHub flareup clearly shows a need for issue trackers with deep GH integration; perhaps we can find and work with one.

[orangemocha]

Nice work @indutny! The question-driven auto-tagging seems useful to me.

Mentions could also be useful if done well. I am slightly worried about them becoming too noisy... as you noted it's already a challenge not to be flooded with too many notifications. Would it make sense to create a level of indirection so that people can actually choose what automated mentions to subscribe to? For example, if the subsystem is windows, assign the windows label and notify @nodejs/label-windows (or call it @nodejs/caine-windows)? The idea is to keep the mentions raised by caine separate from the ones raised by humans, at least while we test this out.

Also, is it possible to do a pilot test run on a subset of issues and PRs? Like one every X?

[bnoordhuis]

https://github.com/dear-github/dear-github/issues/125 <- looks like GH added issue templates? If so, huzzah!

[indutny]

OMG