Building issue traige team for k/community

[spzala]

update: @ameukam will be leading the triage team for kubernetes/community

Recruiting the following: Triage Lead Shadow 2 Triage Members

We will use this issue to gather current issue triage challenges, thoughts on improving related processes, define related roles etc. For example: How to minimize the mislabelling of issues? How to triage a huge backlog? How SIGs are using triage processes? Ref: https://groups.google.com/forum/#!msg/kubernetes-sig-leads/7CmdCIy-efU/77U6pca9BQAJ

[spzala]

/cc @parispittman @nikhita @castrojo

[spzala]

/sig contributor-experience

[parispittman]

I'd like for the end result here to be a team of people who are working on a responsibilities doc and a grooming.md doc - does that sound good @spzala ?

[parispittman]

I'd like for us to get a team in place to help triage the kubernetes/community repo asap

[spzala]

I'd like for the end result here to be a team of people who are working on a responsibilities doc and a grooming.md doc - does that sound good @spzala ?

Thanks @parispittman Sounds good. So this will be a team of Triage Captains right? How big team you would suggest - I think a team of 3-5 is good enough to meet and brainstorm on defining new processes or modifying existing ones, pass them to SIGs etc. We can have the team members as an owner of this issue. So let's all interested people start assigning :-). Also, we have https://github.com/kubernetes/community/blob/master/contributors/guide/issue-triage.md , we can create a new grooming doc using the issue-triage.md as base or modify the issue-triage.md if not a lot of new content is needed. I like the idea of creating a responsibilities doc.

[spzala]

/assign @spzala

[ameukam]

Hi @spzala, interested to be Triage Captain.

[spzala]

/assign @parispittman

[spzala]

/assign @nikhita

[spzala]

/assign @ameukam

[spzala]

Adding some related discussion and valuable thoughts from @thockin @spiffxp from contribEx slack channel today March 8 to have it handy for future discussions. We will eventually compile all the thoughts/challenges in a doc and go over them in a meeting of triage interested parties. ... Hey contribex! I have almost 300 issues tagged sig/network. Many or most have NEVER been looked at by the SIG. We are going to go through them methodically every week until it is done. In the meantime, I have no idea which issues are or are not triaged. There is no one label I can look at.Some issues are marked with triage/support but honestly those should just be closed. Some are marked triage/duplicate but those should be closed too. I don't know what the triage labels are for. I am reluctant to build a query for the NOT of each of them. I'd rather have all new issues come in an get a triage/not-triaged label, so we could scan for that. Am I doing it wrong? nikhita [2:32 AM] Right now, "an issue is triaged" means that an issue has a sig/* label, a kind/* label and maybe a priority/* and an area/* label (if relevant). The first two are necessary on each issue. So, "technically", a sig/network issue is considered triaged if it has a kind/* label associated with it.Also -- if an issue has a lifecycle/frozen label or is assigned to someone, it's probably triaged.I'm curious to know what you had in mind for defining whether an issue is triaged though. :slightly_smiling_face: (edited) TLDR: If you see an issue with a triage/* label, it should be closed.The triage/* labels are mainly used to signify why an issue was closed while triaging (i.e. when a PR doesn't close it). These labels aren't used often -- even though technically we should use this label before closing an issue.Whenever someone chooses the "Support Request" option in the issue template while creating a new issue, we automatically apply the triage/support label. The issue tracker should definitely not be used for support requests but we don't close them automatically because that's bad UX. :grimacing: I saw your question about triaging on a mailing list too...I usually use the following queries to go through issues periodically for SIGs I am involved in. This isn't scientific but works well for me personally. Posting here in case it helps someone? :slightly_smiling_face:I'm not sure if it will help SIG Network since iiuc the idea is to go through all issues and make sure they are triaged? For me, whenever I see an issue with the following queries, I'll try to assign it someone/add the help wanted label, or add the lifecycle/frozen label if it isn't actionable right now/is a long-winded discussion, or modify the milestone, or put it on the backlog in the project board.- is:open is:issue label:sig/network label:lifecycle/rotten sort:updated-asc - rotten issues no one has paid attention in a long time (mostly can be closed)- is:open is:issue label:sig/network label:lifecycle/stale sort:updated-asc - stale issues, sorted (mostly can be closed)- is:open is:issue label:sig/network no:assignee -label:lifecycle/rotten -label:lifecycle/stale -label:lifecycle/frozen sort:comments-asc label:kind/bug -> not assigned to anyone, is possibly a bug, sorted according to least comments.- is:open is:issue label:sig/network no:assignee -label:lifecycle/rotten -label:lifecycle/stale -label:lifecycle/frozen sort:comments-asc label:kind/feature -> same as above, but for features.- is:open is:issue label:sig/network no:assignee -label:lifecycle/rotten -label:lifecycle/stale -label:lifecycle/frozen sort:reactions-+1-desc - issues having lots of +1s but no assignees and no lifecycle/frozen label. You can't apply two sort filters, so I manually look for issues with fewer comments and see what can be done about them. There are around 5 pages of results for sig/network...but issues with few comments are lesser and should be easier to go through. :)...and some more filters like these. These filters usually make it easier for me to go through the list and not feel overwhelmed. :) thockin: I'd love to help design a workflow. What I mean by triaged is "someone from SIG has looked at it". If an issue survives that (e.g. support requests should be closed ASAP), it's in some state that we could enumerate. E.g. kind/bug or kind/feature My assertion is that a state-machine exists for bug triage, but we're nt representing it through labels But mostly, I'd be super happy if all new issues got an automatic "not triaged" label. Given the existing use of "triage" to mean something slightly different, maybe we can quibble about words, but you get the point for now I have labelled 250 sig-network issues as "triage/unresolved" so that I can iteratively chip away at it every week spzala: @thockin agree about the confusion. So we already have this in the issue triage guidelines ..`Issues that are identified as a support request, duplicate, not-reproducible or lacks enough information from reporter should be closed..https://github.com/kubernetes/community/blob/master/contributors/guide/issue-triage.md#closing-issues thockin: got it. Any objections to something like "needs-sig-triage" or "needs-sig-input" being automatic on all new issues? spiffxp: Mild objection. OMG so many labels why. This puts a mandate on all issues for something else that is? Because they already have needs-kind needs-priority needs-sig. You want a label that represents yes a human has looked at this? Would “assignee” meet that? spiffxp: If the counter argument is people who aren’t the sig can add these things that I’m suggesting, the same can be said of yet another triage label thockin: no, assignee implies responsibility nobody volunteers to be assigned issues but they might volunteer to triage yeah, I need a signal that the sig has looked at it and deemed it worthy of existing spiffxp [12:04 PM] How do you gate that signal to the sig (edited) thockin [12:05 PM] E.g. auto-label every new issue "needs-sig-input". Every SIG meeting we spend 30 minutes looking at issues that have that label. People volunteer to do intake: close it, dup it, ask for more information, loop someone else in to ask more information, try to repro it, confirm it as a bug or RFE or whatever then that label can be removed and we know the issue is worthy but while the "need more info" state is happening the label stays I'm making ths up as I go, but there's a natural state machine here, we just don't have a way to identify the states by labels or if we do it is some combination of labels I don't understand spiffxp [12:08 PM] Sure I’m just trying to warn it sounds like you want to gate this to a select group of people. Most of what we do is allow anyone to apply labels. It’s way cheaper in complexity Milestone is something gated to sig chairs spzala [12:09 PM] A related thought - so a similar thing we have today that anyone looking at the issue from triage perspective, add a label to best of their knowledge like sig-net (vs needs-sig-input" ) and that gives a signals to the sig to look at the issue spiffxp [12:10 PM] Anyway this should be a :thread: and I was just adding input. We do need automated help spzala [12:12 PM] I agree it's probably not as clear as needs-sig-foo but and not an auto-label. thockin [12:12 PM] spzala - but I have no way to ACK that the SIG has looked at it spzala [12:12 PM] I totally agree thockin [12:12 PM] specifically I have 250+ backlog issues I want to redux on spzala [12:15 PM] True, so today it's like if an issue has a sig label and it's not removed then it's ACKed (which is not quite correct) Also, even if sig removes the label there is no guarantee that someone else will add a same sig label afterwards, which can pile up backlog thockin [12:15 PM] (to be clear - I am iterating but trying to find a process) spiffxp [12:16 PM] Again. How will the bot know the appropriate sig looked at an issue. Please iterate in a thread. spzala [12:16 PM] (Yup, that's how I see it :slightly_smiling_face:, this is a very valuable discussion) thockin [12:16 PM] spzala: precisely. My thought was to have an unarguable signal that an issue has or has not been "accepted"

[thockin]

Email sent to contribex list, too

On Fri, Mar 8, 2019 at 9:53 AM Sahdev Zala notifications@github.com wrote:

Adding some related discussion and valuable thoughts from @thockin https://github.com/thockin @spiffxp https://github.com/spiffxp from contribEx slack channel today March 8 to have it handy for future discussions. We will eventually compile all the thoughts/challenges in a doc and go over them in a meeting of triage interested parties. ... Hey contribex! I have almost 300 issues tagged sig/network. Many or most have NEVER been looked at by the SIG. We are going to go through them methodically every week until it is done. In the meantime, I have no idea which issues are or are not triaged. There is no one label I can look at.Some issues are marked with triage/support but honestly those should just be closed. Some are marked triage/duplicate but those should be closed too. I don't know what the triage labels are for. I am reluctant to build a query for the NOT of each of them. I'd rather have all new issues come in an get a triage/not-triaged label, so we could scan for that. Am I doing it wrong? nikhita [2:32 AM] Right now, "an issue is triaged" means that an issue has a sig/ label, a kind/ label and maybe a priority/ and an area/ label (if relevant). The first two are necessary on each issue. So, "technically", a sig/network issue is considered triaged if it has a kind/ label associated with it.Also -- if an issue has a lifecycle/frozen label or is assigned to someone, it's probably triaged.I'm curious to know what you had in mind for defining whether an issue is triaged though. 🙂 (edited) TLDR: If you see an issue with a triage/ label, it should be closed.The triage/ labels are mainly used to signify why an issue was closed while triaging (i.e. when a PR doesn't close it). These labels aren't used often -- even though technically we should use this label before closing an issue.Whenever someone chooses the "Support Request" option in the issue template while creating a new issue, we automatically apply the triage/support label. The issue tracker should definitely not be used for support requests but we don't close them automatically because that's bad UX. 😬 I saw your question about triaging on a mailing list too...I usually use the following queries to go through issues periodically for SIGs I am involved in. This isn't scientific but works well for me personally. Posting here in case it helps someone? 🙂I'm not sure if it will help SIG Network since iiuc the idea is to go through all issues and make sure they are triaged? For me, whenever I see an issue with the following queries, I'll try to assign it someone/add the help wanted label, or add the lifecycle/frozen label if it isn't actionable right now/is a long-winded discussion, or modify the milestone, or put it on the backlog in the project board.- is:open is:issue label:sig/network label:lifecycle/rotten sort:updated-asc - rotten issues no one has paid attention in a long time (mostly can be closed)- is:open is:issue label:sig/network label:lifecycle/stale sort:updated-asc - stale issues, sorted (mostly can be closed)- is:open is:issue label:sig/network no:assignee -label:lifecycle/rotten -label:lifecycle/stale -label:lifecycle/frozen sort:comments-asc label:kind/bug -> not assigned to anyone, is possibly a bug, sorted according to least comments.- is:open is:issue label:sig/network no:assignee -label:lifecycle/rotten -label:lifecycle/stale -label:lifecycle/frozen sort:comments-asc label:kind/feature -> same as above, but for features.- is:open is:issue label:sig/network no:assignee -label:lifecycle/rotten -label:lifecycle/stale -label:lifecycle/frozen sort:reactions-+1-desc - issues having lots of +1s but no assignees and no lifecycle/frozen label. You can't apply two sort filters, so I manually look for issues with fewer comments and see what can be done about them. There are around 5 pages of results for sig/network...but issues with few comments are lesser and should be easier to go through. :)...and some more filters like these. These filters usually make it easier for me to go through the list and not feel overwhelmed. :) thockin: I'd love to help design a workflow. What I mean by triaged is "someone from SIG has looked at it". If an issue survives that (e.g. support requests should be closed ASAP), it's in some state that we could enumerate. E.g. kind/bug or kind/feature My assertion is that a state-machine exists for bug triage, but we're nt representing it through labels But mostly, I'd be super happy if all new issues got an automatic "not triaged" label. Given the existing use of "triage" to mean something slightly different, maybe we can quibble about words, but you get the point for now I have labelled 250 sig-network issues as "triage/unresolved" so that I can iteratively chip away at it every week spzala: @thockin https://github.com/thockin agree about the confusion. So we already have this in the issue triage guidelines ..`Issues that are identified as a support request, duplicate, not-reproducible or lacks enough information from reporter should be closed.. https://github.com/kubernetes/community/blob/master/contributors/guide/issue-triage.md#closing-issues thockin: got it. Any objections to something like "needs-sig-triage" or "needs-sig-input" being automatic on all new issues? spiffxp: Mild objection. OMG so many labels why. This puts a mandate on all issues for something else that is? Because they already have needs-kind needs-priority needs-sig. You want a label that represents yes a human has looked at this? Would “assignee” meet that? spiffxp: If the counter argument is people who aren’t the sig can add these things that I’m suggesting, the same can be said of yet another triage label thockin: no, assignee implies responsibility nobody volunteers to be assigned issues but they might volunteer to triage yeah, I need a signal that the sig has looked at it and deemed it worthy of existing spiffxp [12:04 PM] How do you gate that signal to the sig (edited) thockin [12:05 PM] E.g. auto-label every new issue "needs-sig-input". Every SIG meeting we spend 30 minutes looking at issues that have that label. People volunteer to do intake: close it, dup it, ask for more information, loop someone else in to ask more information, try to repro it, confirm it as a bug or RFE or whatever then that label can be removed and we know the issue is worthy but while the "need more info" state is happening the label stays I'm making ths up as I go, but there's a natural state machine here, we just don't have a way to identify the states by labels or if we do it is some combination of labels I don't understand spiffxp [12:08 PM] Sure I’m just trying to warn it sounds like you want to gate this to a select group of people. Most of what we do is allow anyone to apply labels. It’s way cheaper in complexity Milestone is something gated to sig chairs spzala [12:09 PM] A related thought - so a similar thing we have today that anyone looking at the issue from triage perspective, add a label to best of their knowledge like sig-net (vs needs-sig-input" ) and that gives a signals to the sig to look at the issue spiffxp [12:10 PM] Anyway this should be a :thread: and I was just adding input. We do need automated help acontini [12:10 PM] @Paris https://github.com/Paris sounds good - I just asked Chris for a timeline so we will know how long it takes spzala [12:12 PM] I agree it's probably not as clear as needs-sig-foo but and not an auto-label. thockin [12:12 PM] spzala - but I have no way to ACK that the SIG has* looked at it spzala [12:12 PM] I totally agree thockin [12:12 PM] specifically I have 250+ backlog issues I want to redux on spzala [12:15 PM] True, so today it's like if an issue has a sig label and it's not removed then it's ACKed (which is not quite correct) Also, even if sig removes the label there is no guarantee that someone else will add a same sig label afterwards, which can pile up backlog thockin [12:15 PM] (to be clear - I am iterating but trying to find a process) spiffxp [12:16 PM] Again. How will the bot know the appropriate sig looked at an issue. Please iterate in a thread. spzala [12:16 PM] (Yup, that's how I see it 🙂, this is a very valuable discussion) thockin [12:16 PM] spzala: precisely. My thought was to have an unarguable signal that an issue has or has not been "accepted"

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/kubernetes/community/issues/3364#issuecomment-471017090, or mute the thread https://github.com/notifications/unsubscribe-auth/AFVgVK1TKZgUbajSLDshM8uBdZGr31ZXks5vUqN8gaJpZM4bf4Z9 .

[nikhita]

/kind feature /priority important-soon

related https://github.com/kubernetes/community/issues/3443

[nikhita]

/milestone May

[parispittman]

AI: @nikhita to build rolebook for triage-team under contribex; @spzala to add info into triage captain responsibilities with shadow, nikhita to do PR Wrangler from sig-docs guidance

[fejta-bot]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

[ameukam]

/remove-lifecycle stale

[ameukam]

@spzala @spzala Now #3622 is merged, what's the next step for this ?

[parispittman]

@ameukam ID'ing the team members and then building it out more.

hypothetically, if you signed up to do triage role, do you think you could do it from this document? if not, what can we improve?

[spzala]

@ameukam agree with @parispittman Making sure that the document provides good enough material would great. I am not sure but @nikhita probably was working on creating role descriptions. @parispittman may be four of us meet for a quick call/slack chat if the timing works and have some discussion around it. Thanks!

[ameukam]

@paris IMO the document is enough to guide a volunteer for the task at the beginning. @spzala agree with your idea to start a group chat on slack.

[fejta-bot]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

[cblecker]

/remove-lifecycle stale

[spzala]

I believe we can close this issue as we now have a dedicated team, a new slack channel for the team and bi-weekly meetings /cc @ameukam @parispittman @nikhita

[spzala]

We have a dedicated team, a new slack channel for the team and bi-weekly meetings set up. Closing this issue. Anything new we should iterate with new one. Hope that sounds good to everyone. Thanks!