search

trilinos / Trilinos

Primary repository for the Trilinos Project
https://trilinos.org/
Other
1.2k stars 564 forks source link

Add new labels to manage ATDM Trilinos GitHub Issues #3963

Closed bartlettroscoe closed 5 years ago

bartlettroscoe commented 5 years ago

CC List:

Description

This Issue is to document the need for and to add some new labels to improve the management of ATDM Trilinos GitHub issues as part of the ATDM Trilinos triaging and issue management process currently documented at:

There are a few problems with that process as currently documented:

Proposed Solution

For Problem-1 the proposed solution is to define three new Trilinos GitHub labels to represent the severity of the problem as it relates to that ATDM APPs usage of Trilinos:

For Problem-2 the proposed labels for the Trilinos Product Areas are:

For Problem-3, there is the label:

bartlettroscoe commented 5 years ago

FYI: I added the following Confluence page to document these new labels:

bartlettroscoe commented 5 years ago

FYI: I added the new Trilinos Product Area Labels to all of the ATDM Trilinos GitHub issues which you can see by product area in the links under the bullet "Breakdown by Trilinos Product Area" here

I also send out an email to the Trilinos Product Area Leads shown below. I have set up my Outlook client to send out this email every Monday morning at 6 AM MST through the end of Jan 2019. As mentioned below, we will try to get a more targeted automated email put together at some point in the next few months but hopefully this will do for now.

So now the Trilinos Product Area Leads should know exactly what issues are in their product area that need to be addressed at any point in time and they will be reminded of this every week.

I am putting this in review ...

From: Bartlett, Roscoe A Sent: Thursday, November 29, 2018 6:52 PM To: Willenbring, James M ...; Devine, Karen D ...; Rajamanickam, Sivasankaran ...; Pawlowski, Roger P ...; Perego, Mauro ... Cc: Wolf, Michael M ...; Frye, Joe ...; Heroux, Michael A ... Subject: Unresolved ATDM Trilinos GitHub Issues in your Trilinos Product Area

Hello Trilinos Product Area Leads,

The link to all of the unresolved ATDM Trilinos GitHub Issues that may need your attention (or the attention of someone in your Trilinos Product Area) is: • All open ATDM Trilinos GitHub Issue “bug” without “Disabled Tests” or “Stalled”

The breakdown of the above GitHub Issues query by Trilinos Product Area is given in the links below: • Trilinos Framework Product Area Issues • Trilinos Data Services Product Area Issues • Trilinos Linear Solvers Product Area Issues • Trilinos Nonlinear Solvers Product Are Issues • Trilinos Discretizations Product Area Issues In the above GitHub Issue queries, to see Issues for just the Promoted ATDM Trilinos builds or the builds we are trying to clean up and then promote, select the GitHub Issue “Milestone”: • Keep promoted ATDM builds of Trilinos clean, or • Initial cleanup of new ATDM Trilinos builds

Tasks that you may need to perform on these GitHub Issues in your Product Area include: • Work with the Trilinos developers and the ATDM APPs teams to determine the ATDM Severity/Criticality level of each issue, and assign the associated GitHub Label. • Assign the Issue to a member of your Trilinos Product Area Team and find a P/T for them to work the issue. • Check up on the status of GitHub Issues that are not getting resolved in a timely way.

NOTES: • These GitHub Issues are created by the process documented at Triaging and Addressing ATDM Trilinos Failures. • One can generally NOT determine the health of Trilinos for ATDM by just directly viewing the ATDM Trilinos Builds on CDash. (Instead, please focus on the ATDM Trilinos GitHub Issues in your Product Area in the links above.) • All information about that these builds can be found at ATDM Builds of Trilinos (or pages linked to from that page). • If you have any questions about this process, please contact Joe Frye or Ross Bartlett.

Thanks,

-Ross

P.S. At some point in the near future we hope to be able to send out targeted emails for each product area lead that directly lists the Issues in that product area and a basic summary of the status of those issues. Also, we will be setting up automated tools and processes to update those ATDM Trilinos GitHub issues when the associated tests change status and once a week to remind assignees about their status.

bartlettroscoe commented 5 years ago

@maherou (Trilinos overall Lead) @jwillenbring (Trilinos Framework Product Area Lead) @kddevin (Trilinos Data Services Product Area Lead) @srajama1 (Trilinos Linear Solvers Product Area Lead) @rppawlo (Trilinos Nonlinear Solvers Product Area Lead) @mperego (Trilinos Discretizations Product Area Lead)

Any time to look over the list of labels used with the ATDM Trilinos GitHub issues (and their meaning) defined at:

@srajama1, at the ATDM/Trilinos Testing Meeting last Friday, you commented that the ATDM Trilinos GitHub issues have too many labels. Can you please comment on which labels listed at the site that you would get rid of and why?

mperego commented 5 years ago

@bartlettroscoe I cannot access that link. If the link is correct then I don't have the right permissions.

bartlettroscoe commented 5 years ago

I cannot access that link. If the link is correct then I don't have the right permissions.

@mperego, I think the issue is that you were not in the 'wg-atdm' group. I added you and @kddevin as users with view and comment access. I also tried to add 'wg-trilinos' so hopefully all SNL Trilinos staff can pick this up. (There is nothing on that page that SNL Trilinos developers can't see.)

bartlettroscoe commented 5 years ago

@mperego, please try again to view:

Thanks!

mperego commented 5 years ago

@bartlettroscoe Thanks, it works now!

srajama1 commented 5 years ago

If we have 3 labels per issue criticality, product area, bug/enhancement/question when it gets created, I am fine with it. We could package name and other labels later. Why do you need Framework and PA: Framework ? I would also get rid of ATDM as criticality has ATDM as a prefix anyway.

bartlettroscoe commented 5 years ago

If we have 3 labels per issue criticality, product area, bug/enhancement/question when it gets created, I am fine with it. We could package name and other labels later. Why do you need Framework and PA: Framework ? I would also get rid of ATDM as criticality has ATDM as a prefix anyway.

@srajama1, if GitHub Issue queries supported ORing labels, I would agree with you. The problem is that if you remove the "ATDM" label then there is no way to query all of the issues impacting ATDM. (Also, "enhancement" and "question" issues don't have a criticality label.)

Basically, all of these labels are needed to provide the GitHub issue views provided at:

(Even if you don't need to see all of these views, we do. So to remove some of these labels makes our job harder.)

If you get rid of the package label, then there is no way to see, for example, all of the Anasazi GitHub issues impacting ATDM (which is important to see in cases like TRIL-238):

(And we are going to need to do queries and create stat reports by package to see which packages are most problematic for ATDM. That will be needed to inform staff and management about where resources need to go to improve things.)

If we get rid of the "Disabled Tests" label, then it is not possible to filter out GitHub issues that have been (partially) resolved by simply disabling tests. (And people have asked us not to close issues that have been resolved by disabling tests.) But the label "Disabled Tests" label should never show up in the GitHub issues or Issue queries that you are dealing with (because we filter those out of your view).

The labels "ATDM Config" and "ATDM DevOps" should likely never show up in the GitHub issues or GitHub Issue queries that you are having to address.

I agree that the "Framework" and "PA: Framework" labels may be redundant. But those labels should never show up in the GitHub Issue queries or Issues that you are responsible for. So it is really up to the Trilinos Framework Product Area Lead (i.e. @jwillenbring) to make the call on that. While the purpose of the label "PA: Framework" is well defined, the meaning of the label "Framework" is not. (But that is @jwillenbring's problems, not your).

One label that seems like we should not need it is the "Stalled" label. There are very few issues that are given that label as you can see in:

These are basically used for non-critical issues that are just not getting worked on or resolved. We don't want to close the issue but we don't want them to clutter up the issues that don need to get worked on. But, again, you will not see GitHub issues in your queries because we filter them out.

How do other people feel about the usage of GitHub Issue Labels?

bartlettroscoe commented 5 years ago

Also, I think we need the "ATDM Env Issue" label to be able to provide statistics about ATDM Trilinos GitHub issues that are caused by problems/changes in the env vs. Trilinos bugs. Otherwise, it does not seem fair to ding the Trilinos Framework Product Area, for example, for an issue like #2410 that has been open and unresolved for nearly 9 months (which is caused by problems with LAPACK on the Power systems, not a problem with Trilinos as far as we can tell). So we need to show separate stats for ATDM Trilinos GitHub issues with the label "ATDM Env Issue" (which shows how good the sysadmins are at addressing env issues) and show stats for Issues without the label "ATDM Dev Env" (which are bugs in Trilinos, which the Trilinos Product Area Leads and Trilinos developers are responsible to resolve).

So, in conclusion, given the views that we need to see, the stats we need to be able to compute, and the limitation of GitHub Issue queries, the only labels I think we could consider eliminating are:

Anyone else agree or disagree with this?

nmhamster commented 5 years ago

@bartlettroscoe - I think its a good idea to have an environment label.

srajama1 commented 5 years ago

@bartlettroscoe I can live with the ATDM label. I will grumble when I see it, but it is ok :) Yes, package labels are fine too.

You don't need to spend your time generating package level statistics. The product level statistics are enough for now. We can talk over phone if you want to know why :)

bartlettroscoe commented 5 years ago

@maherou (Trilinos overall Lead) @jwillenbring (Trilinos Framework Product Area Lead) @kddevin (Trilinos Data Services Product Area Lead) @srajama1 (Trilinos Linear Solvers Product Area Lead) @rppawlo (Trilinos Nonlinear Solvers Product Area Lead) @mperego (Trilinos Discretizations Product Area Lead)

If there is no other suggestions or objections, I would like to close this issue with the final product in pointed to in various links in:

bartlettroscoe commented 5 years ago

Since there have been no objections, I am closing this issue as complete. The new labels are in usage. (Now we just need to get people to assign the correct ATDM Severity/Criticality Labels and we should be good to go.)

bartlettroscoe commented 5 years ago

@jwillenbring (Trilinos Framework Product Area Lead) @kddevin (Trilinos Data Services Product Area Lead) @srajama1 (Trilinos Linear Solvers Product Area Lead) @rppawlo (Trilinos Nonlinear Solvers Product Area Lead) @mperego (Trilinos Discretizations Product Area Lead)

Given that there are a few GitHub labels that start with "ATDM" I think we should change the names of the ATDM Severity/Criticality Labels to ATDM Sev: <level>. That will make them sort together and make them easier to find when adding labels in GitHub.

Any objections or suggestions for a better prefix for these labels?

P.S. I forgot to close the issue in my last comment so I will leave it open until we make this last change.

bartlettroscoe commented 5 years ago

Any objections or suggestions for a better prefix for these labels?

Since there were no objections or suggestions, I updated the ATDM GitHub Severity/Criticality labels to be ATDM Sev: <severity> and updated all documentation that I could find and and linked to ATDM Builds of Trilinos Wiki.

Closing this as complete.

bartlettroscoe commented 5 years ago

Closing as complete ...