search

w3c / horizontal-issue-tracker

Tools and pages to track horizontal issues
https://www.w3.org/PM/horizontal/
MIT License
6 stars 4 forks source link

Feature request: Concise all-HR summary of the status of a document, for AC/Director and community to see #21

Open dwsinger opened 4 years ago

dwsinger commented 4 years ago

I'd still like the red/yellow/green indications when the document goes to the AC/Director:

summarized succinctly, e.g. as a table covering all HR groups, when we're about to make a formal decision.

I really think the AC and Director should be able to notice, at a glance, things like

I think this could be easily automated with common "final HR disposition labels" used in the HR repos, and then an automated system can collect them and display them in one place.

So, an AC email

The slow working group requests transitioning Snails1.0 to Proposed Rec. The HR review summary is (linked as):

  • privacy: yellow
  • security: red
  • TAG: green
  • i18n: not applicable
  • accessibility: yellow

and the yellow or red would be linked to wherever the details of the concerns are stated.

r12a commented 4 years ago

I'm not sure we'd be able to reliably label issues manually, however that may not be necessary anyway, given the currently envisaged process for transition which incorporates some automated checks.

Before the transition meeting the tracker repositories of each HR group should be scanned for issues related to the spec(s) in question, looking for open issues with the 'needs-resolution' label. If any are found, a flag is raised, since this suggests that there are still issues not yet resolved to the group's satisfaction. Ideally, the WG should run this tool before booking a transition meeting, and if the result shows outstanding issues, check with the HG whether the issue(s) can be resolved and closed. Come the transition meeting, the issues still open indicate a conflict, at which point the Director should step in to decide what should be done.

So your green state would equate to a situation where the checker tool is run and there are no open issues in the HR groups tracker repos with the label needs-resolution.

The yellow state equates to a situation where the checker reveals that there are relevant issues open.

The red state would equate to a situation where the transition request is made to the Director and there are still open issues.

For good measure, the tool could also look in the closed issues of the HG's tracker repo for issues related to this spec. If there are none, this could either be because the review raised no issues, or because no review was done. In the case of i18n, those cases can be easily separated by looking at our review radar project page, which lists each spec against the current review status.

samuelweiler commented 4 years ago
  • more than half the HR groups timed-out and didn't file a color on this spec. .... I think this could be easily automated with common ...

The current labels, at least as PING is using them, don't have a way to detect that. We are filing issues when found, but there's no indication for "review complete".

plehegar commented 3 years ago

Some progress on this front. We now have the possibility to a report on horizontal issues linked to a specification. For example, for the Payment Request API: https://www.w3.org/PM/horizontal/review.html?shortname=payment-request

This only track, so far, horizopntal review comments and does NOT track yet wide review requests.

dwsinger commented 3 years ago

wow, that's great. cool

I don't understand what the box before the text in only some of the lines is, and how it differs from the box after the text.

(Is there an issue with colored boxes and accessibility, by the way? are they readable or do we need symbols like ✅ for resolved and 🛑 for a blocker?)

plehegar commented 3 years ago

there is a list of what the boxes mean on the left of the page.

Regarding color labels, we had #2 . The result was to separate the colors to avoid confusion.

Right now, we're following the colors from the horizontal repositories but past attempts at changing the colors were meant with resistance. We could instead use a dedicated set of colors. I love your suggestions.

r12a commented 3 years ago

I don't understand what the box before the text in only some of the lines is, and how it differs from the box after the text.

The lines which don't have a box to the left are all pending: ie. they have been raised for review by a reviewer and are pending discussion by the HR group as to whether or not to send raise the issue with the spec group. (I believe we reviewed most of these on our i18n call last week, and Addison has an action to forward them.)

It could be argued whether such issues (which are not yet sanctioned by the group) should appear in this list. In other words, perhaps only issues with a box on the left-hand side should appear in the list for these reports. (We certainly need them there on our own lists for internal i18n review mgt)

r12a commented 3 years ago

I see that Security, Ping, & TAG are not removing the 'pending' tags when a needs-resolution or tracker status has been assigned. I'd recommend that they do so. The red colour is intended to get the attention of the HR group to say that something needs to be done urgently here – it doesn't have a meaning outside that group. PLH, maybe in report pages like these we should just not show the red pending boxes at all. The HR groups, however, will need to see them - i'm not sure whether the above groups rely on your pages to track their word, so perhaps a URL-catchable switch would allow a user to say whether they want to see from an HR group perspective (ie. with pending) or not (ie. without pending)?? Just brainstorming (i18n uses a different page, which is the model for these).

plehegar commented 3 years ago

@weiler @ylafon , see the comments from Richard above.

plehegar commented 3 years ago

@r12a , the main consumer for those specification reports is the Director and the AC. I'll check with @swickr to see if I should remove the issues that have not been raised to the Group working on the specification.

I included a link to the list issues for each horizontal repositories, thus I don't think I'd need to create a special report since the HR Group could see the list of issues directly on GH.

plehegar commented 3 years ago

I added the list of specifications known to the horizontal review system (so far): https://www.w3.org/PM/horizontal/reviews.html

It contains all of the specifications that received at least one comment in the past (since we switched to use GitHub for tracking) and the list of specifications known to the W3C database. I expect to grow the list beyond this.

r12a commented 3 years ago

@plehegar One thing we realised during the i18n telecon was that the title of a pending issue links to our i18n tracker issue (of course, there's no spec WG issue yet). We are worried that people might start discussing the topic in our tracker db issue as a result, which would not be appropriate, not least because the spec WG won't see the discussion. Better not have a link at all for the title. (For example, the title Localization mechanism unclear links to https://github.com/w3c/i18n-activity/issues/1041).

plehegar commented 3 years ago

@dwsinger , I added a status indicator and cleaned up the colors @r12a , I removed the issues that did not have tracker/needs-resolution

dwsinger commented 3 years ago

thanks, but what does a status of 'tracker' mean? Does that mean 'resolved'? (It appears on both the summary and the tracker page)

plehegar commented 3 years ago

I removed the mention of "Status". The indicator is still there as a summary. This indicator is red only if a 'needs-resolution' issue is not yet closed in one of the horizontal groups.

r12a commented 3 years ago

@plehegar the tracker pages don't show any i18n items. Compare https://www.w3.org/PM/horizontal/review.html?shortname=epub https://w3c.github.io/i18n-activity/reviews/#epub

I don't see i18n issues at https://www.w3.org/PM/horizontal/ either.

Also, the Clear filter doesn't seem to be working on that page. I'm stuck only seeing (no) i18n issues, even though i have nothing indicated in the URL.

Also, i think it would be useful to have a link on each page that says "Choose another spec: [Shortname here]" or some such. Either that or a link back to https://www.w3.org/PM/horizontal/

plehegar commented 3 years ago

I see i18n issues in https://www.w3.org/PM/horizontal/review.html?shortname=epub . Maybe a glitch?

Re Clear filter, it seems to work.

The link back is there on top left: "Horizontal issues trackers".

I could add a drop down to choose an other spec but that will a long one to scroll though...

r12a commented 3 years ago

Perhaps it was indeed a glitch. Certainly all seems to be working now.

plehegar commented 3 years ago

keeping this issue open until the review requests part has been taken into account. Right now, only comments resulting from review requests are integrated.