search

w3c / modern-tooling

Work of the modern tooling task force
http://w3c.github.io/modern-tooling/
MIT License
44 stars 39 forks source link

Tracking smarts for generating DoC #25

Open r12a opened 9 years ago

r12a commented 9 years ago

Something i've wanted for some time is the ability to make an issue tracker (dashboard, whatever...) carry metadata about each comment tracked that can be used to automatically generate a disposition of comments for a spec phase transition.

For example, a. i'd like to be able to store links to the initial comment (and any subsequent thread starts where discussion takes place on email systems, since email threads often get fragmented) b. i'd like to be able to store information about the status of the comment per such things as: accepted/rejected/out-of-scope/invalid/retracted/deferred, and solution-proposed-to-commenter/not-yet-proposed and commenter-happy/commenter-objects/no-response-yet. c. i'd like to store pointers to where we propose a solution, where the commenter replies d. i'd like to be able to automatically produce a list for the DoC report, with basic info such as commenter name, links to more info, status, etc, with colours to reflect status of each comment tracked, and allow for filtering and sorting of the list in various ways. e. i'd like to be able to record whether the comment led to a spec change, and whether that was substantive or just editorial

The report should also automatically include links to the document in TR and ED, of course.

dontcallmedom commented 9 years ago

although I don't claim this is the right response to this need, we at least have some experience with such a tool with the last-call comments tracker: http://www.w3.org/2006/02/lc-comments-tracker/

darobin commented 9 years ago

I was thinking of that. There's also the XSLT that takes a Bugzilla search and produces a DoC. @r12a how would you say those compare to what you have in mind?

darobin commented 9 years ago

Requirements:

Others?

dontcallmedom commented 9 years ago

Part of what the Last Call comment tracker does is that it tracks comments, not issues; in particular, it tracks whether a comment has been responded to or not, and if the response was satisfactory or not to the commenter.

darobin commented 9 years ago

Right, but shouldn't a comment be an issue? That's how we did it in HTML.

darobin commented 9 years ago

Note: we could use "standard" labels across repos.

dontcallmedom commented 9 years ago

re comment as an issue, the two are not equivalent (although I agree they're often managed as if they were):

None of these are insurmountable, but I've found that considering them as clearly separate notions helps deal with the formality of Last Call / wide review processing.

plehegar commented 9 years ago

One refinement on "list an arbitrary number of issues". It's convenient to be able to display a summary of 200/300 issues at least on one single page, similar to bugzilla view.

r12a commented 9 years ago

Whenever i've done this in the past, the set of things that are tracked were determined by the group receiving the comments. Emails with multiple comments were discouraged, but if they arrived they were split into separate issues for tracking and response. Let's here, for the sake of clarity, call these 'reviewItems'. When the same comment was sent by different people, we typically establish a link between them, but still track the originators by having separate reviewItems, so that we can ask them whether they are satisfied. reviewItems are what i think this system needs to manage - this is an abstraction from the actual comments, and can associate various items of metadata with each thing to be tracked.

For that, my requirements include (off the top of my head):

  1. each reviewItem needs a unique id
  2. be able to display and link to the initial comment and the commenter name - and for email particularly to the email that starts the thread
  3. make it possible to quickly view all the discussion related to the initial comment (easy enough in bugzilla, but slightly more complicated when dealing with email, since the email system sometimes breaks what should really be a single thread and you have to pick it up again later).
  4. store information about the status of the comment per such things as: accepted/rejected/out-of-scope/invalid/retracted/deferred, and solution-proposed-to-commenter/not-yet-proposed and commenter-happy/commenter-objects/no-response-yet. Note that these are multiple statuses with alternate values, which should all be visible alongside the other information for a reviewItem.
  5. it should be possible to sort a table of reviewItems by any combination of information/status, etc.
  6. there should be an explicit pointer to the relevant part of the spec unless this is a non-specific comment
  7. there should be a specific pointer to the place where the reviewed WG responds to the commenter, and possibly a separate pointer to their reply
  8. i'd like to be able to show whether the comment led to a spec change, and whether that was substantive or just editorial
  9. it may be necessary to list an assignee (who will track the reviewItem)
  10. for a long list, it would be helpful to filter out items based on status (eg. 'show me just the reviewItems for which we haven't replied to the commenter yet', or 'show me the reviewItems we haven't yet established are accepted or rejected', etc.)
  11. each reviewItem should have a short title, which may differ from the original comment title.

So all the above information needs to be assocaited with each reviewItem in the database, but it should also be possible to automatically produce a list showing all information for the DoC report, but also to assist in managing the process, with colours to reflect status of each comment tracked, and allow for filtering and sorting of the list in various ways. (one very useful sort for working through long lists is by section in the spec)

I'm not sure how important the date and milestone information is, especially in the new process, since all comments need to be responded to anyway.

iherman commented 9 years ago

On 09 Apr 2015, at 16:48 , Robin Berjon notifications@github.com wrote:

Right, but shouldn't a comment be an issue? That's how we did it in HTML.

That is what we did in CSWV, too. Regardless on whether the comment was from the outside or from within the group. Actually (although this was not planned) since we have become 'issue driven' the group has hardly used the mailing list at all; all discussions happened through issues only.

r12a commented 9 years ago

fwiw, i deleted the example from my last comment here's a slightly better example, though not a perfect one, and certainly not sortable http://www.w3.org/International/docs/encoding/encoding-cr-doc.html