Moderation of proposals?

ChrisBarker-NOAA commented 5 years ago

In the discussion of #148, the issue of a moderator was brought up:

https://github.com/cf-convention/cf-conventions/issues/148#issuecomment-437349098

Which refers to theRules for CF Conventions Changes:

The question at hand was whether the "moderator" could be the proposer / a proponent of the proposal, or whether that is a conflict of interest.

I suggest that we should modify rules to make this more clear, an maybe make a change/addition:

In that doc the moderator:

""" The moderator periodically summarises discussion on github, keeps it moving forward and tries to achieve a consensus. It is expected that everyone with an interest will contribute to the discussion and to achieving a consensus during this stage. During the discussion, if an objection is raised, answered and not reasserted, the moderator will assume the objection has been dropped. However, since consensus is the best outcome, it will be helpful if anyone who expresses an objection explicitly withdraws it on changing their mind or deciding to accept the majority view.

The moderator is encouraged to organize conference calls and/or webex-type interactions if this might help resolve an issue more quickly. """

This is a LOT of work I expect that we will be most likely to get someone to do it well if that have some "skin in the game" -- granted, lots of folks have skin in the game in the sense that they want CF to be as good as it can be, but I think someone that really wants the new feature is going to be more invested. And anyone that knows about and cares about CF will probably form an opinion anyway, so a truly "unbiased" moderator is kind of impossible.

So I suggest that the role of the moderator be divided (though it could still be one person, I suppose)

Role 1 (Propoonent?):

The moderator periodically summarises discussion on github, keeps it moving forward and tries to achieve a consensus.

The end result is a document that summarises both the proposal, and the discussion / rejected alternatives, objections, etc.

Role 2 (Moderator):

"attempt to move toward a decision on the proposal by summarising the discussion and indicating the outcome as consensus, near consensus, or not near consensus"

I guess what I'm suggesting is that the development of a final proposal and the guiding of a decion on that proposal be handled a bit differently.

I also think that the document that summarises both the proposal, and the discussion: e.g. rejected alternatives, objections, etc. be maintained in a single place -- could be the initial issue, or better, yet, somewhere else in the repo to be preserved for posterity. See #130 for more on that idea.

martinjuckes commented 5 years ago

Hi @ChrisBarker-NOAA ,

I agree that it should be possible for one person to be moderator and proponent, but there are risks to that approach. To avoid any confusion, could the guidelines suggest that "moderator" posts should be clearly separate from "proponent" or "contributor" posts, so that when a moderator/proponent switches roles it is clear.

Unfortunately, it is possible for discussion to evolve in a complex way, so that maintaining a record in one issue may not be practical in all cases, but I like the idea of using the first entry as some form of summary. Would it help if we asked that the first post be made in the "moderator" role, stating the scope but not proposing the solution, and the "proponent" role, even if it is the same person, should not be started until the 2nd post? We could then have the top issue describing the state of the discussion as seen by the moderator .. is there consensus etc, and the 2nd post could perhaps describe the proponents view of the current state of the proposal.

ChrisBarker-NOAA commented 5 years ago

It is certainly god to clearly define the perspective when someone is poting. However, I think that a challenge to the idea of an unbiased moderator is that anyone that cares enough about CF to be involved will probably form an opinion about a proposal, even if they weren't the original proponent.

Most import in all of this is that do have a single "document" that is updated as the discussion continues. The discussion can get pretty long (and sometimes contentious), and very hard to follow, particular for someone joining in the middle (or late). So the "moderator" or "proponent" or someone really needs to keep an up to date version of one document that lays out:

The problem to be solved
The proposed solution(s)
The pros / cons of the options
if / when a decision is made, a summary of the deciding factors.

This could be the initial entry in an issue -- I'm pretty sure that it can be constantly updated an edited.

If so, like Martin proposed, then it would probably be good for the initial entry to be a first draft of the above doc, and any real advocacy be kept for the second post on an issue.

Though I think it would be better to have a single place for proposals, see: #150 for my ideas on that.

But in any case, we really do need one place that folks can see a summary of the current state of the discussion.

graybeal commented 5 years ago

I like the idea very much of making the first issue both the statement of the problem, and the summary of the discussion.

In GitHub issues any of the comments can be edited, including the first entry.

If a template is created, it can be used for the first ticket and added to as discussion continues. For complex issues this format may be limiting, and you may want to allow instead an 'offline' summary that is pointed to by the ticket. I like #150 https://github.com/cf-convention/cf-conventions/issues/150 for complex issues, because having all those in one place can be quite illuminating.

One caveat: Sometimes the problem definition evolves from what the original writer suggested. Of course Git has a record of the previous versions, but they can be awkward to get to. My suggestion is that if that is happening, someone save a copy of the original problem statement (for example, at the end of the first entry, or beginning of the second entry), so that people who are reading the thread much later can see what the early comments were responding to.

The other place that the decision and deciding factors can go, which would be more 'traditional' for issue tracking systems, is in the comment where the ticket is closed. That's a common place to look for such info.

John

On May 24, 2019, at 6:01 AM, Chris Barker notifications@github.com wrote:

It is certainly god to clearly define the perspective when someone is poting. However, I think that a challenge to the idea of an unbiased moderator is that anyone that cares enough about CF to be involved will probably form an opinion about a proposal, even if they weren't the original proponent.

Most import in all of this is that do have a single "document" that is updated as the discussion continues. The discussion can get pretty long (and sometimes contentious), and very hard to follow, particular for someone joining in the middle (or late). So the "moderator" or "proponent" or someone really needs to keep an up to date version of one document that lays out:

The problem to be solved

The proposed solution(s)

The pros / cons of the options

if / when a decision is made, a summary of the deciding factors.

This could be the initial entry in an issue -- I'm pretty sure that it can be constantly updated an edited.

If so, like Martin proposed, then it would probably be good for the initial entry to be a first draft of the above doc, and any real advocacy be kept for the second post on an issue.

Though I think it would be better to have a single place for proposals, see: #150 https://github.com/cf-convention/cf-conventions/issues/150 for my ideas on that.

But in any case, we really do need one place that folks can see a summary of the current state of the discussion.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/cf-convention/cf-conventions/issues/151?email_source=notifications&email_token=AAJVJUEJLUEBQKQFHP34WBTPW7RJPA5CNFSM4GDQ23NKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWFIF5Q#issuecomment-495616758, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJVJUGPE4K3ZQIENKR5QL3PW7RJPANCNFSM4GDQ23NA.

JonathanGregory commented 5 years ago

I would like to reinforce John's caveat above. I think the initial statement of the problem should remain at the head of the issue. The entire dialogue can be useful in tracing the evolution of the idea and the reasons for decisions, and it needs its starting point.

When ideas are being sketched, it seems to me that it's most convenient to do so in the issue. At that stage there probably is not a single current version. Once the principles have been decided, detailed wording will be debated, for which a pull request is helpful. Since there isn't an automatic link between these, it's important that if someone writes out their precise wording in a pull request, they make a link to it in the issue. I presume that enables the whole thing to be studied afterwards, doesn't it?

Jonathan

ChrisBarker-NOAA commented 5 years ago

My suggestion is that if that is happening, someone save a copy of the original problem statement (for example, at the end of the first entry, or beginning of the second entry)

I think it's REALLY important to have essentially one "document" that is a full summary of the propbel, the solution and the process that lead to that solution. That document should evolve as the discussion occurs, so that at any moment, some can join, and see the current status of the discussion, and when done, it serves as a record of the decision process.

Then, yes the actual text for the CF doc goes in a PR (with a link to the issue).

The other place that the decision and deciding factors can go, which would be more 'traditional' for issue tracking systems, is in the comment where the ticket is closed. That's a common place to look for such info.

yes, but that comment, be definition, can't be written or edited until the issue is ready to be closed, so not a practical place for a summary doc of the discussion-in-progess.

IF we want to keep the summary doc and the discussion in one place, then I think the only option is to have the first post be that doc -- and the actual discussion start in the first comment -- so if there is an "original problem statement" that should be preserved, it goes in the first comment, not the original post.

Again, I refer you to "Enhancement Proposals" as used by the Python development community:

https://www.python.org/dev/peps/

https://www.numpy.org/neps/index.html

https://matplotlib.org/devel/MEP/index.html

Having an archive of all of these is really helpful -- I think we should consider it -- but if not, at least a similar doc in the initial issue post.

graybeal commented 5 years ago

On May 29, 2019, at 2:50 PM, Chris Barker notifications@github.com<mailto:notifications@github.com> wrote:

"""My suggestion is that if that is happening, someone save a copy of the original problem statement (for example, at the end of the first entry, or beginning of the second entry), """

I think it's REALLY important to have essentially one "document" that is a full summary of the propbel, the solution and the process that lead to that solution. That document should evolve as the discussion occurs, so that at any moment, some can join, and see the current status of the discussion, and when done, it serves as a record of the decision process.

For a community like Python where hundreds (or thousands or more) of not-so-technical people need to see a significant number of such summaries, I agree having all those things together and evolving is a nice way to present things.

For a community of CF standard developers—where only a subset of the community access these requests, many of them only monitor or revisit a small subset of the requests, and very few of the community actually read all of them, especially once they are complete—for that group, I think it can be fine for the top ticket to contain the issue that the ticket (currently) addresses, and also the status of the ongoing discussion (updated only now and again); with the 'final answer' showing up at the end of that thread.

In this 'lesser' model, all of the information can be discovered by the very few people who need to discover it in a detailed way (the ticket comments themselves will contain the entire process, so it doesn't have to be massively summarized); all of the core information can be quickly discovered by people who are newly joining the conversation; and ticket maintainers don't have to spend their time constantly updating the current status and process for those very few people who join late. It's really OK in my humble opinion for the late joiners to review the unsummarized part of the thread, that's how everything else in software maintenance works. And Jonathan or anyone can prompt a summary any time, as needed.

Then, yes the actual text for the CF doc goes in a PR (with a link to the issue).

"The other place that the decision and deciding factors can go, which would be more 'traditional' for issue tracking systems, is in the comment where the ticket is closed. That's a common place to look for such info."

yes, but that comment, be definition, can't be written or edited until the issue is ready to be closed, so not a practical place for a summary doc of the discussion-in-progess.

Yes, exactly why that summary comment could go at the end. (I wasn't trying to say the last comment should contain the summary-in-progress.) But I think either place, beginning or end, is OK.

IF we want to keep the summary doc and the discussion in one place, then I think the only option is to have the first post be that doc -- and the actual discussion start in the first comment -- so if there is an "original problem statement" that should be preserved, it goes in the first comment, not the original post.

Either way can be made to work, because the original statement could go at the end of the original post as an addendum.

Again, I refer you to "Enhancement Proposals" as used by the Python development community:

https://www.python.org/dev/peps/

https://www.numpy.org/neps/index.html

https://matplotlib.org/devel/MEP/index.html

Having an archive of all of these is really helpful -- I think we should consider it -- but if not, at least a similar doc in the initial issue post.

I think having a nice template would help organize things in any case. (Maybe a bit too much for some of the simpler proposals, but we could try it.) Then if/when the discussion gets complicated, or we want more visibility for the discussion, the moderator could move the content into a more nicely formatted external document. (Which could even be on the GitHub wiki.) Those nicely formatted documents could form the corpus you want to see for the 'important stuff'—which I agree would be nice—while the nits and details that don't require much discussion could live on as (nicely formatted) issues.

John

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/cf-convention/cf-conventions/issues/151?email_source=notifications&email_token=AAJVJUE7DIZYTEW3SVD5JXLPX33BNA5CNFSM4GDQ23NKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWQXVCQ#issuecomment-497121930, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAJVJUFQVJSXPIGVLIZ24LDPX33BNANCNFSM4GDQ23NA.

======================== John Graybeal Technical Program Manager Center for Expanded Data Annotation and Retrieval /+/ NCBO BioPortal Stanford Center for Biomedical Informatics Research 650-736-1632

castelao commented 5 years ago

By splitting into two roles, the proponent can be unrelated to the CF committee. Thus an important task for the moderator would be to guarantee that the code of conduct is followed during the discussions. It might be worth to be explicit about that in the new description.

JonathanGregory commented 5 years ago

Neither the proposer (proponent) nor the moderator needs to be a member of the CF committee. They should be different people if possible, because it's not so easy to be objective about your own proposal. The main problem with the rules is that people do not often volunteer to act as moderator. The rules are at http://cfconventions.org/rules.html

martinjuckes commented 5 years ago

Hi Jonathan: it is difficult to see how discussions are expected to start with a moderator volunteering ... the rules assume that every "new" proposal has a volunteer moderator, and don't give any advice on how to get to that stage. How are people expected to engage with the community to find a moderator if they need to have a moderator before using the community discussion platform. Perhaps there needs to be a separate area for issues seeking moderation, so that people who have fresh ideas can share them and get some feedback, and also use the platform to ask for moderators. (edited by JBG)

JonathanGregory commented 5 years ago

Dear Martin I don't think it's required for discussions to have a moderator in order to get going, but it helps to have one in place before lots has been said. The rules say, "A member of the conventions committee, or another suitably qualified person, volunteers to moderate the discussion. If no-one volunteers, the chairman of the committee will ask someone to do it." The first case happens sometimes; I don't know whether the second happens. Best wishes Jonathan

castelao commented 5 years ago

@JonathanGregory, you're right. With the split roles concept, I wrongly assumed that the moderator would be necessarily a member of the conventions committee, the first option in the rules for the CF Conventions.

With split roles, I imagined the moderator as someone very familiar with the procedures, that could oversee the proponent without necessarily doing the heavy lift of the proposal, i.e. promoting the discussions, editings, summaries, etc. Someone (the moderator) that would verify if the correct procedure for a proposal is followed, like the timeline; periodic summaries are adequate and all voices were considered; the code of conduct was respected; potentially catch conceptual flaws earlier, ... I'm assuming that the proponent is the most interested in the matter so will probably put the most energy on keeping it moving, while the moderator would guarantee that it is a proper and fair moving. In that case, if the moderator were not a member of the committee, it would be appointed by the committee, thus a trusted one.

I agree that it is not easy to be objective with your own proposal, but to help with that, doesn't need to be a moderator but anyone can join the discussion.

I also agree with @martinjuckes on the importance of assigning a moderator in the early stages of a proposal.

martinjuckes commented 5 years ago

@JonathanGregory : if user X raises an issue and at a later stage Y agrees to moderate it, is it possible for Y to edit the top level entry of the issue? Or is it expected that the moderator completes a review of the current state of the discussion and the proposer then copies that into the top entry? I can see room for confusion when several people are proposing different approaches to a complex problem. This may not be important, but I think we do need clarity about how people will find the latest post from the moderator.

@ChrisBarker-NOAA has raised the issue of the work load for the moderator. Asking someone to present a clear overview of the substance of a discussion which may involve misunderstandings arising from a range of people addressing a complex problem from multiple perspectives is a big ask. This could be simplified by restricting the required contribution of the moderator to a more procedural one: rather than "summarising the discussion", as stated in the current rules, I think it would be more reasonable to ask the moderator to summarise the state of the discussion (e.g. "X, Y and Z have raised objections to the proposal. The proposer has responded, but the objections have not been withdrawn. Y has made a modified proposal, but the proposer has outstanding objections to the revised proposal"). This would be a much easier task for the moderator, and leave the job of merging the original proposal with fresh ideas from the discussion with the proposer.

JonathanGregory commented 5 years ago

Dear Martin Yes, I agree that summarising the content of the discussion is hard, and that may be one reason why few people volunteer to moderate. I feel that the moderator should be like the chair of a meeting, who would summarise the state of the discussion, as you say. If possible, the moderator should direct people's attention to things that have been raised but not settled. I think we should keep the entire discussion in chronological order, including the moderator's summaries at the appropriate places. Does that make it too hard to find the latest summary? You can find it by starting at the bottom and working upwards. We have to make the procedures as simple as possible, to maximise the chance that they will be followed, I feel. Jonathan

martinjuckes commented 5 years ago

Dear Jonathan, It can be difficult to be sure that you have found the latest summary when a discussion has many long entries, so I can see the attraction of having an overview in the first entry. On the other hand, if that is difficult to fit into a simple github procedure, perhaps it is not a priority. If moderator posts are distributed, we could make them easier to find be asking the moderator to start them with a title: "MODERATOR POST [date]".

The difficulty with appointing a moderator early in an ongoing discussion is that there is no clear point at which that should happen, and the discussions tend to attract people who are interested in the content. Perhaps there should be a pro-active effort from the committee or some other group to find a moderator for discussions which are ongoing and not moving to a clear conclusion.

Another option for providing a quick overview of the status of issues is in the github "project" feature. If we had a project for "Convention modification", issues related to that could be tagged as part of the project and the moderator could be asked to provide a one sentence summary in the project note.

regards, Martin

ChrisBarker-NOAA commented 5 years ago

AS far as I can tell, you (or at least the originator of the issue) can go in an edit the first entry of the issue at any time.

So we could use the first entry as the Summary document, which teh modreator and/or the proponent, could keep up to date.

THe real discussion could start with post # 2.

I think we should keep the entire discussion in chronological order, including the moderator's summaries at the appropriate places. Does that make it too hard to find the latest summary? You can find it by starting at the bottom and working upwards.

Please, no! there needs to be ONE summary, in ONE place, that is easy to find -- it could be in a totally different place (I still think we should have "CEP"s: CF Enhancement Proposals #150), but having an updated summary every one in a while scattered throughout the discussion is not helpful.

Using the first post does not mean that the moderator needs to be selected right away either. They can start editing that summary post at any time.

-CHB

martinjuckes commented 5 years ago

@ChrisBarker-NOAA : I think you are right in saying that the originator of an issue can edit the first entry (and the title) -- anyone can edit their own posts.

I can see the advantage of having a summary at the top of the discussion, but I was pointing out above that the moderator can't do this if he/she is not the originator (at least I don't see how this could be done ... you are not allowed to edit other people's posts, for obvious reasons). Another option might be to start by finding a moderator so that they can originate the discussion .. but that raises other problems (see post here).

Hence, it is easy for the proposer to edit the first comment, but, I don't think we currently have a means of making it possible for the moderator to edit the proposer's first comment.

An alternative may be for the moderator, when they take on the role, to start a new issue that they own (or this could be an option, if the moderator feels there is a need for a fresh start setting out how the moderator sees the process). I can see @JonathanGregory point in wanting to be able to follow a simple sequential process when the flow of the discussion is clear.

ChrisBarker-NOAA commented 5 years ago

Hmm -- I wonder if there is a way to shift or share permissions on a post.....

But if not -- my point is that it would be really helpful for there to be one easy to find place that the moderator can keep an up to date summary fo the discussion.

That could be somewhere else: a PR, another issue.

Or maybe the Moderator could use their first post -- and the originator could edit the first post to add a link to the moderator's post?

-CHB

graybeal commented 5 years ago

I just went into the linked comment below ('see post herehttps://github.com/cf-convention/cf-conventions/issues/151#issuecomment-502039237') and edited it. (See end of that comment, just added 'Edited by JBG'.) Sorry for the presumption, it seemed a good test case.

As far as I know, people can edit each other's posts on GitHub. (I presume because, like code, all changes are tracked. Though I have no idea how to revert an edit.) Not sure exactly how much permission is needed, but I've never been unable to edit one when I've tried.

John

On Jun 17, 2019, at 11:19 PM, Martin notifications@github.com<mailto:notifications@github.com> wrote:

@ChrisBarker-NOAAhttps://github.com/ChrisBarker-NOAA : I think you are right in saying that the originator of an issue can edit the first entry (and the title) -- anyone can edit their own posts.

I can see the advantage of having a summary at the top of the discussion, but I was pointing out abovehttps://github.com/cf-convention/cf-conventions/issues/151#issuecomment-502142605 that the moderator can't do this if he/she is not the originator (at least I don't see how this could be done ... you are not allowed to edit other people's posts, for obvious reasons). Another option might be to start by finding a moderator so that they can originate the discussion .. but that raises other problems (see post herehttps://github.com/cf-convention/cf-conventions/issues/151#issuecomment-502039237).

Hence, it is easy for the proposer to edit the first comment, but, I don't think we currently have a means of making it possible for the moderator to edit the proposer's first comment.

An alternative may be for the moderator, when they take on the role, to start a new issue that they own (or this could be an option, if the moderator feels there is a need for a fresh start setting out how the moderator sees the process). I can see @JonathanGregoryhttps://github.com/JonathanGregory point in wanting to be able to follow a simple sequential process when the flow of the discussion is clear.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/cf-convention/cf-conventions/issues/151?email_source=notifications&email_token=AAJVJUEGPMYXKN7EPTRIZGLP3B5AFA5CNFSM4GDQ23NKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODX5J5LQ#issuecomment-502963886, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAJVJUBDTFTNOXMEPGLXT23P3B5AFANCNFSM4GDQ23NA.

======================== John Graybeal Technical Program Manager Center for Expanded Data Annotation and Retrieval /+/ NCBO BioPortal Stanford Center for Biomedical Informatics Research 650-736-1632

ChrisBarker-NOAA commented 5 years ago

I was the originator of this issue -- could one of you go and see if you can edit the original comment? Just to make sure?

martinjuckes commented 5 years ago

Interesting that @graybeal can edit my comments ... I can edit my own comments, but not those of other people

This github article appears to give a clear answer: in order to edit other peoples comments you need "Write", "Admin" or "Maintain" access to the repository. Everyone else can only edit their own issues.

Would it make sense to give anyone who is moderating a discussion at least "Write" access?

mwengren commented 5 years ago

Would it make sense to give anyone who is moderating a discussion at least "Write" access?

As long as you also don't mind them having write access to the repository content as well. Generally, write privileges are granted to a 'contributor' to a repository. Not sure how that role correlates with the moderator role you've been discussing here.

ChrisBarker-NOAA commented 5 years ago

Yes, giving the moderator write access to the repo is a fine idea — maybe we’ve converged on a solution?

martinjuckes commented 5 years ago

I think we have converged on the parameters of a solution, but there are some open issues. @graybeal commented that editing an issue is just like editing comment a piece of code -- but I'm not sure that this is strictly true. As far as I can tell (and I'd be happy to be corrected on this), the comment editing interface offers a limited range of options, and reverting edits is not one of them. In contrast to code edits, people cannot look back at the history of the edited text. If this is true, we should perhaps be cautious about how we use comment editing. There is a risk that some parts of a discussion may refer to text in a comment which is subsequently edited and these discussion segments could become opaque or mis-leading when the text they refer to is being changed.

My question on the technicalities of editing the top comment has been answered, but @JonathanGregory, on June 14th, raised a different objection: a preference for a simple chronologically ordered discussion.

A compromise option may be offered by a suggestion from the markdown formatting guide, which suggests keeping a task list in the first comment of an issue. A task list might look like this (perhaps with a few more issue specific items added by the moderator):

TASK LIST

[x] Scope of the problem agreed [moderator could link to a comment summarising agreed scope];
[x] Practical uncertainties around different proposals clarified;
[ ] Agreement that a repository branch should be created to reflect the latest state of the proposal;
[ ] Agreement to launch a pull request;
[ ] Pull request accepted;

More detailed moderator and proposer comments on each stage could be placed in chronological order, with links inserted in the task list which the moderator could maintain in the first comment.

@JonathanGregory has referred to the preparation of a pull request, and the fact that discussion of technical details is better supported there. The current rules skip over the intermediate stage: the creation of a branch from which to launch the pull request. I've put this stage explicitly in the task list, as it may be useful at some stage to create a branch and continue the discussion on details of the text in the branch before the final decision launch a pull request. The editing options in the branch are richer than in the issue comments, so it is perhaps a better place to keep the current state of the proposal. This would preserve the chronological nature of the discussion (I agree with Jonathan that there is significant value in this) and ensure that the editing of the proposal details is done in a way which is traceable (with history, reversion options, etc).

ChrisBarker-NOAA commented 5 years ago

“”” raised a different objection: a preference for a simple chronologically ordered discussion. “””

The chronological discussion would still be fully captured in the issue comments — but thinking that that is “simple” is very optimistic.

That’s the entire point of having a “current state of the discussion” summary in the first post the entire issue thread can become extremely unwieldy.

--

Christopher Barker, Ph.D. Oceanographer

Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

ChrisBarker-NOAA commented 5 years ago

the PR is the right place to create and discuss the actual text that will go in the document.

But there is a lot of discussion to capture before getting to that point.

-CHB --

Christopher Barker, Ph.D. Oceanographer

Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

taylor13 commented 5 years ago

I have not studied all of the comments, but I would like to support @martinjuckes suggestion of an index at the beginning providing links to pertinent summary comments (and perhaps other important things). Something along the lines (nb. these are not real summary comments, but just illustrating the formatting):

Summaries provided at:

https://github.com/cf-convention/cf-conventions/issues/151#issuecomment-497133445 (by @graybeal 5/19/19) https://github.com/cf-convention/cf-conventions/issues/151#issuecomment-503467718 (by @martinjuckes 6/19/19)

When new summaries are written, they would be added to the index. The latest summary should repeat relevant information that might appear in earlier summaries (and eliminate irrelevant material). None of the comments appearing under the issue would be edited for this purpose (except the first one with the list of summaries). This would maintain a clear record of how thinking has evolved in considering a change in the convention.

ChrisBarker-NOAA commented 5 years ago

@taylor13 's proposal would work, yes.

But as a user, I'd much rather have a single document, somewhere, that summaries the current state of the discussion (and nicely, would then summarize the final state as well for posterity.

I also fail to see the advantage to having a pile of summaries mixed in with the chronological discussion -- would each of these summaries summarize teh entire state of the discussion? or would each be a note about a single point of contention?

That is, could someone new to the discussion be able to go to the last one, and get what they need to know about the current state of the discussion?

Question for all:

Do folks not agree with my idea of having a single summary document that evolves as the discussion evolves, eventually being a final summary of the discussion and decisions? if so, then the debate is where to put it and how to manage it. But if folks don't see the utility in having such a single document, then yes, some sort of index into the thread would be adequate.

-CHB

martinjuckes commented 5 years ago

Hi @ChrisBarker-NOAA ,

I don't think it is feasible to maintain a current summary of a discussion in which, often, people are struggling to reach a common understanding. Karl's suggestion reflects the fact that different people might review the same discussion and arrive at different interpretations of the what it means. If there is no consensus, anyone coming to the discussion will need to read the views of several people.

In my view, it would be useful for a moderator to add links to significant contributions (and perhaps add two or three words about why they are significant) and, if the discussion moves on, perhaps remove links to comments which have been superseded.

I also feel that we should avoid editing content if we don't have a mechanism for tracing the history of the edited document. This issue/comment interface is designed to show the history of a discussion through a sequence of comments, the repository is designed to show history through a system of tracking changes to a document. If a discussion gets to the point where there is sufficient agreement to start editing text, then I think that editing should be done in a branch of the repository. If a discussion is complex there are likely to be further clarifications needed as it is translated into a revised document.

Hence, I feel that the disadvantages of trying to maintain a summary of the discussion in top comment outweigh the advantages.

ChrisBarker-NOAA commented 5 years ago

I think I haven’t been clear about what I mean:

I don't think it is feasible to maintain a current summary of a discussion in which, often, people are struggling to reach a common understanding.

Then the “current” state of the discussion Is that there is confusion about a particular issue: it is the job of a moderator to identify the confusions or disagreements in order to move the conversation forward — I’m suggesting that these identified issues are easy to find.

Karl's suggestion reflects the fact that different people might review the same discussion and arrive at different interpretations of the what it means.

Indeed — another good reason for a clear single place to capture the state of the discussion. The moderator might think consensus has been reached about a particular issue — if they write that down somewhere, then if others disagree, they can say so. If that is buried in the thread, folks may not see it at all, and you will have some folks thinking consensus has been reached, when it has not.

If there is no consensus, anyone coming to the discussion will need to read the views of several people.

Of course they will — which is why the entire thread is there, and the summary will ideally have links to particular comments in the thread.

In my view, it would be useful for a moderator to add links to significant contributions (and perhaps add two or three words about why they are significant) and, if the discussion moves on, perhaps remove links to comments which have been superseded.

Yup — that would be a summary of the discussion :-)

I also feel that we should avoid editing content if we don't have a mechanism for tracing the history of the edited document.

In general, I totally agree, but in this case, the “real work” will have been done in the discussion thread — my idea of the “current state of the discussion” doc is inherently ephemeral— so OK not have under version control.

In a way, it would be what is in the moderator’s head — but written down for all to see.

. If a discussion gets to the point where there is sufficient agreement to start editing text, then I think that editing should be done in a branch of the repository.

If the “text” is the text of the CF standard, then yes, absolutely.

But there is a lot of “meta discussion” that it would be good to capture. The CF doc specifies what to do, but it is not a place to explain in any detail why it is being done that way, what other options were considered, etc.

I would like to capture that — a potentially huge, freewheeling thread in an issue is not the way to do that.

That doc could be written after the fact, but 1) I’m not sure anyone would get around to it, and 2) a draft of such a doc could be very helpful in guiding the discussion.

Which brings us back to my idea of having “CF Enhancement Proposals”, which would be developed in a versioning system, just like you want :-)

Hence, I feel that the disadvantages of trying to maintain a summary of the discussion in top comment outweigh the advantages.

Are you proposing another place to maintain a summary? I fail to see how having no way to figure out the state of the discussion without reading the whole darn thing is helpful.

Consider the recent attempt to address Calendars, UTC, leap seconds, etc. There was a lot of circular discussion, and it ultimately fizzled out without a conclusion.

Sure, a moderator would have helped, but I dread picking up that discussion again without a way clearly communicate what has already been discussed and hashed out.

-CHB

martinjuckes commented 5 years ago

@ChrisBarker-NOAA : are you saying that what @taylor13 proposed four days ago is consistent with what you think of as a summary (above, you say "Yup — that would be a summary of the discussion")?

I had though you were arguing for a more detailed piece of text, but, if you agree with what Karl suggested, we may be close to resolution.

ChrisBarker-NOAA commented 5 years ago

@ChrisBarker-NOAA https://github.com/ChrisBarker-NOAA : are you saying that what @taylor13 https://github.com/taylor13 proposed four days ago https://github.com/cf-convention/cf-conventions/issues/151#issuecomment-504109227 is consistent with what you think of as a summary

Not quite. I understood that idea as being for having a set of summaries in chronological order, interspersed with the comments.

Then the first comment would have links to those summaries.

I am suggesting that the first comment be a readable up to date summary on its own, with links to the relevant comments for further detail.

Wouldn’t it be nice if had that for this thread ;-)

-CHB

martinjuckes commented 5 years ago

So, to be clear, I'll repeat: I am opposed to this idea (and I don't think it would help in this discussion), and I support what Karl proposed.

ChrisBarker-NOAA commented 5 years ago

OK then -- let's give it a try and see how it goes.

The real limitation will be getting a moderator that can find the time to do this right, no matter the mechanics.

Note that it would be good if someone wrote up the result of this discussion for posterity, and maybe inclusion in the CF rules for changes doc:

http://cfconventions.org/rules.html

martinjuckes commented 5 years ago

I'd be happy to update the document, but let's see if we have, within the terms of the current rules, got a level of agreement which justifies going on to the next step.

The moderator, if appointed, should be "in charge of the discussion and make[s] sure that it is conducted in a fair and organized way" (Collins dictionary);
The moderator may/is encouraged to provide links to key milestones of the discussion in the top issue of a discussion;
Participant's in a discussion are expected to respect and comply with requests from the moderator (e.g. to suspend discussion on one topic until another is resolved or to answer specific questions from the moderator);
The individual acting as moderator may also act as a participant, but should clearly distinguish between the two roles;
The moderator should oversee the implementation of proposed changes which are agreed and close the issue when this process is complete.

There are some things in http://cfconventions.org/rules.html peripheral to moderation which appear to be generally ignored and unrealistic. Pull requests are not mentioned. The current rules imply that once something is agreed in a discussion it will be a mere formality to update the standards document. In practise, I believe, there often needs to be further discussion of the text itself. Finality only really comes when the pull request is accepted and the changes merged into the master branch.

As I said, this is somewhat peripheral to the topic of moderation, so I have worded point 5 above to be neutral about the actual process which leads to agreed changes being implemented.

If there is support for the above (3 people expressing support, no objections), I'll generate a branch and move towards a pull request. @taylor13 , @JonathanGregory , @graybeal , @ChrisBarker-NOAA, @castelao : do you agree that it is worth generating a branch and updating http://cfconventions.org/rules.html based on the above?

JonathanGregory commented 5 years ago

Dear Martin Thanks for drafting these guidelines for moderation. I agree this would be useful to incorporate in the rules. I also agree that providing links to key milestones is a practical way to help people navigate the discussion, and not as hard or subjective as summarising it. Jonathan

taylor13 commented 5 years ago

thanks, Martin for these guidelines, which I support. I have a question. Are we agreeing that all potential discussion "moderators" should be given "write access to the repo"? If so, we will need to obtain a list of the moderators and make certain that they understand that they should only edit their own posts and the first posting of an issue. In editing the first posting, they can enter summary text at the beginning of the post (and subsequently edit that), but they should not alter any text provided by the person opening the issue.

graybeal commented 5 years ago

It's all good but for this nuance: I can't figure out what "provide links to key milestones of the discussion" should be. I would not like to see a list of pointers to different comments in the summary, what I'd like to see are (a) points of agreement, (b) main points of discussed disagreement, (c) issues raised but not discussed/resolved (e.g., a question "What about case X?" that no one responded to).

If the moderator deems it helpful for any of those items to include a link to comments, especially for item (b), that's great. But the summary will mostly be sufficient, and it is very painful to link to every milestone on top of all the other work the moderator has to do. I and other participants (of whom there are rarely a large number) can search for a topic if I need to.

ChrisBarker-NOAA commented 5 years ago

hmm -- kind of sounds a lot like what I was proposing :-)

We need to face it -- well moderating a discussion is alot of work -- but maybe we should leave it up t the moderator exactly how to do it.

But I do think we should specify that the the first comment in the issue should be used for the moderators summary -- however that particular moderator chooses to use it.

As for permissions -- we don't need to assign permissions to all 'moderators" ahead of time - that can be done on a case by case basis.

martinjuckes commented 5 years ago

Dear All,

thanks for those response. I may be suffering from optimism bias, but I think I see a route to a solution in what John has suggested.

We obviously have different perceptions of what is easy and what is difficult -- and this may be influenced by specific discussions that each of us were involved in. Chris has referred to #148 -- for which my moderator summary might be:

Jan. 16th : Karl proposing addition of one new calendar (utc) [NOT ACCEPTED];
- Jan 18th : Jim summarises an alternative; [NOT ACCEPTED]
- Jan 28th : Martin revised proposal; [NOT ACCEPTED]
- {NB: for illustrative purposes only}

In this case the range of issues being discussed is quite large and I feel it would be unhelpful for a moderator to attempt to explain the points of disagreement. There are many questions where a response has been given and not accepted. Linking to the posts in which people have tried to put forward a review of the status is, as far as I can see, the best that can be done. John is worried that there may be too many "milestones" -- but I think we should be able to leave it up to the moderator to decide on what is worth listing. This goes a little further than Karl's suggestion, but avoids getting into summary of content.

I'm keen to avoid the idea that the moderator should summarise or rephrase text (in the above example, Karl's post is 1,695 words -- trying to have a meaningful summary without making subjective choices about priorities is not really realistic). When people join the discussion, they need to base their comments on contributions of other contributors (the moderator text will evolve, so referring to it will be unhelpful).

In simpler discussions it may be possible, as John suggests to state points of agreement and disagreement.

Item 2 could be rephrased as:

The moderator may/is encouraged to list major points of agreement and contention, and, if appropriate, provide links to key milestones of the discussion in the top issue of a discussion;

Moderator comments such as "Can we discuss issue X now?": I think these belong in the flow of the discussion. If the moderator wants to pose a question it will generally need some elaboration, and that elaboration is best placed in the sequence of comments that set the context for his comments. This is a very important part of the moderator role, which I tried to cover in point 3 in my last post ... though I didn't say how the moderator should to this. Instructions such as this could act as "milestones" in the summary, e.g.,

[June 25th] : start of discussion on X

Similarly, the moderator may seek to summarise the points agreed on and ask people whether they agree with that summary. Again, I believe such comments, which may be long and detailed, belong in the flow of the discussion. It is also optional (too much work to require from a volunteer) ... a moderator may prefer to ask another participant or to wait until someone shows some initiative. @ChrisBarker-NOAA refers to this a "proponent" role in his first post : we agree, I think, that it is not a required part of the moderator role. Can we agree that, if the moderator chooses to provide an extensive summary, then, it should be in the flow of the discussion?

@graybeal , @ChrisBarker-NOAA : would you agree to guidance stating that moderator comments which request a change in direction of the discussion ("Please give me your views on Y?"), or ask questions about the current state of the discussion ("Have we agreed X?"), should be placed in the flow of the discussion rather than in the top comment? It is important that the connection between responses to these questions and the questions themselves is preserved, and this will work better if they are in the flow of the discussion.

Point 3 could be revised to:

Participant's in a discussion are expected to respect and comply with requests from the moderator (e.g. to suspend discussion on one topic until another is resolved or to answer specific questions from the moderator). Such requests from the moderator, with appropriate explanations, should be placed in the flow of the discussion.

On @taylor13 's question about access: this does imply that all moderators should have access at the point when they start moderating. I agree that there do need to be rules about acceptable use of the authority that this gives them ... but presumably these would be rules applying to all people who have write access. It might help to have a list of the people who have this access available.

On the specific issue of what should be edited in the top comment, we could guide this by updating the template for change requests, which currently has sections for Title, Moderator, Requirement Summary etc. If we add, after Moderator, a section for Status Summary (by Moderator) it should be clear that the updates will be confined to that section.

graybeal commented 5 years ago

I'm OK with just about everything Chris and Martin said, with the caveat that we shouldn't overspecify/overdesign now what moderators 'have to do'. Yes require the summary go in the first comment if you want (you'll have to pre-allocated that comment when the ticket is created), or at the end of the original post. Trust the moderators to not mess up the process, so offering some suggested approaches are fine, but let's leave some room for appropriate adjustments.

ChrisBarker-NOAA commented 5 years ago

@graybeal wrote: "we shouldn't overspecify/overdesign now what moderators 'have to do'."

+1 on this.

I could go into detail about how I think moderation should be done -- but: a) each proposal is different, and requires different levels of moderation b) each moderator is different, and it is a volunteer role -- so they should be free do perform those duties as they see fit.

So other than specifying that that first comment can be used by the moderator to summarize the discussion (with links) and they see fit, I think we are done.

And we'll have to see how it goes, and can adjust if need be in the future.

But we should not forget #148 -- I think that was really a failure of process, not a failure of ideas -- and it is not resolved, which really is too bad. :-(

martinjuckes commented 5 years ago

Hi @ChrisBarker-NOAA , I'm sure we all agree that we should not "overspecify/overdesign". Nevertheless, Karl, Jonathan, Guilherme and John have all expressed support for the specific set of rules which I've outlined, which I think implies that they don't consider them excessive.

You would clearly like to give the moderator the option of providing a more extensive review of the discussion in the top comment, but 3 people in the discussion (including myself) have raised objections to that. Some would prefer to exclude editing of the top comment completely. I'm suggesting a compromise in which editing of the top comment is confined to brief remarks about the state of the discussion. The text on the use of the top comment is 30 words long -- do you really think this is excessive?

The moderator is not the only one who is a volunteer: the process relies on contributions from a range of people engaging the discussions.

I agree that progress on #148 has been frustratingly slow ... but it is a topic which has been around for years. It would obviously be useful if someone came along with the ability to understand all points of view and set it out clearly for everyone, but we shouldn't be asking that of a moderator.

ChrisBarker-NOAA commented 5 years ago

just a small comment about a comment :-):

""" I'm keen to avoid the idea that the moderator should summarise or rephrase text (in the above example, Karl's post is 1,695 words -- trying to have a meaningful summary without making subjective choices about priorities is not really realistic). """

I am completely confused why anyone would think there is a downside to providing a summary for a 1,695 word post -- do we actually expect newcomers (or even folks that missed a day or two) to go and read multiple multi-thousand word posts in order to get up to speed? If the moderator mischaracterizes that long post, the OP can help clarify it. And at the end of the process, we really should have that compact summary, or everyone will continue to second-guess the whole thing.

That being said, what I think is on the table now is fine -- there is room for the moderator to provide a summary, and they can decide how best to do that for the given example.

""" Some would prefer to exclude editing of the top comment completely. I'm suggesting a compromise in which editing of the top comment is confined to brief remarks about the state of the discussion. """

Either the top comment is used as SOME sort of "Keep up to date on the state of the discussion" or not. I would highly prefer it were, because that should be somewhere easy to find. It can't be the last comment, because we never know when a comment will be the last.

Anyway, let's go with Martin's text, and see how it goes.

-CHB

martinjuckes commented 5 years ago

For those of you who are not following the CF email list, note there a related discussion, which touches on moderation of proposals, has appeared there: http://mailman.cgd.ucar.edu/pipermail/cf-metadata/2019/020975.html

dblodgett-usgs commented 5 years ago

At the NetCDF workshop in Tacoma, we determined that the moderator for an issue should be assigned to the issue and should be someone with history with CF and a goal to find consensus and come to conclusion (close the issue!). We will firm up this definition in the process of #172. Going to close this issue and we will refer back to the discussion.

martinjuckes commented 5 years ago

@dblodgett-usgs : Hi Dave, I'm not sure how the decision at Tacoma on the selection of a moderator relates to this discussion on the process of moderation. Can you clarify?

We had provisional agreement in this discussion on text relating to the process of moderation.

dblodgett-usgs commented 5 years ago

Dear @martinjuckes : Apologies, I overlooked that there was near consensus here and should not have closed the issue so hastily.

The discussion is very lengthy and could use a top-level summary of the proposal as something of a "current version" motion for us to consider. Can text for the candidate document(s) be prepared at this stage? Maybe @ChrisBarker-NOAA wants to update the issue using the issue template and the latest version of the proposal? Or are we far enough along to open a pull request?

martinjuckes commented 5 years ago

Hi @dblodgett-usgs : I have updated the two relevant documents and could submit a pull request ... I'm finishing a proposal today, but I may get to submitting the pull request this afternoon or on Monday morning,

regards, Martin

erget commented 5 years ago

Just adding my support here, keeping a summary in the top issue is a good way for everyone to stay sane. That's also the reason that I read a good deal of the comments here but by far not all ;)

davidhassell commented 4 years ago

Hello @ChrisBarker-NOAA,

It'd be very useful if you briefly summarize this issue https://docs.google.com/document/d/1urPWngzDCuHTrfpA8nedGoRDVKXs5OmjqO8M6i3UZJM/edit#, including what might be good outcomes from a discussion at the CF meeting. If this could be done today or tomorrow that would be best, as we will use it to help people decide on which sessions to attend in advance of the meeting.

Many thanks, David

cf-convention / cf-conventions

Moderation of proposals? #151

TASK LIST

[ ] Pull request accepted;

{NB: for illustrative purposes only}