search

cf-convention / cf-convention.github.io

sources for website cf-conventions.org
cf-convention.github.io
Creative Commons Zero v1.0 Universal
35 stars 45 forks source link

Refactor "rules" #102

Open erget opened 4 years ago

erget commented 4 years ago

The idea here is

  1. Update "rules" to point to various CONTRIBUTING.md files for each repository.
  2. Migrate the content of "errors" into "rules"
  3. Spruce up rules for making changes to the CF Conventions in "rules", adding an activity diagram so that it doesn't seem so scary.

This supersedes the actions on @erget in #94 so that they can be tracked here, separate from the others.

Originally this issue was also intended to add CONTRIBUTING.md to this repository. This was done in f9ee1000cc4959aaad48b83cc75e189c751ac181 but now is being pursued in #121 and an associated issue (pending creation at this time).

davidhassell commented 4 years ago

Hi Daniel, This sounds good to me. Just to clarify, I presume that you mean add to a link from the "rules" to the CONTRIBUTING.md, rather than replace the body of the rules with a link?

erget commented 4 years ago

Yes, I mean add a link from "rules" to CONTRIBUTING.md (which must be created). I'm putting together a WIP PR right now so that the changes are visible; I won't merge until a few people have weighed in and said they like it.

And to be clear I also don't intend to change any of the content of "rules" - only to clarify.

erget commented 4 years ago

To all those interested - I've made my changes and you can see them deployed to this address if you like to see things live. Looking forward to hearing your feedback!

twhiteaker commented 4 years ago

The rules state that after three weeks of no contributions, the moderator summarizes the status, and after another three weeks without comment, the change can be accepted (with consensus, etc.), for a total of six weeks plus discussion time. However, cf-conventions CONTRIBUTING only mentions one three week period. These two documents should be aligned with each other.

erget commented 4 years ago

@twhiteaker that's correct - as "rules" is canonical I think a correction of cf-conventions CONTRIBUTING is in order, even if I am in favour of revisiting the dual-3-week rule. I've revised the text in CONTRIBUTING in PR https://github.com/cf-convention/cf-conventions/pull/256/

We should discuss this at the Community Meeting in 2 weeks.

ethanrd commented 4 years ago

Enhancements can apply to the CF Conventions themselves or to this website.

Does this sentence imply that changes to the website would operate under the same rules as the conventions document?

Other than the constitution, rules, and maybe a few other pages, the level of scrutiny for a convention document does not seem necessary for the content of a website. Perhaps instead we should simply require review/approval by some number of members of the committees, governance panel, or information management support team (basically, anyone with write permission to the GitHub repos).

ethanrd commented 4 years ago

@erget - Sorry, I just noticed that the CONTRIBUTING.md file in the PR corresponding to this issue follows the "approved by N members of the committee or team" approach. So maybe the mention of the website in the rules.md file just needs to be removed?

erget commented 4 years ago

@ethanrd good eye, I've fixed this in 60416e9.

JonathanGregory commented 4 years ago

Dear Daniel @erget

Thanks for your work on the rules. You've rearranged a few things and I think you've improved its clarity, from comparing the old and new versions side by side (since the rearrangement is too radical for the rich diff to help). I have a few comments.

See you later.

Jonathan

JonathanGregory commented 4 years ago

I suggest that we should use labels on issues in this repository to indicate how they ought to be handled, and I think that's something which would naturally belong in CONTRIBUTING, so I hope it's OK to raise it here. At the moment the labels available are: bug duplicate enhancement invalid question wontfix. Is this a standard set? It doesn't seem appropriate for this repository. I would suggest

I don't think questions or discussion not aimed at changes should take place in this repo. That's a purpose of the discuss repo.

With that classification, I would propose that

Finally, that would mean that membership of the Information Management and Support team has a particular function associated with it. This means (a) we should keep it up to date, add new people as appropriate, and move people to the "former" list below if they are no longer active, (b) consider whether to add some members of the governance panel and conventions committee to this list. The governance panel decided to create this list on the website in order to acknowledge the help given by those people, but without a particular function that I'm now proposing.

davidhassell commented 4 years ago

@JonathanGregory I like your label definitions, thanks.

Should enhancement issues be raised first in the discuss repo (as we have advocated in the past), and then only raised in the cf-conventions repo if they are clearly heading towards a PR? Or, should the discuss repo be reserved for questions, announcements, general discussion, etc.?

The former has the advantage that some issues that don't make it that far will not clutter up the cf-conventions repo, but also the disadvantage that the discussion is split between two repos. This was ever the case, however, with the old e-mail list and Trac.

Whichever is the right way, we should be clear in the description, and respectfully move issues that have been raised int he wrong place.

JonathanGregory commented 4 years ago

@davidhassell, I think I prefer the former i.e. start something in discuss if you're not sure you have a proposal to make. Related to this, a point I forgot to make is that I think that the rules for changing the convention are a subject for this repo, and not the conventions repo - though I'm not suggesting the ongoing discussions should move here. The conventions repo is concerned with the contents of the convention, not the rules or the website, I think.

erget commented 4 years ago

I'm answering inline for brevity :)

From https://github.com/cf-convention/cf-convention.github.io/issues/102#issuecomment-641318312:

  • The diagram is good, but could it be a bit smaller, so that the start of the text is also visible on a reasonably sized browser window? The rules are the primary thing, and the diagram is a useful illustration, but at present the reader wouldn't see the rules upon arrival. Maybe you could make it a pasted-in image or something that pops up?
  • The diagram needs something to represent the new text in http://cfconventions.org/rules.html about the data model.

Yes, I'll do both of these things next week.

  • The paragraph about remedying mistakes in the convention is absent, unless I've missed it. Although this hasn't yet happened, it's important: If the change, once implemented in the conventions, subsequently turns out to be materially flawed, meaning that data written following the convention could be somehow erroneous or ambiguous, a github issue should urgently be opened to discuss whether to revoke the change. If this is agreed by a majority of the committee, a new version of the conventions will be prepared immediately, with the second digit of the version number incremented, and will be recommended to be used instead of the flawed version. The flawed version will be deprecated by a statement in the standard document and the conformance document. However, any data written with the flawed version will not be invalidated, although it may be problematic for users. Errors or lack of clarity in wording, when the convention itself is not at fault, can be corrected by defect tickets on the usual schedule. Please could you put this back somewhere? Maybe it would be appropriate to give it its own heading e.g. "Correcting mistakes in the convention".

I would like to discuss this more - I believe that this is possible under the existing rules for correcting "Errata" and had removed it because of my fanatical love of short documents. Do you see a deficiency in the "Errata" section that would preclude using it to this end? Or do you think that it should be in the nominal path for implementing changes (I don't but I'm happy to be convinced otherwise!)?

  • I suggest that "Errata" sould be called "Correcting defects in the documents", which would contrast with my above suggestion, and uses the word (defect) which we use to label this kind of issue and in the rules about them.

Perhaps "Errata and defects" and then differentiate between the 2 cases? In this I'm contradicting myself and somewhat agreeing with your previous comment, but although I understand how you're classing these differently I lean toward lumping them together as a single category.

From https://github.com/cf-convention/cf-convention.github.io/issues/102#issuecomment-641346514:

I suggest that we should use labels on issues in this repository to indicate how they ought to be handled, and I think that's something which would naturally belong in CONTRIBUTING, so I hope it's OK to raise it here. At the moment the labels available are: bug duplicate enhancement invalid question wontfix. Is this a standard set? It doesn't seem appropriate for this repository. I would suggest

  • governance: to alter anything of substance concerning the constitution of CF, its committees, the rules for altering the convention, and the ways in which CF is maintained and published (including GitHub and the website)
  • enhancement: to improve the website regarding its usefulness and presentation
  • update: to change the text on the website in a way which reflects decisions already made by the governance panel or the committees (or for simplicity this could be labelled as defect, although it's not a mistake)
  • defect: to correct a mistake in the text of the website to make it consistent with the agreed intended content
  • typo: to correct a trivial mistake (a defect so minor that no comment is needed in the issue, but a pull request would say it all).
  • GitHubProblem: to correct a technical problem with GitHub and its production of the website
  • duplicate: to indicate that an issue duplicates another (and should I suppose be closed as redundant)

I don't think questions or discussion not aimed at changes should take place in this repo. That's a purpose of the discuss repo.

Fully agreed. Another thing I've noted for next week.

With that classification, I would propose that

  • governance issues should be decided like conventions enhancements issues, with the governance panel in the role of the conventions committee i.e. the three-week rule and number of assenting members.
  • enhancement and GitHubProblem issues should be decided like conventions enhancements issues, with the Information Management and Support team (see http://cfconventions.org/governance.html) in the role of the conventions committee.
  • all other issues should be treated like conventions defects i.e. accepted by default after three weeks.

I really like that. Will do.

Finally, that would mean that membership of the Information Management and Support team has a particular function associated with it. This means (a) we should keep it up to date, add new people as appropriate, and move people to the "former" list below if they are no longer active, (b) consider whether to add some members of the governance panel and conventions committee to this list. The governance panel decided to create this list on the website in order to acknowledge the help given by those people, but without a particular function that I'm now proposing.

That's reasonable, of course we should talk with IM&S in order to clarify that. I think we can fairly unilaterally grant them powers but we should send out a message outlining our intent behind it - we're not obligating them to always jump up and participate, just as the Conventions Committee participates voluntarily.

From https://github.com/cf-convention/cf-convention.github.io/issues/102#issuecomment-641364635:

@JonathanGregory I like your label definitions, thanks.

Should enhancement issues be raised first in the discuss repo (as we have advocated in the past), and then only raised in the cf-conventions repo if they are clearly heading towards a PR? Or, should the discuss repo be reserved for questions, announcements, general discussion, etc.?

@davidhassell - I'm with @JonathanGregory in that I think using discuss is fine if you're not sure if you have a proposal, but if you already know you have one you can go straight to proposing.

JonathanGregory commented 4 years ago

Dear Daniel @erget Thanks for your replies. The only outstanding one is that about mistakes in the convention. The paragraph I quoted (from the rules) maybe doesn't make the distinction clearly enough. A defect is the case where the text of the convention doesn't say what we intended. A mistake is where the text is correct, but the change which was made in the convention introduces a problem. This could happen for instance if the convention, when followed correctly, leads to files being written whose metadata cannot be interpreted correctly or unambiguously. To minimise the damage caused by erroneous files being produced and archived, and thus giving a long-term problem, this kind of situation must be corrected as quickly as possible. That's why we agreed the procedure for deprecating the version of the convention which contains the mistake, and producing a new corrected one straight away. Defects, on the other hand, can wait to be corrected to the next usual opportunity. So far, a mistake has never been made of this kind, but it could happen! Of course, the exhaustive and careful discussion we usually have is a good protection against writing faulty conventions. Does that make sense? That's why I believe we need this text still. Jonathan

erget commented 4 years ago

Dear @JonathanGregory, What you're saying about the possibility of quickly deprecating and making a change makes total sense to me. However, I believe that this capability is afforded by the current rules anyway. Although we intend to release roughly yearly, we are now taking care of the Conventions in such a way that allows us to potentially "ship" a new release at almost any time. Furthermore, under the current rules, any file produced using a version of the Conventions would still be valid, even if we deprecate a practice used in that file in the future version of the Conventions - we simply don't have a mechanism to invalidate existing files that comply with any version of the Conventions (and I think this is a good thing!).

What I'm trying to say is that I agree with you about the approach to use, but I think that writing it down in that detail is not necessary because there's nothing that would prevent us from doing that even without this text. Therefore I would favour brevity by leaving the text out - shorter documents are more likely to be read and understood in their entirety.

Despite the fact that I am often very opinionated (as you know 😉) in this case I don't feel the need for protracted discussion on this topic - if my reasoning in this comment hasn't brought you round to my point of view I'm happy to re-insert the text and leave it at that. Just let me know.

JonathanGregory commented 4 years ago

Dear @erget

What you say makes sense as well, but of course, I want to convince you just as much as the reverse! The spirit of CF is consensus, not majority rule or veto. The special rule was decided by the conventions committee, as recorded in https://cf-trac.llnl.gov/trac/ticket/146, as a safeguard because we were removing provisional status. You are right that we don't need a special procedure to make a new release. The difference from a defect is that, if we think the convention itself was mistaken, then any file written according to that version (as indicated by its Conventions attribute) is potentially erroneous. We did explictly say we aren't invalidating those files (which, I agree, is a good thing), but we are signalling that they might be problematic to use, and we are telling people not to use the flawed version of the convention to produce any more files.

This doesn't apply to defects. A defect doesn't mean the convention is wrong, just that the text was wrong, and usually it's harmless. It might not be harmless, for instance the case that @martinjuckes has raised in https://github.com/cf-convention/cf-conventions/issues/274; in that case, the lack of clarity, which needs to be remedied, means that some files could have been written with incorrect syntax, but it wouldn't prevent them from being interpreted correctly if the software was tolerant. The convention didn't have a flaw in the case, only the document was faulty. When there is a only a defect in the document, there's no need to deprecate that version of the convention.

Is that any more convincing? Best wishes Jonathan

JonathanGregory commented 4 years ago

After a bit more thought, I'd like to modify my label proposals slightly. Above, I wrote

enhancement: to improve the website regarding its usefulness and presentation

enhancement and GitHubProblem issues should be decided like conventions enhancements issues, with the Information Management and Support team (see http://cfconventions.org/governance.html) in the role of the conventions committee.

Since enhancements might involve both content and technical issues, and the content could relate to governance issues or the convention, I'd now like to suggest:

enhancement issues should be decided like conventions enhancements issues, with the governance panel, CF committees, and Information Management and Support team all together in the role of the conventions committee.

meaning that the two people who must express support could be from any of this combined group. I think it's fine to assign GitHubProblems to the Information Management and Support team alone, since they're most competent. Of course, anyone can contribute to any issue, as usual. The requirement for support by people in the designated groups is intended as a guarantee that the proposal is consistent with the general CF ways of doing things.

Jonathan

JonathanGregory commented 4 years ago

Another small point about the rules and CONTRIBUTING, in the new world of GitHub. I think that "express support" for an enhancement should mean saying so in a comment on the issue. I don't think adding a thumbs-up to a previous comment should count as explicit agreement.

davidhassell commented 4 years ago

I agree with Jonathan's point about thumbs-ups

JonathanGregory commented 4 years ago

This issue would benefit from having a moderator, ideally not me since I've contributed substantially.

erget commented 4 years ago

Yes, optimally I think this should be somebody from the @cf-convention/gov-panel because it potentially produces changes that would need to be vetted by them. To be clear however there are still some outstanding items to be completed before we can consider merging it. One thing I intend to do when I find the time, probably next week, is to separate out the CONTRIBUTING.md for this repository into its own PR so that it's clearer how to make changes to the website, as a number of people are interested in contributing such changes. That way that can be adopted without being slowed down by the changes to RULES.md.

I also intend to respond to your comments from last week (ff.) - I just haven't had time yet :)

mwengren commented 4 years ago

@JonathanGregory thanks for the mention to comment about the label proposal over in this issue.

I think using the label system in GitHub makes a lot of sense. I'd assume labels will be assigned automatically based on the Issue templates used?

In terms of how this would work in practice, when someone submits a new issue following a template, an issue label is assigned accordingly. Comment on the issue proceeds as normal.

Then for later followup or prompting, if necessary, they'd need to know how to cross-reference labels to corresponding committee(s) - presumably the mapping you proposed above would be published somewhere like on a CONTRIBUTING.md or README.md page for each repo?

They'd also need to see a list of committee member GitHub handles as well, I believe, so they'd know how to @ mention them, which would mean publishing them on the Governance Page or another obvious location?

Using GH teams would be so much easier than publishing individual GitHub handles, it's too bad it doesn't seem to be possible. Since I'm on the topic, maybe an informal test anyway. When I follow the '@' reference link to cf-convention/gov-panel in @erget's post above, I get a 404 from GitHub (because I'm not a member of the cf-convention GH org), but since @erget used it in the previous comment, I now see that the team exists.

Unfortunately, if I try to use it myself: @cf-convention/gov-panel, it is not hyperlinked, and therefore doesn't notify anyone, presumably. Oh well.

I suggest publishing committee member GH handles is the only alternative.

erget commented 4 years ago

Good thoughts. @mwengren IIRC you know some stuff about Kanban automation, right? Would it be possible to use Kanban transitions to trigger messages to different teams? That way we wouldn't have to maintain the GitHub handles on the Governance page. This should not preclude adding those, either in the PR associated with this issue or in another issue.

Alternatively these could be split out into other issues to keep this one nice and focused - because I'm in a rush I will cease thinking about that for the moment :)

erget commented 4 years ago

I'm still pursuing this issue but for the moment I've shifted focus to getting CONTRIBUTING.md out so that we have clear guidelines on how to update the website. This will allow other work to take place for which this PR is not a prerequisite.

This means that I have created a new issue (linked above) with the sole focus of adding CONTRIBUTING.md. I've tried to capture the discussions on that specific topic from this issue there and have cross-linked to specific comments. Please have a look if CONTRIBUTING.md is of interest to you.

For this issue, I have renamed it to reflect its narrower focus and will now try to finish up the rules refactoring here.

@JonathanGregory your note https://github.com/cf-convention/cf-convention.github.io/issues/102#issuecomment-642206483 still needs to be addressed and I'll come back to it. I think the main issue here is to deprecate or not deprecate and I think I see your point so I'd like to address it with more thought.

On the note of having people's GitHub handles stored somewhere publicly, this might be helpful, but in the case that people should be actively pinged, we should consider how this dovetails with the chairs of committees requesting people's participation as moderators. Everyone is welcome to communicate with everyone, but we should avoid the potential of wanton spamming. Perhaps this would be best to pursue separately from this issue? It's not so much related to the rules but to how we communicate, so I think "Discuss" might be a better place to find a solution first. I'd be interested to hear how the various Committee members see having their handles posted - I for one have no problem with it, but I can't speak for all.

mwengren commented 4 years ago

@erget I was not the Kanban enthusiast on the call, but I remember the discussion.

I can absolutely understand not wanting to post GitHub handles anywhere, including on the Governance page. It's certainly not ideal for many reasons.

If there's no other option for team-based communication about an issue, there's always the generic fallback for a contributor to just post a 'ping' comment along the lines of 'anyone out there...' to get some attention on their contribution if it hasn't drawn any (moderator assigned or feedback).

It's too bad there isn't a non-invasive way to manage this type of communication in GitHub. If this ends up as the solution, it may or may not make sense to add it to CONTRIBUTING.md or elsewhere, since it's sort of obvious to those comfortable with GitHub. I'll leave it to your and others' judgement what to do.

erget commented 3 years ago

Oops - in fiddling with this as I resume activity, I accidentally merged the corresponding PR. My apologies; I have reverted this with some funky git magic and will open a new PR wrapping up the latest status.

erget commented 3 years ago

OK! Welcome back friends, after about a year I'm trying to finish this up :) 3 things are outstanding:

  1. Would anybody be willing to moderate this discussion?
  2. @JonathanGregory - we discussed in https://github.com/cf-convention/cf-convention.github.io/issues/102#issuecomment-642206483 the potential of adding a procedure for correcting erroneous versions of the Conventions. Do you have a text to propose here? We could also pursue that in a subsequent update.
  3. @erget - I need to decrease the diagram size and add data model updates to it.

When those 3 are finished we should be basically good to go. I plan on resolving (3) today.

Note for all needing a refresher: The most notable changes are found here, with a diagram that describes them on a high level. The other changes are mostly of a non-controversial and cosmetic nature, although all are welcome to subject them to full scrutiny.

erget commented 3 years ago

The diagram has been updated to include a review of potential impacts on the data model.

Opinions from the rest of the Governance Panel are welcome; @cf-convention/gov-panel - any thoughts?

JonathanGregory commented 3 years ago

Dear Daniel @erget

Thanks for the nice diagram. I have a few comments.

Jonathan

JonathanGregory commented 3 years ago

Dear @erget

Regarding flaws, we are discussing the replacement of this text:

If the change, once implemented in the conventions, subsequently turns out to be materially flawed, meaning that data written following the convention could be somehow erroneous or ambiguous, a github issue should urgently be opened to discuss whether to revoke the change. If this is agreed by a majority of the committee, a new version of the conventions will be prepared immediately, with the second digit of the version number incremented, and will be recommended to be used instead of the flawed version. The flawed version will be deprecated by a statement in the standard document and the conformance document. However, any data written with the flawed version will not be invalidated, although it may be problematic for users.

Actually this is too limited, because it covers only the situation where a newly introduced change contains a mistake. It might be that there was a mistake introduced some time ago, perhaps even in the very first version. Hence I would propose deleting this paragraph (as you done in the pull request) and adding a new section at the end of the rules, as follows:

Additional rules for correcting flaws in the conventions

A flaw in the convention is a mistake in its formulation, which means that data written following any version of the CF conventions that contains the flaw could be somehow erroneous or ambiguous. This is different from a defect in the conventions document, which means only that the words are wrong, but the intention is correct. Suspected flaws should be addressed urgently by the above procedure, as changes to the convention, and the flaw should be described in the GitHub issue, with the proposed solution.

If consensus is reached on the issue and it is agreed to be a flaw, the affected part of the conventions and conformance documents, in every release which contains the flaw, should be modified to contain a statement which deprecates the use of that aspect of the convention. This is to avoid any more potentially problematic data from being written. The conventions committee will decide by majority exactly what sections and versions of the document are affected, and whether to make a new release of the documents outside the normal schedule in order to provide a correct version. Data written according to the versions of the convention affected by the flaw are not invalidated by this procedure, but the committee should consider warning users of the data where care may be needed.

If you would rather leave this discussion to a separate issue, that's fine. If so, I feel we should retain the existing text in the rules.

Best wishes

Jonathan

taylor13 commented 3 years ago

This seems like an improvement because it generalizes what was said before to cover long-standing flaws and it provides additional procedural detail.
The only bit I thought was perhaps unnecessary and somewhat vague was the second clause of: "Data written according to the versions of the convention affected by the flaw are not invalidated by this procedure, but the committee should consider warning users of the data where care may be needed." I'm not sure how the committee would go about warning users. What did you have in mind?

JonathanGregory commented 3 years ago

I think "Data written according to the versions of the convention affected by the flaw are not invalidated by this procedure" is an important reassurance which we should include. The caveat is that the data may be problematic to use, but I agree that what I wrote is vague, a bit deliberately because it depends on the circumstances. The committee could write down recommendations for how to use possibly affected data and what to be aware of. The committee can't contact all possible users, of course, but it could put this advice on the website or in the conventions document, for example.

JonathanGregory commented 2 months ago

I propose that

I think that will tidy up the discussion. I hope it's agreeable, Daniel @erget?