search

carpentries / maintainer-RFCs

Requests for comment for technology changes and other issues affecting lesson Maintainers.
18 stars 0 forks source link

Maintainer Input on Decision Making Responsibilities of Curriculum Advisory Committees #18

Closed ErinBecker closed 3 years ago

ErinBecker commented 3 years ago

TL;DR

Maintainer input is requested on the rubric for when Maintainers should consult and/or seek approval from the aligned Curriculum Advisory Committee. This RFC will be open through 23 August 2021.

Context / Introduction

In early 2018, The Carpentries formed Curriculum Advisory Committees (CACs) for Software Carpentry and for several Data Carpentry curricula (Social Sciences, Genomics, and Geospatial). Recently, Library Carpentry has also introduced a Curriculum Advisory Committee. These committees were formed in response to feedback from Maintainers, calling for more guidance and clarity in understanding the role and scope of authority of Maintainers in making changes to lessons.

A Curriculum Advisory Committee consists of 5-8 individuals with significant domain expertise representing the breadth of the field that a curriculum is intended to reach. CACs should include at least one member who is actively teaching in the field and one individual serving as a Maintainer for the curriculum. More details about the composition of a CAC can be found in The Carpentries Curriculum Development Handbook.

The SWC and DC CACs were active for ~1.5 years and advised on large directional shifts in curricula including:

The Curriculum Advisory Committees have been on informal hiatus since mid-2019, in part due to decreased administrative support from The Carpentries Core Team in organising meetings and coordinating communications. The Core Team is working to revive the CACs, including providing additional structure to help these committees function effectively independent of Core Team involvement. This revival will include two phases:

Phase 1: June - August 2021

Phase 2: September - November 2021

Timeline

CAC Consultation Rubric

The proposed rubric for when Maintainers should consult and/or seek approval from the relevant CAC is below. Maintainer input on this rubric is exceedingly welcome. This RFC will be open through 23 August 2021.

Items which require approval through CAC vote

Issues where the CAC welcome / invite consultation, but where official CAC approval is not required.

Issues that Maintainers should keep CAC informed about

In advance of each CAC meeting, the Curriculum Team will ask Maintainers to notify the relevant CAC about changes to the following:

Issues that Maintainers have full authority over and do not need to involve CAC.

Edited to fix formatting on nested lists

ErinBecker commented 3 years ago

(Capturing comments from July Maintainer meetings):

From @katyfelkner:

Changing the software being used in the lesson. This does not include cases in which a new version of the existing software is being used (e.g. R 3.x → R 4.x)

I agree with this...changing from, e.g. Python 3.6 to 3.7 is fairly trivial. However, Python 2.x to Python 3.x is a MAJOR change, and I think that domain experts should be consulted about when major versions are widely used enough to be applicable to learners. (my apologies if R major versions are less different than Python major versions)

maybe a good heuristic is we need CAC approval if we move to a version that is NOT backwards-compatible with the previous version.

From @Teebusch:

It's the same with R. The change from 3.x to 4.x broke parts of the R for ecology lesson code and required a lot of changes in the lesson.

From @katyfelkner: in any case, minor updates like 3.6 -> 3.7 should never require CAC approval.

Thanks for the technical info, Tobias. Another thing I think we should consider is what versions are widely used in relevant fields. E.g., there was a time when CS people were using Python 3 while a lot of researchers outside CS were still using Python 2. We want our learners to get exposed to whatever version is currently prevalent in their field.

ErinBecker commented 3 years ago

(more comments from July Maintainer meetings):

Removal or addition of entire episode(s)

from @kekoziar and @emcaulay (paraphrased):

Episode removal and addition should be treated separately. Removal might require CAC approval, but addition shouldn't.

ErinBecker commented 3 years ago

Issues where the CAC welcome / invite consultation, but where official CAC approval is not required.

From @katyfelkner:

suggested addition to this section: Update minor versions of software/packages (e.g. Python 3.7 -> 3.8) when the new version is backwards compatible with current version

ErinBecker commented 3 years ago

Workflow for twice-annual meetings.

Feedback from several at July Maintainer meetings that CAC meetings should be more frequent than every six months. Suggestions include quarterly or every other month meetings.

kekoziar commented 3 years ago

I think implementing experimental features should be listed, at least as a CAC consultation. As an example, git changed one of their functions, checkout, to multiple functions, including restore and switch. The new functions are in the code and output, but are marked as "THIS COMMAND IS EXPERIMENTAL. THE BEHAVIOR MAY CHANGE" in the git docs.

I want to make the change, but implementing experimental features may have unwanted impacts on the lesson in the future, hence wanting to consult with the CAC about something like this.

ErinBecker commented 3 years ago

Comment received privately:

Should CAC approval be needed for any change to a lesson that impacts the content/scope of another lesson in the curriculum? Or should we instead say that, in this case, the Maintainers for the impacted lessons should contact each other directly and try not to involve the CAC?

ErinBecker commented 3 years ago

Addition of a new lesson within the curriculum (e.g. adding Julia as an alternative to R / Python)

Feedback from @emcaulay: doesn't this go through the lesson development structure (and therefore doesn't belong in this rubric for maintainers?)

from @kekoziar: Maybe this is about how maintainers can promote content that is out of scope of their lesson to a newer, advanced lesson?

ErinBecker commented 3 years ago

Promotion/graduation of a lesson from alpha to beta to stable. Decisions on approval can be based on recommendations from the Curriculum Team, CAC member involvement in lesson pilot workshops, and/or open peer review of lessons in The Carpentries Lab.

Feedback from @emcaulay: again, i don't think this is related to the relationship between maintainers and CAC

ErinBecker commented 3 years ago

From @mcmaurer:

Is there any process by which the CAC would do regular lesson reviews? I could see how some of these "approval not necessary" changes could add up to be quite significant over time.

villanueval commented 3 years ago

One suggestion is to include some language on the (expected) roles and responsibilities of the CAC. Otherwise, it sounds like the CAC has no power to say anything about the lessons unless there is consultation coming from the Maintainers side.

A mention of the impact the CAC, and other external groups, may have on the lessons and that should not be covered by "Basically anything else". Hope this makes sense.

tobyhodges commented 3 years ago

Addition of a new lesson within the curriculum (e.g. adding Julia as an alternative to R / Python)

Feedback from @emcaulay: doesn't this go through the lesson development structure (and therefore doesn't belong in this rubric for maintainers?)

@emcaulay You are right that the Maintainers of existing lessons within a curriculum/Lesson Program will not be involved in this process, unless one or more of those Maintainers are also developers on the new lesson in question. In that sense, discussion of this point in the rubric is probably out of scope for this RFC. But the whole rubric is being presented here for the Maintainer community to read, so that you are aware of the full scope of the CAC's proposed responsibilities.

Promotion/graduation of a lesson from alpha to beta to stable. Decisions on approval can be based on recommendations from the Curriculum Team, CAC member involvement in lesson pilot workshops, and/or open peer review of lessons in The Carpentries Lab.

Feedback from @emcaulay: again, i don't think this is related to the relationship between maintainers and CAC

The CAC will be asked to approve proposals that an existing lesson in an earlier stage of development (e.g. the Extended and Conceptual Curriculum in Library Carpentry) be moved into beta or stable state. Some of our official curricula have lessons, with Maintainers, that are in these earlier stages of development, e.g. The Python lesson from the Data Carpentry Social Science curriculum. This point in the rubric is relevant for the Maintainers of those lessons.

ragamouf commented 3 years ago

Nice work! This rubric is set out really well and I particularly appreciate the clear definitions of scope.

If it is written for Maintainers as the key audience, I suggest reversing the order so that it starts within the scope of what Maintainers are empowered to undertake. Progressing stepwise from 'keep CAC informed' through 'invite comment' to 'please approve significant changes' would also make the 'basically anything else' bullet point redundant in the Maintainers' tranch of responsibilities.

In the same direction, perhaps include a new bullet point under

Issues where the CAC welcome / invite consultation, but where official CAC approval is not required.

  • Issues which are not covered anywhere else in this rubric

I would also suggest rephrasing "Issues that Maintainers should keep CAC informed about" to "Helpful issues for Maintainers to share with CAC". Choosing 'helpful' over 'should' places emphasis on the relevance of the issues, and why info sharing is important. In a rubric context I find 'should' less clear as a directive.

ErinBecker commented 3 years ago

Great feedback! Thank you @ragamouf!

doingarchives commented 3 years ago

This structure seems adequate. It also has the advantage of treating all three 'domains' equally. I would like for the CAC role to include summary data each cycle for the approvals granted, withheld and suspended in each meeting cycle. The volume of work needed for the consultation process is not transparent (for example is the 'docket' of consultations cleared at each CAC meeting, or do certain types consultations time out within a period of days).

The data can be quite brief - numbers of issues in each consultation bucket, the length of time (days) between initial consultation and final decision / outcomes in the lesson. While not essential - these statistic per lesson (or lesson grouping) might also be useful - but it can be aggregated. Another data point that is helpful - but not essential is the number of learners for each lesson in a consultation cycle. A 'cycle in numbers' is the gist of what maintainers and the audiences might best be interested in receiving a few days in advance of any CAC meeting.

One issue I'm hoping to see reflected in the data (when it becomes available) is whether the pace of consultations is consistent with the pace of software development changes. R has a very frequent update cycle for instance.

Without actual numbers from prior work - I'm guessing the consultations will number in the low 10s - but I might be off base on how actively issues needing consultation arise.

Best, Christopher

HaoZeke commented 3 years ago

I disagree with the opinion that CAC inputs are required for version changes. We'd do our learners a great disservice if we were to teach Python 2.7 upto the deprecation date of 2020.

Similar arguments apply to R. For all STABLE major releases there is no point dithering simply because the transition is hard. We need to stay relevant.

HaoZeke commented 3 years ago

The rubric is in principle not bad; however the timeline makes it practically useless. This means for example (as per my understanding) that if I propose to teach/solicit content for plotnine over matplotlib I would need to plan on a timescale of around 3 months; so the lesson would not be practically updated until almost half a year passes.

Perhaps something which should be clarified elsewhere, is which scenarios allow maintainers to "ask for forgiveness instead of permission".

HaoZeke commented 3 years ago

Another comment / clarification should be the scope of IDE changes. Given that maintainers are allowed to add callout boxes it would be relatively simple to say; add instructions for working in multiple IDEs; only it is currently prohibited (as per the rubric proposed).

HaoZeke commented 3 years ago

Change from GitHub to GitLab

^ This should be an admin decision; the carpentries currently has no presence on GitLab (AFAIK) and should not be under the ambit of the CAC or Maintainers.

emcaulay commented 3 years ago

Change from GitHub to GitLab is a bulletpoint on its own and I assume this bullet is intended to refer to the LC-Git and the SWC-git-novice lessons.

I recommend more verbose bullet:

Change from GitHub as remote hosting platform to a different remote hosting platform, e.g., GitLab

tobyhodges commented 3 years ago

The rubric is in principle not bad; however the timeline makes it practically useless. This means for example (as per my understanding) that if I propose to teach/solicit content for plotnine over matplotlib I would need to plan on a timescale of around 3 months; so the lesson would not be practically updated until almost half a year passes.

Given that this would represent a quite significant change to the content of the lesson, I would say that 3-6 months is appropriate. (I see changing the plotting library in a Python/R lesson as broadly equivalent to changing the hosting platform used in a Git lesson.)

To use your example, consider that Instructors may not be familiar with plotnine and the significantly different syntax and approach to plotting that it requires in comparison to matplotlib. Any such change would require some communication to the Instructor community, to make sure that everyone is aware of the change and is prepared to teach the new version when the changes come in. It would also require changes to many exercises, example images, and perhaps the introduction of new explanatory figures etc.

In my opinion, an exception should be made to this point only if there is a time-sensitive reason to move away from the library currently used in the lesson, e.g. if some feature essential to the lesson were being deprecated.

hoytpr commented 3 years ago

I find the Rubric to be not just satisfactory, but I strongly agree with virtually all points. It's liberating to have this well thought out structure. From a prospective of a maintainer, there's little to disagree with. Many helpful comments are more precisely part of curricula advancement and development, and some explicitly elevate the maintainer role to a developer (many based on the skills and preferences of the individual maintainer).

My opinions may seem minimalist but maintainers should focus on... maintenance. This important role is also a critical interface with the community, notably new members. We educate new users on PRs and we encourage them to "try" those next steps in collaboration. This process can be time consuming. To me, the freedom to clarify wording, improve images or figures, update data sets, and consider reasonable requests for reorganizations or better cross compatibility are enough. Please do create new lessons with the most up to date software, but put them in development and eventually involve and trust the CAC.

ErinBecker commented 3 years ago

Thanks to all who have left feedback here and at Maintainer meetings. I am closing this issue to further comments now and will post a summary of the feedback, changes made, and link to the final rubric within the next week.

ErinBecker commented 3 years ago

The rubric is now live on The Carpentries Handbook, and is referenced in our new Maintainer Onboarding curriculum.

A summary of changes made:

As we move forward in using this rubric, I'm sure there will be things that come up that aren't covered, or that fall between the cracks. If you come across any issues in applying this rubric, feel free to contact me directly (ebecker@carpentries.org), re-open this issue, or send an email to our TopicBox group.

Finally, a note that most CACs have not yet been convened. (The Library Carpentry CAC is the only one currently active.) If any urgent issues arise which would normally be under the authority of the CAC, and the CAC for your curriculum has not yet convened, please contact the Curriculum Team for support and assistance. The call for CAC applications will go out next week. Applications from Maintainers are highly welcome!

Thanks again to all and a special thank you to @emcaulay for providing additional editorial assistance in preparing the rubric for publication!