Some comments on the letter

[wch]

I'm commenting here as a member of the R community. I work at RStudio, but what I say here does not represent the company. I would have commented via survey form, but it turns out I had too much to say and it wouldn't have fit there. I felt that I needed to register these thoughts somewhere, but please feel free to close this issue if you want.

I've had my share of frustrating interactions with CRAN, and I agree with the overall goal of making CRAN friendlier and more predictable. That said, I think that this letter could be setting things up for more conflict and entrenchment of CRAN's current practices, instead of setting things up for positive change.

Since you are trying to convince CRAN to do things differently, it's important to consider their perspective. From their point of view, they are providing a service to R package authors and the broader R community by enforcing quality standards, and users of the service should be appreciative of that.

I think it's important to model the ways this could play out. It is vanishingly unlikely that CRAN will just implement the 11 changes in the letter. The letter asks them to give up a lot of control and do extra work, and in exchange, they get... not much. If they only implement a handful of the changes, or (more likely) none at all, then what happens? With the letter in its current form, it seems to me that the path forward is to antagonize the CRAN maintainers until they make the changes, and I don't think that will be a successful strategy.

The two items that stand out in that regard are number 2, submitting packages to CRAN with a badge/link criticizing them and petitioning for them to do things differently, and number 5, calling them "administrators" instead of "volunteers".

Imagine a world in where packages under the rOpenSci umbrella started adding badges to their READMEs, linking to a letter criticizing how rOpenSci is run and demanding changes. And imagine that people asked rOpenSci to add their new packages with the same badge linking to the letter. If that happened, it wouldn't be a surprise if the people within rOpenSci were irritated by it. If someone publishes criticism and a petition for change, that's one thing; asking the organization to publish packages that prominently link to the criticism and petition is another. Now just substitute "CRAN" for "rOpenSci" above. I believe CRAN would have an even stronger negative reaction.

As for the other item: pointedly calling the CRAN maintainers "administrators" instead of "volunteers" will only serve to annoy them, and for what purpose? The justification for calling them "administrators" has changed since I saw it in an earlier version of this letter, but the real reason, I suspect, has not. In their emails, they refer to themselves as "CRAN maintainers." Why not just use that?

I have a prediction about what CRAN would do with packages and package authors who do these things, and I think there's a good chance that it would escalate into a big public fight that ultimately hurts the R community and makes it harder for change to happen.

I saw an earlier version of this letter, and it was clearly written out of frustration and anger. The letter has been revised significantly since then, but that foundation is still there. For example, the very first sentence: "This letter seeks to draw attention to deficiencies in empathy and fairness that are impacting the R community." This is an airing of grievances, when it could instead be a call for positive change.

It may not be easy, but I suggest taking a step back and thinking about how you can convince CRAN to make changes; think about their perspective and goals as you try to convince them to change, instead of focusing energy on the things they do that you dislike.

I don't claim to have the answers about how to convince CRAN to do things differently. But please consider this: In the event that CRAN does not implement the changes you're proposing, what will you do, and will it help the R community?

[humanfactors]

I haven't made any contributions to this letter, but I have been following the updates. I must declare that I am not well positioned to address many of the insights and points you have raised above.

Nonetheless, I agree that with your suggestion of "taking a step back and thinking about how you can convince CRAN to make changes". I think it would be worthwhile to consider initiating this process by opening dialogue with the CRAN volunteer/maintainer team. I'm not sure on the process of doing so, but I envision that acting in good-faith and seeking feedback and collaboration from CRAN maintainers/volunteers may help accelerate any future change processes, and identify items of concern which may have immediate barriers, relative to proposals that perhaps are more readily actionable.

For instance, I don't foresee that item 9 should be particularly controversial ("Establish a code of conduct for the behaviour of CRAN administrators and contributors in their dealings" — though I suggest package maintainers too), particularly given establishing codes of conduct is increasingly commonplace in open source projects. However, while perhaps uncontroversial, the prospective contents of such a code are only implied in the letter, and I imagine that CRAN volunteers would have good insights into the factors that should be considered in developing such a code. I would think that raising such discussion is beneficial to the longevity of R and CRAN beyond the immediate horizon.

On the other hand, my understanding is that there are conventions not formally listed in the CRAN policies (e.g., 2 week bans). Thus, there is a substantial gap between the proposed reforms and current practice. Perhaps one approach would be to consider a longer-term pathway to change, that might involve for instance simply starting by requesting clarifications on CRAN's policies (relevant to items 1, 3, 4, 6, and 7).

Again, I iterate that I am mostly naive to the processes or feasibility of what I've said here. However, I do think that to put one's best foot forward here may likely involve open dialogue and fostering an environment in which invitations to appraise/critique proposed changes are offered.

[MilesMcBain]

Thanks for the feedback @wch I appreciate your thoughts.

I think it's important to model the ways this could play out. It is vanishingly unlikely that CRAN will just implement the 11 changes in the letter. The letter asks them to give up a lot of control and do extra work, and in exchange, they get... not much. If they only implement a handful of the changes, or (more likely) none at all, then what happens? With the letter in its current form, it seems to me that the path forward is to antagonize the CRAN maintainers until they make the changes, and I don't think that will be a successful strategy.

I agree that it's unlikely they'll implement the 11 changes. But so far no one has disputed the proposed reforms are reasonable and I think that's a good sign. To @humanfactors point, nearly all the proposed reforms are high level with implementation details TBA. I see this letter as the start of a community discussion, the 11 proposed reforms are the opening position.

It seems my proposal about the badges was not clear. I am not proposing embedding badges linking to this petition in our submissions to CRAN. That would definitely be kind of petty and annoying.

I am proposing, only where people feel comfortable, they should withhold submitting their work to CRAN, and use the badge as an explanation of why they are doing so.

To explain the badge I have to talk about trust. CRAN has a monopoly on trust in the R package ecosystem. CRAN acceptance is the standard way someone can signify their work is trustworthy - e.g. of high quality and actively maintained. I think the fact CRAN has this monopoly is what has let some of the practices that really should have evolved as the community grew persist in unsustainable forms.

This situation means that people who have created packages they want others to trust have no option but to proceed according to CRAN's terms, even if they don't want to support CRAN due to problems with the way it is working. I thought the badge might help people feel better about passing on CRAN. It's a statement that says: "I believe this work is of CRAN quality, but I am not submitting to CRAN just now because some things need reforming" or something like that.

The trust issue is probably not felt heavily by senior people like you @wch, but I'd ask you to empathise with people just starting out now trying to bring innovative ideas to the community.

Again with action number 5 I feel like I am being misinterpreted. I tried to explain this more clearly on Twitter so as a start I'll just paste that here:

the idea is that it feels kind of weird to try to evaluate the performance of "volunteer" work. The word has baked into it a notion of "best effort" service. Irrespective of whether they are paid or not, the CRAN team have very important administrative roles that affect the day to day experiences of a great many people now. Part of the reform conversation is what is expected of that role and how we can resource it appropriately.

I think names are important. A conversation framed around the responsibilities of "volunteers" will go a very different direction to a conversation framed around the responsibilities of "administrators" IMHO. It feels awkward to hold "volunteers" accountable, so I think we need to define a role we can do that for. I'm not trying to take anything away from people doing this work by doing this. As I have said, they probably need more resources and formalising the role would help with that.

As for "maintainers", I don't think there's much to separate that from "administrators" although I think the latter is a better fit, since administrators could imply a broader responsibility to all of CRAN's stakeholders, rather than maintainers who you could interpret as having responsibility mainly to the system itself.

From my perspective this is consistent with the ways I have drafted this point so far. I am not really sure what you are trying to say about my motivations.

I saw an earlier version of this letter, and it was clearly written out of frustration and anger. The letter has been revised significantly since then, but that foundation is still there. For example, the very first sentence: "This letter seeks to draw attention to deficiencies in empathy and fairness that are impacting the R community." This is an airing of grievances, when it could instead be a call for positive change.

A lot of people don't like that first sentence. Even I am only lukewarm on it. It's almost certainly gone.

Since you've been following the drafting process you'll know I have been really struggling with how to open this thing. Part of it is definitely about managing my anger, but I'm not really going to be made to feel ashamed of that. Things that have happened have made me angry. I do feel frustrated. It turns out so do a lot of people. CRAN administrators should probably know they're making people feel this way and the effect it's having. (I guess they probably do by now)

The challenge is not to let that overpower what nobody seems to be disputing are some solid points about the way CRAN is working and how it can be improved. I don't want to cause great offence to individuals, I am trying to make it about the practices not individuals. But I do think the message needs to be in an authentic voice that can include all the people who feel upset.

It may not be easy, but I suggest taking a step back and thinking about how you can convince CRAN to make changes; think about their perspective and goals as you try to convince them to change, instead of focusing energy on the things they do that you dislike.

I will/am. I am now in two minds about whether this letter can actually do that. I was discussing this with @llrs last night: My draft was not written for CRAN, it's written primarily for community members. The idea was if you can tap into enough community support, you can force CRAN out of its hermetic posture and into some engagement on reform. I never really imagined CRAN would engage with the petitioners to do that, rather it would be the community leaders (some whom you work with) on the R Foundation et. al. who (with the benefit of acting with clean hands) could use the petition as a wedge to open up a dialogue.

This was based on some private conversations I'd had with various community leaders about CRAN over the last couple of years, who all acknowledged the issues but had no way forward due to the politics involved.

So I am hearing many people now say they think such a letter could in itself be something that could be convincing to CRAN. I am not exactly convinced of this myself, but I have been thinking, and talking about ways it could be rewritten along those lines.

In the event that CRAN does not implement the changes you're proposing, what will you do, and will it help the R community?

The happy path is: We identify and communicate the practices that aren't working for us. We propose solutions. We get engagement on them and make some compromises leading to progress.

If CRAN won't engage on changing its practices, yeah things get messy. Some people who've had enough will probably be withdrawing from the community at that point. Others will be trying all avenues to motivate change. I can't see them resisting a big enough push from the right people and organisations. I don't really want to think about what happens if I am wrong.

[wlandau]

@wch, I am really glad you opened this thread.

Imagine a world in where packages under the rOpenSci umbrella started adding badges to their READMEs, linking to a letter criticizing how rOpenSci is run and demanding changes. And imagine that people asked rOpenSci to add their new packages with the same badge linking to the letter. If that happened, it wouldn't be a surprise if the people within rOpenSci were irritated by it. If someone publishes criticism and a petition for change, that's one thing; asking the organization to publish packages that prominently link to the criticism and petition is another. Now just substitute "CRAN" for "rOpenSci" above. I believe CRAN would have an even stronger negative reaction.

The comparison did not cross my mind because rOpenSci has always been responsive and inclusive. I know I could talk to rOpenSci without badges or a petition. What would this letter look like if we assumed we already had CRAN's full attention and they were friendly and willing to listen? I think that should guide the next round of revisions.

Do we know of other times folks have have brought up policy issues with CRAN? How does that generally go?

[llrs]

I think this letter must be addressed to CRAN as they are the ones doing the work and the ones that could implement these petitions. Of course if we propose some changes and these are implemented against the community wishes it won't work. But as you pointed out all the petitions have been seen favorably.

I don't see CRAN or the R core as hermetic. I've been for several months participating on the R Contribution Working Group. What I observe is a mismatch between the R core petition of help from the recent blog posts (bug reviews, and alpha testing R 4.1.0) and other discussions (renew the R core on last useR! panel discussion) and the main friction points for the R package developers (CRAN, mailing lists, ...?).

I searched on R-devel and R-pkg-devel for mentions to CRAN policy and I haven't seen any discussion before a change or after it. Most emails mention that the policy says this, you can('t) to that. I think most of the petitions are already on that direction: changing how the communication has been going. Instead of being a communication from the CRAN policy to the package developers most of the petitions go towards increasing the communication between the CRAN team and the package developers in a bidirectional channel.

Compared to rOpenSci where people is paid to work on it and hear users R core is not. I think it is safe to assume that they are not on purpose ignoring R users. I think a better comparison would be with Bioconductor. They get paid by their own grants to work on some aspects of the project while also supporting the whole project.

It was mentioned that the letter needs a big push from the right people and organizations. What would bring more/the right people onboard? Reducing the tone without changing the petitions or removing some of the surrounding text? So far I only seen people saying they wouldn't sign this draft due to the tone currently used. So I think it should focus on the petitions and leave the motivations outside the letter.

[MilesMcBain]

So far I only seen people saying they wouldn't sign this draft due to the tone currently used.

@llrs I shared the feedback form results with you. This just isn't true. Many people endorsed it. More than half in fact.

[MilesMcBain]

What would this letter look like if we assumed we already had CRAN's full attention and they were friendly and willing to listen?

I think this is a good idea @wlandau.

[MilesMcBain]

I think this letter must be addressed to CRAN as they are the ones doing the work and the ones that could implement these petitions.

This point does carry weight with me. The draft in current form kind of presupposes the outcome of engaging with CRAN.

I can see that assumption causing offence if there is any will to improve.

[llrs]

@MilesMcBain I have not expressed well: Those that say they wouldn't sign this letter is because of the tone used. If we want to them to sign the petition we should change the tone as they agree on the petitions.

[MilesMcBain]

One other thought I had on the tone of the letter:

I think it's no wonder there is such variance in the way people think this should be communicated if you consider that part of the issue we are highlighting is lack of consistency. Communication styles among the CRAN administrators vary wildly and criteria are being applied differently.

The second aspect that needs to be taken into account is the fact that different people have different capacities to absorb the harmful practices we identified. For example if you are paid to submit work to CRAN, then the harm probably translates into a bad day at work combined with some frustration at deadlines needing push out.

If you are contributing on your own time, then CRAN's practices are taking you away from family and friends, and other important life things. I could easily see the practices creating a much deeper sense of frustration for people in this bucket.

Taking it further, imagine the scenario for people who are already battling uphill to make the contribution due to some kind of disability or disadvantage. CRAN has the potential to feel like an extremely hostile place to these people and I can imagine their anger being deeper still.

[jonocarroll]

Thank you, @wch, for raising the issue of sided-ness. I do believe it's important to understand a conflict from both perspectives, and I have no expectations that any of the CRAN team are operating with malicious intentions, but I do believe that the approach which has been the de facto for a long time is fastly becoming untenable in the openly-developed world. The excuse (for lack of a better term) that these are volunteers working towards supporting an appropriately funded community seems oddly placed when Julia Computing was able to raise millions towards developing their language and a mature Pkg infrastructure. If CRAN were a language support feature developed under external funding there would be few questions that this should be held to community standards. The ad-hoc and at-discretion nature of CRAN reviews is a long-held source of frustration for the community, which should instead be a cause for celebration - few package repositories can boast such extensive testing and review criteria.

I'd like for the intention of this letter to reflect the admiration the community holds for what CRAN represents while at the same time I feel it is appropriate to criticise the way in which this is currently handled. No progress will come from simply criticising, I appreciate that. I suspect that given the structure of CRAN, change will need to come from within, and I'd like this letter to spell out what the community would like that to look like, but it won't be at no cost to CRAN. What I think would be worthwhile presenting to CRAN are the benefits to their service which will come with these changes, most importantly that the community will have faith that they can submit packages and understand, based on documentation, why they are accepted or rejected.

[MilesMcBain]

reflect the admiration the community holds for what CRAN represents while at the same time I feel it is appropriate to criticise the way in which this is currently handled.

Oof yes, really appreciating this as a way to walk the line.

[MilesMcBain]

What I think would be worthwhile presenting to CRAN are the benefits to their service which will come with these changes

I guess one angle we could work a lot harder is how self-defeating a lot of CRAN's practices are.

If you have *ambiguous or undocumented criteria you're going to burn more time in review.

If you expect tests to pass in inaccessible environments you're going to burn time chasing down maintainers for fixes.

If you spring policy changes suddenly, without consultation and announcement you're going to burn time chasing violations.

If you're going to allow people to submit more often than you feel is reasonable, then you're going to burn time trying to deter them from doing that.

If you're quick to archive packages you're going to burn time re-reviewing re-submissions.

If you wanted to keep yourself busy working on CRAN and CRAN alone you'd be troubled to devise a better system to ensure that than these practices.

[MilesMcBain]

Nearly all our proposals will reduce the burden for both sides.

[MilesMcBain]

And before anyone says it: yes we can flip all those negative statements about cost into positive statements about savings.

I might be getting the hang of this. 😛

[adamhsparks]

This is something I was trying to get to as well. If properly implemented, these suggestions reduce the burden on CRAN personnel because the expectations are clearly communicated and we as package developers know what needs to be done rather than guessing and resubmitting over and over.

[MilesMcBain]

The letter is now substantially different, in no small part thanks to feedback in this thread. I appreciate any new feedback in a new thread!