search

cranchange / cran_change.org

A petition calling for CRAN reform.
86 stars 6 forks source link

Launch #27

Closed MilesMcBain closed 3 years ago

MilesMcBain commented 3 years ago

@cranchange/authors and anyone else interested,

There are still some tweaks I want to try with the text, but I think it is already done enough to set a timeline for launch.

4 weeks or so from now seems reasonable, which would be: 2021-08-02.

Some tasks before then

adamhsparks commented 3 years ago

I'll have a browse over it in the next few days again and give any feedback before this deadline.

llrs commented 3 years ago

4 weeks is a reasonable timeline to share with the community to sign it. However, when would it be formally shared with CRAN team and what would be the goal of number of signatures on the change.org website? For reference CRAN has around 16000 packages.

wlandau commented 3 years ago

Thanks for the heads up, @MilesMcBain. It has been a long time since I looked at the letter, and I proposed changes at https://github.com/cranchange/cran_change.org/pull/28.

MilesMcBain commented 3 years ago

Thanks @wlandau good stuff in there!

MilesMcBain commented 3 years ago

@llrs

llrs commented 3 years ago

@MilesMcBain

mmaechler commented 3 years ago

The tutorial was by Gabe Becker (75%, @gmbecker, not R core) and me (25%, R core). The final round table (questions to R Core etc) included Michael Lawrence (@lawremi), R core, indeed. Yes, R Core (20 members) is indeed not the same as the CRAN team -- which is a small subset of R core plus hired staff.
Are you really sure it's a good idea what you are planning here (cranchange) ?? It will cost a lot of time and create a lot of heat in the wrong places. For sure. The effect could be negative rather than positive.

Do you know some of the CRAN team members personally, e.g., Uwe Ligges, of whom you can also watch presentations of earlier useR! meetings? and are you really aware how much personal free time R core and also their CRAN team subset have invested into serving the R community by setting up and maintaining CRAN for ca 20 years, now??

MilesMcBain commented 3 years ago

Hi Martin, thank you for contributing to this discussion.

I'll answer only for myself. I have complete conviction that these issues are valid and that it would serve our community to see them addressed. I am not at all certain that a petition is the best way to get that done. I do not know any CRAN members personally. I did watch Uwe's last useR talk back in 2017.

A question for you: Shouldn't it be possible for CRAN contributors and community members to raise issues with the way CRAN is working, and propose better ways it could work, without that being construed as disparaging individuals or their legacies of contribution?

What do you think is the best way to do that? I would welcome your input.

Edit: Forgot to tag you @mmaechler.

llrs commented 3 years ago

Apologies @mmaechler for being unclear about who taught the Contributing to R tutorial and many thanks for making it possible. It was very well designed and very useful, I learned a lot.
I speak for myself and not for other contributors and authors of this letter. Many thanks for your work on R, your comments and questions about this letter and its authors.

I do not know anyone of the CRAN team personally, I haven't met them at any conference or as part of my work, although I have exchanges some emails besides those related to a package submission. I do not want to heat any discussion or relationship, offend or disrespect anyone. These are the opposite of my intentions and given this letter is not well received, I do not think that sending it to the CRAN team is a good idea. Thanks for the warning.

Watching Uwe Ligges' talk about CRAN on useR!2017 gives a slight idea of how much have costed to set up and maintain CRAN. The personal time, the emotional and financial cost must have been huge over the span of years, not only to him but to many contributors. I really appreciate all the past and current efforts that are made, thanks! I would like to contribute on the best of my abilities to the R and CRAN system to give back some effort made and show my gratitude for the personally and professionally benefit I have had due to it.

Thanks to the contributing to R tutorial and the tutorial about translating R I know how to contribute to R source code. How can I, as a regular R user, help the CRAN team? Perhaps something similar to this Contributing to R tutorial but focused on contributing and collaborate with CRAN might be very useful for the R community.

wlandau commented 3 years ago

@mmaechler, you bring up excellent points about the dedication required to envision, implement, scale, and maintain CRAN for over two decades. CRAN was only possible due to the incredibly persistent and monumental efforts of a small number of individuals over a long period of time. It is staggering to reflect on this, and I am grateful. However, because reliance on CRAN has increased to an unprecedented level, even to large companies and regulatory authorities, I believe it is worth reevaluating the ways we have traditionally thought about this critical infrastructure. The global need is far bigger and more serious than individual sensibilities. Still, we should make every effort to respect these sensibilities, and I fully agree that we should pause and discuss how to raise these issues in a manner more constructive than a petition.

gmbecker commented 3 years ago

@MilesMcBain There is a difference between respectfully requesting changes, under the understanding that they may or may not be taken up, and requesting/demanding control and oversight which is what the petition appears to be doing.

One of the points I made in the tutorial, which again as said above about was regarding collaboration with R-core, not CRAN, is essentially that we don't get to have our way. R-core, or when translated to here, CRAN, has the final say. When I am contributing a new feature and R-core members i'm working with and I disagree about the design of the feature, which has happened, their design is what I implement in my patch and their design is what goes in. Full stop. If I had refused to bend and been headstrong in implementing my (still, being honest) preferred design, I have to imagine the patch would simply not have been applied, the feature would have died on the vine, and that would have been that. I say this as, I believe, probably single most accomplished collaborator with R-core members on adding new features to R "in recent times".

Put another way differently in the same slide-deck:

Each step of the above is important to understand, in my opinion, and that last one is hard, for everyone, to really fully grasp. It's just not how people think.

The CRAN team has thought about issues of testing and distributing R packages more than anyone else on the planet (excepting, very arguably imho, the Bioconductor infrastructure maintainers who I would say are in 'second place' there), very much including me, and I have thought about it much more than most, including literally writing a paper (that is partially) about it and having implemented a CRAN/Bioconductor hybrid build build system from scratch in the GRAN packages which is used in production in at least (and probably exactly 🤪) one place. Sometimes when we disagree with their decisions, it will be because they have thought of things we have not.

Just as an example from the content of the letter, giving substantially longer to provide fixes leaves the packages in the repository in a broken state (yes, a broken state, not passing tests means a package is broken) longer with absolutely no warning or information about that apparent to the average user going to install it. So while that would be very nice for package developers, its actually quite bad for R users more generally all of whom consume the packages on CRAN as production software they use in their work/play/whatever.

It is not that you're wrong about the burden short turn around times place on developers, its just that, in my opinion and maybe, without speaking for CRAN because I am not affiliated with CRAN, the CRAN maintainers' opinion, it is counterbalanced by other concerns.

I do not want to come across as as some sort of CRAN-No-Matter-What partisan here. There are issues raised, here and elsewhere that I do think should change. As one example, I am very uncomfortable with changes to code without corresponding changes to package version number and I do think we should, respectfully, try to get CRAN to not do that when they find themselves needing to alter a package.

But (assuming you've even read to this point in the novel I'm writing here) the larger issue, which i agree with what I suspect @mmaechler is saying is going to cause major problems and likely undermine the full effort here, is the demands for oversight, both implicit and especially explicit. That is an extremely touchy subject for a well (decades!) established thing like CRAN, and I don't think a petition of non-CRAN people, rather than trying to engage with them directly, is going to be particularly successful at it.

gmbecker commented 3 years ago

Another point I agree on is more transparency on is the details of the build environment, but I don't think an unfunded mandate style demand is the best way for us, as a whole community, to get that.

There is a very big difference between

"we want this thing do it for us in addition to all the things you're already doing on a volunteer basis"

on the one hand and

"if we get funding from the R consortium or the Moore Foundation, or Google or NSF or whoever, are you willing to work with someone so that they have the information they need to create docker/podman/whatever containers that mimic the CRAN build systems closely enough to be useful so that we can provide those to the community of package developers and reduce the work for you and for them."

on the other.

Its difficult IMHO to overstate how big a difference that is. Without having talked to the CRAN people about any of this, I think the latter has a reasonably good chance of success. In my honest opinion, I don't think the former does, regardless of how good and beneficial and "right" the desired changes might be.

wlandau commented 3 years ago

Given what we discussed so far, I think we should pause this effort as currently implemented in a petition directly addressed to CRAN. I do not think we know our audience well enough, and we have not thought through the consequences (what a favorable outcome looks like, what a likely outcome looks like, what we do if they say no, and what is the worst that could happen).

I think we would learn a lot by directly engaging the R Core Team as a whole, the R Consortium, and potentially the Linux Foundation. Linux has gone through a similar transition similar to what we may face now. To prepare, we might do well to recast the petition for a different audience.

MilesMcBain commented 3 years ago

@gmbecker I don't find the equivalence you are trying to draw between requesting R core changes and petitioning for CRAN reform very fitting from my perspective. If CRAN was run as per R Core, this letter would probably have never been drafted. R Core is very much 'open for business' - there are multiple ways the community can engage them in open discussions about improving R. The same is not true for CRAN.

I am all for engaging in the most effective way to get reform. But let's be real here: If we sent email to CRAN or R Core from 4 concerned R developers, what would the response be? This is assuming we can even find an email address to send it to.

In my estimation, the best case would be we'd thanked for our efforts and probably never hear anything further on the subject. These are all really busy people (so they keep impressing upon us), and as such, I couldn't really blame them for not giving deep consideration to proposals from a handful of people that don't hold influence with them. They're not going to leap to fight a fire when there's no smoke.

So to get any favourable outcomes on our proposed reforms, I see only two options:

  1. Find someone who has influence over CRAN to patronise our proposal and broker some kind of community engagement with CRAN.
  2. Inflate our own influence by consolidating it with a great many other people who hold similar views. In other words, form a union or create a petition.

The petition is not meant to be letter of demand, but rather as something that publicly registers the issues (since we have no other place to do that), and documents an opening position on reform. Of course reform is a negotiation. I would hope a diverse set of community representatives are to be brought into that conversation. I made the mistake of thinking that was obvious, and I think I need to make that clearer in the letter.

I have attached very low likelihood to us finding (1.). Maybe that was also a mistake. I did actually already try to contact someone from R Core that I thought might be the most likely person to support something like this, but I received no reply. If there's any suggestions I am happy to hear them. Otherwise (2.) still seems like a viable route to me, though there is clearly more work to do on the communication side.

wlandau commented 3 years ago

(1) has a lot of options we have not yet explored: among them, formal exchanges with different but equally influential organizations (R Core, Bioconductor, the R Consortium, the Linux Foundation). Whether or not they officially endorse an initiative about CRAN, I think we are likely to come out ahead, even if that only means we are better informed. And we can repeat (1) as much as we need to. With (2), it seems like we only get one attempt, and it could very likely burn bridges.

gmbecker commented 3 years ago

The issue with this approach is that you are looking for a way to force CRAN to do something that they do not want/do not agree that they should do: give up control of their lives' work and subject it the collective control of the wider R community in general and you, the initiators of this effort, in specific. Given that they don't have any relationship with the people who would be placed on that board, or, it seems, any control in its proposed nature, that seems unlikely to be something they will like the sound of.

I don't think you can find people - in the case of (1) - whom you can talk to that can force CRAN to do what you want it to do, and I similarly don't believe that there is a number or set of signatures on a petition, in the case of (2) that will somehow compel CRAN to "come to heel" of their own accord.

If you want changes in how CRAN is implemented to actually occur, the best way I know of to get them is to approach the CRAN maintainers respectfully, as a collaborator willing to do major components of the actual work called for to perform the change. If they agree the change would be good, but simply do not have the time/energy to do so themselves, this helps them help you, and will build up a relationship. Note that requires they agree with the change you are proposing. Under this scenario you have influence, but not control. But your changes are moderately (I'm guessing) likely to actually happen.

If you want control, I don't know of any way I feel has an even mildly high chance of success of achieving that, including (1) or (2), and to be honest its not really clear to me it should.

One additional point, you mention this being a negotiation and the letter being a sort of opening bid. Negotiations are inherently adversarial endeavors, and not, I believe, a good conceptual model for an effort like this. If this is to work, in my estimation it has to be a collaboration with CRAN.

MilesMcBain commented 3 years ago

@gmbecker I don't see it that way. Control is not how I would characterise what we are seeking at all.

To use your analogy: that would be like saying you only contribute to R Core because you want control of R. That's obviously absurd and impossible because R has a robust governance model. You have ideas you think would make R better, and you use the channels that have been created for you to try to negotiate/collaborate to get them realised. You accept when there is a reason they aren't. That's what we want to do with CRAN.

But there's no CRAN bugzilla. There's no CRAN mailing list. There's no CRAN Slack. There are zero channels for any community feedback or engagement with CRAN. CRAN apparently has no mandate to collect feedback from the community that it accepts resources to serve. That fact alone tells you almost everything you need to know about why change is needed.

Some of the issues we are rasising have some important ramifications. Maybe they have been considered in depth, maybe they have consulted people, but we just don't know because there are no accessible records. So what we're supposed to just have blind faith? If things were going smoothly that might be possible.

But they're not. CRAN is doing things that don't make sense. New rules are being introduced that seemingly apply to some people/packages and not others. Contributors are being abused. Packages are being archived without warning. CRAN is altering the source code of people's packages without telling anyone. These are not signs of a system and a team that has thought of everything and has everything absolutely 100% fair-dinkum under control. And this is reason questions are now being raised CRAN's policies and governance.

MilesMcBain commented 3 years ago

TLDR CRAN is looking pretty buggy and I just want to patch it. I don't want to own it.

gmbecker commented 3 years ago

Personally, I do not see package developers as the primary set of people CRAN serves; I believe it is R end users. I strongly suspect (though have not discussed with any of them) that they feel the same way. I say that as someone who has had packages archived and then had to get them back onto CRAN.

So a relevant question then becomes how much degradation of the strong guarantees they are providing end users is it worth to reduce package maintainer pain? As you know, I am not on CRAN, nor have I talked to any of them about any of this, but if I were on CRAN, I suspect my answer to that question would be "very little if any at all". I think its valuable to view the specific proposed policy changes not only in light of CRAN maintainer pain vs package developer pain, but also in this light. That is not to say all proposed changes would degrade the user experience/guarantees, but some of them I think would.

As for my contributions to R, I accept when there is a reason my proposals aren't taken up (though I'm also careful to only submit ones that I think this won't happen); I accept it when i don't agree with the reasons my proposal or particular design choice aren't taken up. And I accept it when there is no reason. How could I possibly not? What would me not accepting it even look like? I have no leverage to not accept my proposals being declined, nor should I.

The bit about "when I disagree with R-core their version goes in" wasn't about them being right (though thats a pretty good prior in any given instance); it was about the necessary structure of the collaboration itself.

So in terms of calls for changes in policy, there are two version of this. The first is that you can ask but they can say no if they don't agree. The second is that there is an expectation that they will/must do what is in the suggestions because that is how the community feels, provided enough people sign it. That is a claim of oversight (ie control).

llrs commented 3 years ago

Bioconductor Community Advisor board discussed this letter on 2021/06/10:

reforming the administration of CRAN's package ecosystem https://github.com/MilesMcBain/cran_change.org/blob/main/letter.md -Discussion: -It is good to be aware and Bioconductor already takes some procedures and precautions to elimite the CRAN concerns -Bioconductor open review and encourages conversations on submitted github issues -Package guidelines and recommendations publicly available(package review committee hopes new updates also will make them easier to find and more public facing) -Packages are deprecated before official removal and after automatic failure notifications from the daily builder and individual email outreach from core team member -Source code for all packages are available for view.Just not for direct editing -In progress of governance revamp -We publicly post minutes of TAB/CAB on website

And today Bioconductor has opened the review of packages to new reviewers outside the Bioconductor core team. The work on the package review guidelines was done mostly by one of the Bioconductor Community Advisors, not a core member.

Since Bioconductor boards post their minutes I've been following and reading them (mostly the Community Advisor Board which are volunteers from the community) to know what are the changes and topics of interest and where could I help. In several occasions just by reading the minutes was enough to me to chime in on the slack channel or on a github issue to provide some links and examples. At worst it has shown what is expected of a board member. I think that more transparency will be good, not to control or demand things but to provide help and assistance when needed, to understand where does CRAN come from and why they take the decisions.

MilesMcBain commented 3 years ago

@llrs Thanks for sharing! That's great to see some evidence that the issues we're raising are being reflected on by repository governance. Not the repository we had in mind, but still!

Also how excellent is it that we can know about that by reading their meeting minutes!? I feel like they deserve some fan mail.

MilesMcBain commented 3 years ago

@gmbecker this point you're raising is a good one. I would love to engage CRAN on this topic. The place I would start would be the recent {targets} example. 100% test coverage with ~ 2400 tests. 2 of those tests failed on CRAN's MacOS and the failures could not be reproduced on RHub. The package was archived without fair warning.

Far from being "broken" I would argue {targets} was 99.9% working, probably more so than many packages without test suites as comprehensive. It should have been deemed low risk of causing user pain and allocated some time period to make an update that reflected this.

Rather than being rewarded for his dilligence in being an exemplary maintainer, @wlandau was punished. This punishment acts as a deterrent to himself and others to exhaustively test on CRAN in the future.

I would argue this is an example of CRAN going beyond what is required to maintain the strong guarantees you allude to, and there are consequences for that which may have long tailed effects. It's worth discussing this with the community in my opinion.

MilesMcBain commented 3 years ago

Ooh and if we're weighing user pain in the balance we can't forget the effect of install.packages("targets") suddenly failing. Quite the violation of the principle of least surprise and does not create an impression of stability.

MilesMcBain commented 3 years ago

And P.P.S. I still have the {reprex} example up my sleeve, which is kind of the ultimate rebuttal to the notion that CRAN places end-user pain above all else.

wlandau commented 3 years ago

Thanks, @MilesMcBain. The targets archival was indeed problematic: https://github.com/cranchange/cran_histories/issues/10 (as was the time plotly was suddenly taken down: https://twitter.com/jaredlander/status/1347929978281844737).

As you said, regardless of who was the original intended priority, both users and developers are suffering. Non-action on our part is no longer working.

wlandau commented 3 years ago

I just reached out to the Linux Foundation for advice: https://www.linuxfoundation.org/about/contact/

The R community is deeply concerned about the policies, performance, and administration of the Comprehensive R Archive Network (CRAN: https://cran.r-project.org/). An effort is ongoing to petition for specific reforms (https://github.com/cranchange/cran_change.org/blob/main/letter.md), but petitioning at all may be too confrontational to work, and it does little to address the underlying issues of governance and sustainability. The Linux Foundation has invaluable expertise fostering healthy management of open source projects, and I am reaching out on behalf of the petition authors in order to seek advice on a constructive path forward. Your guidance in this area could help resolve a severe obstacle for thousands of R users, package developers, and organizations across academia, government, and industry. It would be amazing to have an email exchange or video call with Linux Foundation members who could guide us.

adamhsparks commented 3 years ago

That's a good action, @wlandau. I don't want to charge ahead and do irreparable damage to the community. CRAN is a fantastic resource and serves the R community (users and developers both) in an amazing capacity, but I'd welcome a way to have a constructive dialogue to address developers' concerns. The last thing we need or want is a bitter split in the community.

MilesMcBain commented 3 years ago

Yeah I do see value in making some further approaches to other people to get some insights into minimising collateral damage. Happy to pause the petition launch for now.

MilesMcBain commented 3 years ago

@gmbecker I don't think I thanked you yet for your contributions to this thread. I've learned a lot and I appreciate them.

MilesMcBain commented 3 years ago

I'm going to create a new "Next Steps" issue where we can talk about who we're going to contact.