search

r-lib / usethis

Set up commonly used 📦 components
https://usethis.r-lib.org/
Other
861 stars 285 forks source link

Tooling to support CRAN submission with known breakages #1586

Open hadley opened 2 years ago

hadley commented 2 years ago

CRAN would appreciate it if we reported known failures in a standard format. We need both a human readable text format (for inclusion in the email that CRAN team can easily scan) and something machine readable that they can easily compute with (e.g. attached .rds)

Package: chunked
Check: R code for possible problems
New result: ERROR
Explanation: patch at https://github.com/edwindj/chunked/pull/21

Package: functiondepends
Check: re-building of vignette outputs
New result: WARNING
Explanation: patch at https://github.com/jakubsob/functiondepends/pull/2

We'll need to think about how we should track the needed data from running the revdeps, to tracking progress on issues/PRs, to final submission.

jennybc commented 2 years ago

Seems like it will require coordination with revdepcheck.

hadley commented 2 years ago

There's a little bit of coordination required, but the most important bit here is adding some standardised way for us to capture our responses to a revdepcheck failure.

jennybc commented 2 years ago

Ah, so also seems related to potentially standardising the human bit, where we track a set of revdepcheck failures in a spreadsheet or similar.

DavisVaughan commented 2 years ago

Anecdotal conversation with CRAN that can guide some of this:

We sent in a workflows submission with 3 known breakages. This was cran-comments.md:

## revdepcheck results

We checked 28 reverse dependencies (26 from CRAN + 2 from Bioconductor), comparing R CMD check results across CRAN and dev versions of this package.

 * We saw 3 new problems
 * We failed to check 0 packages

Issues with CRAN packages are summarised below.

### New problems
(This reports the first line of each new failure)

* modeltime
  checking tests ... ERROR

* modeltime.ensemble
  checking tests ... ERROR

* modeltime.resample
  checking re-building of vignette outputs ... WARNING

CRAN responded:

Nice you found these, too, but this is un helpful: Is this expected?
Have the maintainers been informed well in advance and when?

We actually had done a PR about this, but our standard tooling didn't give us a way to let CRAN know about it https://github.com/business-science/modeltime/pull/198

I responded further, asking if we could improve our process:

Is there a form of `cran-comments.md` that we (tidyverse, r-lib, tidymodels) could send you that would be most helpful to you and CRAN? Possibly something that is machine readable in some way for easy processing?

To which CRAN replied:

No. Ideally only information we need. In your case: Which rev deps
failed and what have you done to inform the maintainers.

So which nonstandard things have to be coinsidered that may prevent auto
publication and are supposed to be fine. And which additional issues
have  been fixed.

Machine readable does not help here, as we have to inspect the package
manually if auto publication was not possible. Ideally I do not want to
read what our checks will find anyway.
Most people include cran comment files with information from former
submissions, which is also irrelevant for us.

Honestly, I hate these cran-comments files, and would like to see the
comments that people want to tell us for the new submission. So what
people who do not use devtools would simply enter in the webform.

So it seems like:

hadley commented 1 year ago

I think we should update this for our latest workflow: managing a separate issue of all breakages and links to PRs. That seems to be working well for us.

DavisVaughan commented 1 year ago

My plan for cran-comments.md going forward was going to be:

This seems to hit the highlights of what CRAN told me they actually care about, while being minimal extra work for us

DavisVaughan commented 1 year ago

Here is what we are doing in cran-comments.md for tidyr 1.3.0 https://github.com/tidyverse/tidyr/blob/69b74ef967b6170fe2346c71236996f0a8e6347f/cran-comments.md