Features for checks API

[jeroen]

As a starting point the standard packages API can be used to get check status for the r-release for the various platforms: https://r-multiverse.r-universe.dev/api/packages?fields=_linuxdevel,_winbinary,_macbinary,_wasmbinary,_status,_buildurl

Where _status != "success" indicates problems with building vignettes and documentation. The other fields are binary build/checks. They can be "success" or "failure" (package was built, but there was a check error) or "none" (failed to build package for this platform) or "skipped".

Also note INLA which is what it looks like when cannot even build the source package.

What is not part of this API is checks for r-oldrel, which we od have for some platforms, although it usually gets dropped in the later stages of the R release cycle when we start testing for the next release of R. FWIW CRAN does not enforce checks for oldrel either.

[jeroen]

Cross reference: https://github.com/r-universe-org/help/issues/370

[wlandau]

Thank you @jeroen, this is excellent! I can get started on using this API in R-multiverse after I return from next week's vacation.

Is it true that "_winbinary", "_macbinary", and "_wasmbinary" include R CMD check but "_status" does not?

As far as the list of checks that would make sense for R-multiverse, what about these?

R version	platform	modifications to R CMD check
release	Mac
release	Windows
release	WASM
release	Linux
release	Linux	Clang-Asan/UBSan
release	Linux	Valgrind
devel	Linux

R-devel seems important so the community is not blindsided by every new version of R, but do we need it for all platforms?

[jeroen]

The windows and macos run CMD check. For wasm we only test if we can build. The status is the status of the primary job, which does the checkout, builds the source package, documentation, metadata, etc.

[jeroen]

We have r-devel for Linux and Windows, so I can add those as well.

UBSAN and Valgrind are complicated in many ways. Also the results are not easy to interpret. There is often no clear pass or fail for valgrind, it just emits a lot of diagnostic info that you have to comb through to look for anything concerning.

We can cross that bridge when we get there, hopefully we can go ahead and develop r-multiverse based on the checks that are available? Than we can always plug in another check when later on.

[shikokuchuo]

UBSAN and Valgrind both fall under 'future work' so we can indeed cross that bridge when we get there. Thanks.

[wlandau]

I agree, the absence of UBSAN/Valgrind does not block anything on the R-multiverse side.

[wlandau]

@jeroen, is it possible to view the response in ndjson so we can use jsonlite::stream_in() to stream in all the chunks of paginated output?

[wlandau]

Alternatively, do you have a recommendation for handling paginated output from https://multiverse.r-multiverse.org/api/packages?

[wlandau]

Also, under what conditions would e.g. _macbinary be skipped? Trying to think if we should allow some checks to be "skipped" for the production universe.

[jeroen]

@jeroen, is it possible to view the response in ndjson so we can use jsonlite::stream_in() to stream in all the chunks of paginated output?

Yes you can add &stream=true to the API url above.