fixest to be archived on CRAN

[grantmcdermott]

https://twitter.com/lrberge/status/1495874616153329669

Equal measures frustrating and annoying. But mostly counterproductive.

@zeileis is there someone you (we?) can speak to about a temporary reprieve? More generally, it seems clear that the current time limit is unreasonable given most people's schedules. I know that Laurent is currently balancing an severe amount of new tenure track faculty work in addition to vanilla parent-of-young-children stuff.

[rsbivand]

The remaining problem seems to be in https://www.stats.ox.ac.uk/pub/bdr/memtests/clang-UBSAN/fixest/tests/fixest_tests.Rout, at least in:

lm_related.cpp:146:22: runtime error: inf is outside the range of representable values of type 'int'

0 0x7fb1c8ea33ae in cpp_cholesky(Rcpp::Matrix<14, Rcpp::PreserveStorage>, double, int) /data/gannet/ripley/R/packages/tests-clang-SAN/fixest/src/lm_related.cpp:146:22

https://github.com/lrberge/fixest/blob/40ac466b0e677a03d4ad53f7bc09c5e2ac8a1bef/src/lm_related.cpp#L142-L146

which looks like sensitivity to integer overflow. Removing indicated SAN issues resolves odd crashes, so packages are expected to be SAN clean. This matters much more in packages with important reverse dependencies, because the odd crashes seem to users to come from packages importing from the package with the *SAN vulnerability. This is related to https://developer.r-project.org/Blog/public/2019/03/28/use-of-c-in-packages/index.html. @zeileis, @eddelbuettel can you see an obvious reason for the UBSAN issue (I only think that the UBSAN report is from the same code as that on HEAD). Is it just the iteration counter providing the possibility to interrupt?

[eddelbuettel]

Concur. SAN/ASAN/UBSAN is real and often indicative of issues. Little to suggest here other than package maintainers need to take this seriously, take advantage of existing tools (such as the https://github.com/wch/r-debug container building on/extending an earlier Rocker one by myself).

Empirically speaking, based on my reads of CRANberries, many packages also bounce back after a removal. It's still not a fun process and likely 'second-best' for all involved, but hey I don't make the rules. (And in fact, the rule-makers currently don't even reply to my emails concerning an upload now pending for well over a week and two trials....)

[rsbivand]

Thanks! Based on local infection levels here in my city, I'd expect slower turnaround rates than normal currently. Useful pointer to the container, is it also clang?

[rsbivand]

In the specific case where a 4-byte integer would overflow for K < 2, is a long integer available?

[grantmcdermott]

(Mini) crisis averted. Looks like Laurent managed to get under the wire with a fix for 0.10.3: https://cran.r-project.org/web/checks/check_results_fixest.html

Thanks all for comments and suggestions. My own take, which has been brewing for some time, is that these incredibly tight CRAN timelines are unreasonable for many maintainers. (Young kids, tenure track, COVID, etc.) It's just going to to lead to resentment and burnout. I know no-one here is "on the inside", so-to-speak, but hopefully we can push them to something more reasonable like four weeks.

[zeileis]

From previous discussions with the CRAN team I don't have the impression that the exact time of the dealine changes the reactions of the maintainers qualitatively. The problem from their side is that a very large share of maintainers does not act at all before it's too late (if then). I think the challenge would be to find an objective mechanism by which maintainers can credibly signal that they will fix the issues within in a certain time frame. Then the CRAN team could grant extensions in such cases.