search

Genentech / jmpost

https://genentech.github.io/jmpost/
17 stars 4 forks source link

Cut `R CMD check` times down to < 10 minutes #264

Open gowerc opened 7 months ago

gowerc commented 7 months ago

This is not a priority issue as we don't currently have any plans to publish to CRAN (at least we can't until cmdstanr does) but if we ever did want to publish we can't unless the total run time of our vignettes and unit-tests is < 10 minutes.

Main issue at the moment is a fresh run on CRAN wouldn't have access to any cached models so would take a long time to run. Might need to be quite selective on which tests we enable on CRAN.

danielinteractive commented 7 months ago

Yep, and can also learn from psborrow2 for these things because it uses a similar framework. I think they are going for caching / pre-compiling quite a few things

gowerc commented 7 months ago

I'm not sure how that works ... ?

As in if they are caching via a local directory or CI/CD cache that won't be of much help for CRAN as the cache wont exist on CRAN servers.

If they are pre-compiling and storing that within their own package then surely you'd have to do that for all the different possible architectures which sounds overly burdensome.

I'm guessing you mean they are storing pre-computed numbers within the package ?

Though I guess maybe @gravesti can elaborate :) ?

danielinteractive commented 7 months ago

yeah you can sync with @gravesti and @mattsecrest on the approaches

gravesti commented 7 months ago

In psborrow we are doing something like this: https://ropensci.org/blog/2019/12/08/precompute-vignettes/ That is we knit the Rmd but don't run render to html or pdf, so the tables and figures are generated in the markdown and stored in the repo. This should allow us to get on CRAN even without cmdstanr. I have wrapped all the checks and tests that need sampling (results) in if statements to check if cmdstan and cmdstanr are available. That means we can still have examples and tests for all of the stuff which sets up the models.

gowerc commented 7 months ago

Ah gotcha, yer that makes sense. Though I think with jmpost at the moment that would wipe out 70% of our tests which isn't ideal, I guess though theres not really a good solution here until cmdstan is posted to CRAN.

Actually, it dawns on me that just getting cmdstan posted is only half the battle. Given its a CLI we need CRAN to actually install the CLI on all their servers :(

danielinteractive commented 7 months ago

And I would not hope too much for seeing cmdstanr making it to CRAN.