Anirban166 / testComplexity

Asymptotic complexity testing framework
https://anirban166.github.io/testComplexity/
Other
36 stars 2 forks source link

Test-time limits for CRAN checks #51

Closed Anirban166 closed 4 years ago

Anirban166 commented 4 years ago

As per CRAN's policies, specifically this one: (reminder: review checklist points once)
"Checking the package should take as little CPU time as possible, as the CRAN check farm is a very limited resource and there are thousands of packages. Long-running tests and vignette code can be made optional for checking, but do ensure that the checks that are left do exercise all the features of the package."

Care has to be taken to not exceed the time limits, which naturally is high for our package with the tests revolving around computation on increasing data sets for a log-linear (PDPA) and quadratic time (cDPA) algorithm. (other general tests & vignettes are fine, especially since I did not enable code running for the vignettes via knitr) I emailed the team to provide the precise time limits. Meanwhile, I checked this useful SO thread to skip tests on CRAN, but not on github (which is what I want ideally) - will follow up soon. (previously, an alternative would was to put the tests on another directory within /tests, which won't be running via a call to test() or R CMD check(), but can be manually run with test_dir(). But these options seem better)

Test time limits as provided by CRAN team member Uwe Ligges via mail:

Tentative expected release time would be within one to two weeks if all the automatic and manual checks pass (otherwise it depends on the number of iterations, expected time being typically less than a month)

Majority of test time is spent on the quadratic algorithm PeakSegDP::cDPA(), so will skip that for CRAN, but I think the other package checks with the PeakSegOptimal::PeakSegPDPA() test can finish under 10 minutes, as I observed for a devtools::check(). (adheers to the point of exercising the complexity classification/testing feature, + its the same example used throughout in the general vignette)