search

poissonconsulting / bboutools

Boreal Caribou Survival, Recruitment and Population Growth Analysis R Package
https://poissonconsulting.github.io/bboutools/
Apache License 2.0
1 stars 1 forks source link

code coverage is throwing error #15

Closed aylapear closed 7 months ago

aylapear commented 7 months ago 
> covr::package_coverage()
Error: Failure in `/private/var/folders/yr/tq_q43k50m795hb4y0njhyth0000gn/T/RtmpLb4b1o/R_LIBS1598e4e529427/bboutools/bboutools-tests/testthat.Rout.fail`
> covr::package_coverage(quiet = FALSE, clean = FALSE)
* installing *source* package ‘bboutools’ ...
** using non-staged installation
** R
** demo
** tests
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
  converting help for package ‘bboutools’
    augment.bboufit                         example
    augment.bboufit_ml                      example
    bb_fit_recruitment                      example
    bb_fit_recruitment_ml                   example
    bb_fit_survival                         example
    bb_fit_survival_ml                      example
    coef.bboufit                            example
    coef.bboufit_ml                         example
    estimates.bboufit                       example
    estimates.bboufit_ml                    example
    glance.bboufit                          example
    glance.bboufit_ml                       example
    tidy.bboufit                            example
    tidy.bboufit_ml                         example
*** copying figures
** building package indices
** installing vignettes
** testing if installed package can be loaded
* DONE (bboutools)
Running specific tests for package ‘bboutools’
  Running ‘testthat.R’
Error: Failure in `/private/var/folders/yr/tq_q43k50m795hb4y0njhyth0000gn/T/RtmpLb4b1o/R_LIBS1598e4435b75c/bboutools/bboutools-tests/testthat.Rout.fail`

Code coverage is erroring.

aylapear commented 7 months ago 
Error: Failure in `/private/var/folders/yr/tq_q43k50m795hb4y0njhyth0000gn/T/RtmpHOQoKC/R_LIBS15c727423e2d4/bboutools/bboutools-tests/testthat.Rout.fail`
ct-growth.R:54:3'): bb_predict_growth 1 year each and no years common ──
Error in `checkReservedVarNames(code)`: Found variable name that conflicts with C++ keywords: if, if, if, if. Please use a different name.
Backtrace:
    ▆
 1. └─bboutools::bb_fit_survival(...) at test-predict-growth.R:54:3
 2.   └─bboutools::model_survival(...)
 3.     └─nimble::nimbleModel(...)
 4.       └─md$setupModel(...)
 5.         └─nimble:::checkReservedVarNames(code)

[ FAIL 2 | WARN 0 | SKIP 69 | PASS 120 ]
Error: Test failures
Execution halted

Appears a function may be failing and causing the issue

aylapear commented 7 months ago

To test this I deleted all the test files and am adding them back in batches. 

Running specific tests for package ‘bboutools’
  Running ‘testthat.R’
Error: Failure in `/private/var/folders/yr/tq_q43k50m795hb4y0njhyth0000gn/T/RtmpHOQoKC/R_LIBS15c726597cf6f/bboutools/bboutools-tests/testthat.Rout.fail`

This error appeared when the test-zzzfit-recruitment-ml.R and test-zzzfit-recruitment.R files were added back in.

aylapear commented 7 months ago

It is unhappy with the test-zzzfit-recruitment-ml.R file. 

Error: Failure in `/private/var/folders/yr/tq_q43k50m795hb4y0njhyth0000gn/T/RtmpHOQoKC/R_LIBS15c726d107003/bboutools/bboutools-tests/testthat.Rout.fail`
f. Please use a different name.
Backtrace:
    ▆
 1. └─bboutools::bb_fit_recruitment_ml(...) at test-zzzfit-recruitment-ml.R:96:3
 2.   └─bboutools::model_recruitment(...)
 3.     └─nimble::nimbleModel(...)
 4.       └─md$setupModel(...)
 5.         └─nimble:::checkReservedVarNames(code)

[ FAIL 6 | WARN 0 | SKIP 52 | PASS 104 ]
Error: Test failures
Execution halted



The test-zzzfit-recruitment.R file fails due to nimble:::checkReservedVarNames(code)

> covr::package_coverage(quiet = FALSE, clean = FALSE)
* installing *source* package ‘bboutools’ ...
** using non-staged installation
** R
** demo
** tests
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
  converting help for package ‘bboutools’
    augment.bboufit                         example
    augment.bboufit_ml                      example
    bb_fit_recruitment                      example
    bb_fit_recruitment_ml                   example
    bb_fit_survival                         example
    bb_fit_survival_ml                      example
    coef.bboufit                            example
    coef.bboufit_ml                         example
    estimates.bboufit                       example
    estimates.bboufit_ml                    example
    glance.bboufit                          example
    glance.bboufit_ml                       example
    tidy.bboufit                            example
    tidy.bboufit_ml                         example
*** copying figures
** building package indices
** installing vignettes
** testing if installed package can be loaded
* DONE (bboutools)
Running specific tests for package ‘bboutools’
  Running ‘testthat.R’
Error: Failure in `/private/var/folders/yr/tq_q43k50m795hb4y0njhyth0000gn/T/RtmpHOQoKC/R_LIBS15c7246b4b887/bboutools/bboutools-tests/testthat.Rout.fail`
(...)
 3.     └─nimble::nimbleModel(...)
 4.       └─md$setupModel(...)
 5.         └─nimble:::checkReservedVarNames(code)

[ FAIL 9 | WARN 0 | SKIP 52 | PASS 105 ]
Error: Test failures
Execution halted
aylapear commented 7 months ago

It appears that the files that cause the nimble:::checkReservedVarNames(code) failure are:

I feel that these files need to be excluded from the covr call or else it will fail. We need to investigate as to why nimble is causing this error.

aylapear commented 7 months ago

@sebdalgarno Can you look into why nimble is causing this issue with covr and if there is anything we can do about it? I noticed they don't have a GitHub so I could look up if the associated issue with nimble.

aylapear commented 7 months ago

We can exclude the set of files causing the issue until we are able to determine if this can be resolved. It does mean our coverage is going to be lower than it is actually is as five files are being excluded.

> library(covr)
> package_coverage(
+   line_exclusions = 
+     list(
+     "test-zzzfit-recruitment.R",
+     "test-zzzfit-recruitment-ml.R",
+     "test-predict-survival.R",
+     "test-predict-growth.R",
+     "test-zzzfit-survival-ml.R"
+   )
+ )
bboutools Coverage: 48.95%
R/data-recruitment.R: 0.00%
R/data-survival.R: 0.00%
R/fit-recruitment.R: 0.00%
R/fit-survival.R: 0.00%
R/model-recruitment.R: 0.00%
R/model-survival.R: 0.00%
R/nimble.R: 0.00%
R/plot.R: 0.00%
R/getters.R: 11.11%
R/terms.R: 18.37%
R/utils.R: 31.82%
R/summary.R: 50.00%
R/model-code.R: 57.14%
R/chk.R: 69.70%
R/predict-growth.R: 80.00%
R/derived.R: 91.67%
R/universals.R: 93.02%
R/tidy.R: 95.35%
R/augment.R: 100.00%
R/chk-data.R: 100.00%
R/coef.R: 100.00%
R/extract.R: 100.00%
R/glance.R: 100.00%
R/logLik.R: 100.00%
R/nobs.R: 100.00%
R/plot-month-survival.R: 100.00%
R/plot-month.R: 100.00%
R/plot-year-growth.R: 100.00%
R/plot-year-recruitment.R: 100.00%
R/plot-year-survival.R: 100.00%
R/plot-year-trend.R: 100.00%
R/plot-year.R: 100.00%
R/predict-trend.R: 100.00%
R/predict.R: 100.00%
R/remove.R: 100.00%
R/test-helpers.R: 100.00%
R/vld.R: 100.00%
aylapear commented 7 months ago

I have excluded the chk related files and they don't appear in the report.

> package_coverage(line_exclusions = list("R/chk.R", "R/chk-data.R") )
bboutools Coverage: 45.63%
R/data-recruitment.R: 0.00%
R/data-survival.R: 0.00%
R/fit-recruitment.R: 0.00%
R/fit-survival.R: 0.00%
R/model-recruitment.R: 0.00%
R/model-survival.R: 0.00%
R/nimble.R: 0.00%
R/plot.R: 0.00%
R/getters.R: 11.11%
R/terms.R: 18.37%
R/utils.R: 31.82%
R/summary.R: 50.00%
R/model-code.R: 57.14%
R/predict-growth.R: 80.00%
R/derived.R: 91.67%
R/universals.R: 93.02%
R/tidy.R: 95.35%
R/augment.R: 100.00%
R/coef.R: 100.00%
R/extract.R: 100.00%
R/glance.R: 100.00%
R/logLik.R: 100.00%
R/nobs.R: 100.00%
R/plot-month-survival.R: 100.00%
R/plot-month.R: 100.00%
R/plot-year-growth.R: 100.00%
R/plot-year-recruitment.R: 100.00%
R/plot-year-survival.R: 100.00%
R/plot-year-trend.R: 100.00%
R/plot-year.R: 100.00%
R/predict-trend.R: 100.00%
R/predict.R: 100.00%
R/remove.R: 100.00%
R/test-helpers.R: 100.00%
R/vld.R: 100.00%

You can see the excluded argument working when nothing provided as the chk files appear now

> package_coverage()
bboutools Coverage: 48.95%
R/data-recruitment.R: 0.00%
R/data-survival.R: 0.00%
R/fit-recruitment.R: 0.00%
R/fit-survival.R: 0.00%
R/model-recruitment.R: 0.00%
R/model-survival.R: 0.00%
R/nimble.R: 0.00%
R/plot.R: 0.00%
R/getters.R: 11.11%
R/terms.R: 18.37%
R/utils.R: 31.82%
R/summary.R: 50.00%
R/model-code.R: 57.14%
R/chk.R: 69.70%
R/predict-growth.R: 80.00%
R/derived.R: 91.67%
R/universals.R: 93.02%
R/tidy.R: 95.35%
R/augment.R: 100.00%
R/chk-data.R: 100.00%
R/coef.R: 100.00%
R/extract.R: 100.00%
R/glance.R: 100.00%
R/logLik.R: 100.00%
R/nobs.R: 100.00%
R/plot-month-survival.R: 100.00%
R/plot-month.R: 100.00%
R/plot-year-growth.R: 100.00%
R/plot-year-recruitment.R: 100.00%
R/plot-year-survival.R: 100.00%
R/plot-year-trend.R: 100.00%
R/plot-year.R: 100.00%
R/predict-trend.R: 100.00%
R/predict.R: 100.00%
R/remove.R: 100.00%
R/test-helpers.R: 100.00%
R/vld.R: 100.00%

This is done with the problem test files deleted.



It doesn't appear it works for excluding the whole file for these cases

> package_coverage(line_exclusions = list("R/fit-survival.R", "R/model-survival.R"))
Error: Failure in `/private/var/folders/yr/tq_q43k50m795hb4y0njhyth0000gn/T/RtmpHOQoKC/R_LIBS15c722a5a9444/bboutools/bboutools-tests/testthat.Rout.fail`
─ Error ('test-predict-growth.R:37:3'): bb_predict_growth multiple years each and one year common ──
Error in `checkReservedVarNames(code)`: Found variable name that conflicts with C++ keywords: if, if, if, if. Please use a different name.
Backtrace:
    ▆
 1. └─bboutools::bb_fit_survival(...) at test-predict-growth.R:37:3
 2.   └─bboutools::model_survival(...)
 3.     └─nimble::nimbleModel(...)
 4.       └─md$setupModel(...)
 5.         └─nimble:::checkReservedVarNames(code)

[ FAIL 1 | WARN 0 | SKIP 58 | PASS 108 ]
Error: Test failures
Execution halted
aylapear commented 7 months ago

It does appear to work to skip the tests that are causing the error with skip_on_covr()

aylapear commented 7 months ago

This is better as it will help to test coverage to be as high as possible