Run tests/testthat.R in the actions pipeline

[paulrougieux]

The automated test suite should be run in a Github pipeline /action. I'm not sure this is a requirement of JOSS, but systematically running the test suite at each commit helps catching bugs early. The review guidelines ask for:

"Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?"

Related to the JOSS review https://github.com/openjournals/joss-reviews/issues/4567

[paulrougieux]

After a fresh install of the development version of COINr

devtools::install_github("bluefoxr/COINr")

When I run the test suite on my laptop

setwd("~/repos/COINr/tests")
source("testthat.R")

I get the following error:

    "Sub-index", "Index"), quietly = FALSE)`: Entries in iMeta$iCode not found in colnames(iData).
Backtrace:
    ▆
 1. └─COINr::change_ind(coin2, add = "LPI", regen = TRUE) at test-regen.R:49:2
 2.   ├─COINr::Regen(coin, quietly = TRUE)
 3.   └─COINr:::Regen.coin(coin, quietly = TRUE)
 4.     ├─base::suppressMessages(do.call(func, args = f_log))
 5.     │ └─base::withCallingHandlers(...)
 6.     ├─base::do.call(func, args = f_log)
 7.     └─COINr::new_coin(...)
── Error (test-results.R:26:3): unit_summary ───────────────────────────────────
Error in `round(y, decimals)`: invalid second argument of length 0
Backtrace:
    ▆
 1. └─COINr::get_unit_summary(...) at test-results.R:26:2
 2.   └─COINr::round_df(tabout, nround)
 3.     ├─base::data.frame(...)
 4.     └─base::lapply(df, function(y) if (is.numeric(y)) round(y, decimals) else y)
 5.       └─COINr (local) FUN(X[[i]], ...)
── Error (test-trends.R:15:3): approx_df ───────────────────────────────────────
Error in `approx_df(Y, tt, 2012)`: could not find function "approx_df"
── Error (test-trends.R:32:3): CAGR ────────────────────────────────────────────
Error in `CAGR(y, x)`: could not find function "CAGR"
── Error (test-trends.R:45:3): prc_change ──────────────────────────────────────
Error in `prc_change(y, x)`: could not find function "prc_change"
── Error (test-trends.R:58:3): get_trends ──────────────────────────────────────
Error in `get_trends(purse, dset = "Raw")`: could not find function "get_trends"

[ FAIL 15 | WARN 0 | SKIP 1 | PASS 299 ]
Error: Test failures
>

[paulrougieux]

Sorry after a restart (I don't know why a restart was necessary), but anyway, after a restart the test suite is successfull on my laptop and returns the following:

Optimisation successful!
[ FAIL 0 | WARN 0 | SKIP 3 | PASS 342 ]

══ Skipped tests ══════════════════════════════════════════════════════════════════════════════
• empty test (1)
• On CRAN (2)

[ FAIL 0 | WARN 0 | SKIP 3 | PASS 342 ]
>

[bluefoxr]

Thanks for this comment. I have now added the following GitHub actions:

• CMD check
• Test coverage
• pkgdown site build

The CMD check has tests built in and also runs them on different platforms so this seems the most thorough way to automate the testing. I added the other two since they are also useful - update of test coverage and rebuilding the pkgdown site on every commit (previously I was doing this locally). The result of the CMD check is always displayed on the badge at the README page: https://github.com/bluefoxr/COINr

Let me know if this is OK to close the issue. Thanks.

[paulrougieux]

Looks great. I usually use gitlab CI and I am not knowledgable about Github workflows, where can I see the console output of tests/testthat.R ?
I was trying to look in the workflow called "test coverage" https://github.com/bluefoxr/COINr/actions/runs/3172012270/jobs/5166029329 I see "Run covr::codecov(quiet = FALSE)" but I don't see the output of the actual tests being run. Where should I look for that?

[bluefoxr]

Yes to be honest I couldn't see how to run only the tests. However the CMD check runs all tests as part of the check, so if a test fails, the CMD check will fail and it will output the error. You can see the output of the CMD check e.g. here, and the tests are being run, albeit without a detailed output:

Since the CMD check is anyway very useful for the package, would you agree that this is sufficient? I could possibly add another action specifically for the tests, but then that would run the testing twice.

[paulrougieux]

More details on which test pass or not might be helpful, but I guess R CMD check is good enough to make sure the test suite passes in its entirety. You can close this issue.

[bluefoxr]

I think that in the event a test doesn't pass, it will output the details of the test and line number etc. Anyway thanks and I will close the issue.