Tests randomly failing for cached objects

[zkamvar]

I was emailed a few days ago about tests that are intermittently failing on cached objects. The troubling thing about these failures is that they appear to add an extra week to the dates. I'm not sure why these appeared and how to reproduce since travis seems to be working fine.... wait... I had changed the repo to be checked on travis monthly as opposed to weekly 😖

On 03/01/2019 18:47, Prof Brian Ripley wrote: See https://cran.r-project.org/web/checks/check_results_incidence.html (which is still updating: all the failing checks were done in 2019). Please correct ASAP and before Jan 19 to safely retain the package on CRAN. This seems intermittent, so for the record the failure was

Check: tests
Result: ERROR
Running ‘testthat.R’ [2s/4s]
Running the tests in ‘tests/testthat.R’ failed.
Complete output:
library(testthat)
library(incidence)

test_check("incidence")
── 1. Failure: Printing returns the object (@test-incidence.R#330) ────────────
capture.output(print(z)) has changed from known value recorded in 'rds/print3.rds'.
5/11 mismatches
x[2]: "[51 cases from days 2015-12-28 to 2016-03-28]"
y[2]: "[51 cases from days 2015-12-28 to 2016-04-04]"

x[3]: "[51 cases from ISO weeks 2015-W53 to 2016-W13]"
y[3]: "[51 cases from ISO weeks 2015-W53 to 2016-W14]"

x[5]: "$counts: matrix with 14 rows and 1 columns"
y[5]: "$counts: matrix with 15 rows and 1 columns"

x[7]: "$dates: 14 dates marking the left-side of bins"
y[7]: "$dates: 15 dates marking the left-side of bins"

x[9]: "$timespan: 92 days"
y[9]: "$timespan: 99 days"

══ testthat results ═══════════════════════════════════════════════════════════
OK: 211 SKIPPED: 18 FAILED: 1
1. Failure: Printing returns the object (@test-incidence.R#330)

Error: testthat unit tests failed
Execution halted

-- 
Brian D. Ripley,                  ripley@stats.ox.ac.uk
Emeritus Professor of Applied Statistics, University of Oxford

[zkamvar]

Note that using saved references for these tests is a bit ham-fisted since it's much easier for someone to accidentally overwrite the binaries if they are moving too fast (which @jrcpulliam was successful at avoiding in 134b0c57ffd0e22e4dcaca377c558caf03c62992 ^_^). My plan for this fix is to re-write these tests so that they don't rely on the saved binaries.

[zkamvar]

As usual, this was a PBKC-type error ⌨️ 🤦‍♂️, aka Zhian trying to be too clever:

https://github.com/reconhub/incidence/blob/68ddb302560e28a5702d32b37838a05746f0f3b0/tests/testthat/test-incidence.R#L4-L15

The above code shows a couple of things:

1. Line 4: The random seed is set randomly to the result of evaluating the current date as an expression (i.e. 2019-01-08 would be equal to 2010).
2. Line 14: I overwrite an existing definition of dat and enforce that I always have 2015-12-28
3. On line 14, I forget to ensure that I always have 2016-04-09 (100 days from 2015-12-31).

library("incidence")
test_seed <- function(i) {
  the_seed <- eval(parse(text = as.character(i)))
  cat("seed: ", the_seed, "\n")
  set.seed(the_seed)
  dat <- as.integer(c(-3, sample(-3:100, 50, replace = TRUE)))
  dat_dates <- as.Date("2015-12-31") + dat
  incidence(dat_dates, interval = 7)
}
test_seed("2019-01-08")
#> seed:  2010
#> <incidence object>
#> [51 cases from days 2015-12-28 to 2016-04-04]
#> [51 cases from ISO weeks 2015-W53 to 2016-W14]
#> 
#> $counts: matrix with 15 rows and 1 columns
#> $n: 51 cases in total
#> $dates: 15 dates marking the left-side of bins
#> $interval: 7 days
#> $timespan: 99 days
#> $cumulative: FALSE
test_seed("2019-01-03")
#> seed:  2015
#> <incidence object>
#> [51 cases from days 2015-12-28 to 2016-03-28]
#> [51 cases from ISO weeks 2015-W53 to 2016-W13]
#> 
#> $counts: matrix with 14 rows and 1 columns
#> $n: 51 cases in total
#> $dates: 14 dates marking the left-side of bins
#> $interval: 7 days
#> $timespan: 92 days
#> $cumulative: FALSE

^{Created on 2019-01-08 by the reprex package (v0.2.1)}