lhs package reverse dependency checks fail

[bertcarnell]

The CRAN reverse dependency checks of my new lhs package failed against GPftit yesterday.

Old version: lhs 0.16 on CRAN New version: lhs 1.0 on github

Error

The error message was:

CRAN teams' auto-check service
Package check result: OK

Changes to worse in reverse depends:

Package: GPfit
 Check: tests
 New result: ERROR
    Running ‘testthat.R’ [7s/7s]
    Running the tests in ‘tests/testthat.R’ failed.
    Complete output:
      > library(testthat)
      > library(GPfit)
      >
      > test_check("GPfit", reporter = "summary")
      GP_deviance: ..1
      GP_fit creates GP model: ..............
      corr_matrix: ......
      GP model for predictions: ..............
══ Failed═════════════════════════════════════════════════════════
      ── 1. Failure: check GP_deviance (@test_GP_deviance.R#45)  ─────────────────────
      `res` not equal to 37.19008.
      1/1 mismatches
      [1] 39.2 - 37.2 == 2.03

══ DONE════════════════════════════════════════════════════════
      No-one gets it right on their first try
      Error: Test failures
      Execution halted

The message was generated by this test

Suggestion

I suggest that you change the test to be less precise. Since this was a major upgrade in the lhs package, I wasn't able to maintain backward compatibility with the random number generation in maximinLHS

require(lhs)
require(GPfit)
require(testthat)

packageVersion('lhs')
packageVersion('GPfit')

n <- 7
d <- 1
computer_simulator <- function(x) {
  log(x + 0.1) + sin(5 * pi * x)
}
set.seed(1)
x <- lhs::maximinLHS(n, d)
y <- computer_simulator(x)
beta <- rnorm(1)
res <- GP_deviance(
  beta = beta,
  X = x,
  Y = y,
  corr = list(
    type = "matern",
    nu = 5/2))
expect_equal(
  object = res, 
  expected = 37.1900808777125)

# option 1 - keep the maximinLHS, but loosen the test
expect_true(res > 0)
expect_true(res < 1E9)

# option 2 - switch to a latin hypercube that is less likely to change the random draw
#  but still might have 32 bit vs 64 bit issues
set.seed(1)
x <- matrix((sample(1:n, replace=FALSE) - runif(n, 0, 1))/n, ncol=1)
y <- computer_simulator(x)
beta <- rnorm(1)
res <- GP_deviance(
  beta = beta,
  X = x,
  Y = y,
  corr = list(
    type = "matern",
    nu = 5/2))
expect_equal(
  object = res, 
  expected = 11.11481, tolerance = 1E-5)

############################
# Install New version of lhs

require(devtools)
devtools::install_github("bertcarnell/lhs")

packageVersion('lhs')

set.seed(1)
x <- lhs::maximinLHS(n, d)
y <- computer_simulator(x)
beta <- rnorm(1)
res <- GP_deviance(
  beta = beta,
  X = x,
  Y = y,
  corr = list(
    type = "matern",
    nu = 5/2))
# this does not pass on Windows 7 64bit
#expect_equal(
#  object = res, 
#  expected = 37.1900808777125)

# option 1 - keep the maximinLHS, but loosen the test
expect_true(res > 0)
expect_true(res < 1E9)

# option 2 - switch to a latin hypercube that is less likely to change the random draw
#  but still might have 32 bit vs 64 bit issues
set.seed(1)
x <- matrix((sample(1:n, replace=FALSE) - runif(n, 0, 1))/n, ncol=1)
y <- computer_simulator(x)
beta <- rnorm(1)
res <- GP_deviance(
  beta = beta,
  X = x,
  Y = y,
  corr = list(
    type = "matern",
    nu = 5/2))
expect_equal(
  object = res, 
  expected = 11.11481, tolerance = 1E-5)

Request

Please let me know what you decide so that I can notify the CRAN maintainers about the pending lhs 1.0 version.

Thanks!

[CSJCampbell]

Thanks for the helpful explanation. I have replaced explicit call to lhs with numeric vector generated by current version of lhs at rev a810e4f. This should exercise the package as currently and not raise any issues due to changes in lhs. Leaving open until patch accepted by CRAN.

[bertcarnell]

GPfit 1.0-8 is on CRAN and passes check with lhs 1.0.1.