Can testing be integrated into Dockerfile build?

[billdenney]

To ensure the validation of my infrastructure, I integrate testing into my Docker builds (see https://github.com/billdenney/Pharmacometrics-Docker). There are many tests in nlmixr (https://github.com/nlmixrdevelopment/nlmixr/tree/master/tests/testthat).

Could the docker build be updated to include installing and running the nlmixr tests?

[RichardHooijmaijers]

I suppose that running all testthat functions within docker is not that difficult. It will make building the docker substantially slower but would be good practice. I will look into this.

[billdenney]

Thanks! (My NONMEM/PsN builds take about 12 hours, but I like knowing that I only get an image if they are tested.)

[mattfidler]

Hi @billdenney

I think that the RxODE tests should run fine in nlmixr; However, the nlmixr tests themselves were created/run on Windows. Like NONMEM and other software packages, the estimation answers are different on Linux and Windows. This is a current gap for nlmixr tests is we don't have the expected answers on different platforms and compilers.

[billdenney]

Can the tests work within a reasonable tolerance? I'll give a quick shot at running the tests in Docker and see if I can make a reasonable variability allowance on them and submit that as a (set of?) PRs to nlmixr.

[mattfidler]

They may be very similar. The current implentation uses nlme, and sometimes I think they swap ETA orders, which is a bit frustrating.

[mattfidler]

I would love help here @billdenney. I think the current docker is a ubuntu gcc, right?

[billdenney]

The current Docker is rocker; it appears to be based on debian:testing (which would presumably use gcc): https://github.com/rocker-org/rocker/blob/master/r-base/Dockerfile

When I just tried to run the tests from the installed package, they all were skipped. I didn't find anything specifically skipping the tests. Can you point me to where the skipping occurs?

[mattfidler]

The skipping occurs based on environmental variables.

nlmixrValidate runs all the code if you wish to use that;

If you want to run it via R and testthat, you can set the following environmental variables.

Sys.setenv(NLMIXR_VALIDATION_FULL="false", "NOT_CRAN"="true")

Most of the skips are because CRAN doesn't have the time to run all the models as well as CRAN doesn't have python.

[mattfidler]

Also set RxODE_VALIDATION_FULL to do a full RxODE validation. This includes elements needed for nlmixr FOCEi

[billdenney]

Initial testing succeeded with 196 tests and failed 11 (with an odd error that appears related to testthat and not nlmixr making me not certain that the numbers are everything). Your concerns. are correct; most values are <=3% off.

The only result that looked truly concerning that I saw off hand was:

── 9. Failure: ODE (@test-model68.R#78)  ──────────────────────────────────────────
signif(as.numeric(z[4, "StdDev"]), 3) not equal to 0.000205.
1/1 mismatches
[1] 1.79e-07 - 0.000205 == -0.000205

(FYI, this was with my docker image not the stock image. Results could be slightly different in the stock docker image because of slightly different versions of the underlying libraries.)

[billdenney]

I think that the discussion of actual testing belongs in an issue for the main nlmixr repository. I'm opening an issue there for that part of the discussion. (The issue here for testing the Docker container still applies after we fix the cross-platform nature of the testing.)