FYI: sctransform 0.3.0 break package tests of Seurat

[HenrikBengtsson]

Hi, just want to let you know that the CRAN checks on Seurat started to fail when sctransform 0.3.0 reached CRAN; at least there's a strong correlation with the new and the old version number. The errors appear to be due to, numerically reproducible, but different numeric results:

Version: 3.2.1
Check: tests
Result: ERROR
     Running ‘testthat.R’ [39s/58s]
    Running the tests in ‘tests/testthat.R’ failed.
    Complete output:
     > library(testthat)
     > library(Seurat)
     >
     > test_check("Seurat")
     ── 1. Failure: SCTransform wrapper works as expected (@test_preprocessing.R#295)
     as.numeric(colSums(GetAssayData(object = object[["SCT"]], slot = "scale.data"))[1]) not equal to 13.3303864.
     1/1 mismatches
     [1] 11.4 - 13.3 == -1.93

     ── 2. Failure: SCTransform wrapper works as expected (@test_preprocessing.R#299)
     as.numeric(colSums(GetAssayData(object = object[["SCT"]], slot = "counts"))[1]) not equal to 123.
     1/1 mismatches
     [1] 129 - 123 == 6

     ── 3. Failure: SCTransform ncells param works (@test_preprocessing.R#312) ─────
     as.numeric(colSums(GetAssayData(object = object[["SCT"]], slot = "scale.data"))[1]) not equal to 11.834969847.
     1/1 mismatches
     [1] 9.41 - 11.8 == -2.42

     ── 4. Failure: SCTransform ncells param works (@test_preprocessing.R#316) ─────
     as.numeric(colSums(GetAssayData(object = object[["SCT"]], slot = "counts"))[1]) not equal to 121.
     1/1 mismatches
     [1] 119 - 121 == -2

     ── 5. Failure: SCTransform ncells param works (@test_preprocessing.R#317) ─────
     as.numeric(rowSums(GetAssayData(object = object[["SCT"]], slot = "counts"))[5]) not equal to 25.
     1/1 mismatches
     [1] 26 - 25 == 1

     ══ testthat results ═══════════════════════════════════════════════════════════
     [ OK: 458 | SKIPPED: 0 | WARNINGS: 3 | FAILED: 5 ]
     1. Failure: SCTransform wrapper works as expected (@test_preprocessing.R#295)
     2. Failure: SCTransform wrapper works as expected (@test_preprocessing.R#299)
     3. Failure: SCTransform ncells param works (@test_preprocessing.R#312)
     4. Failure: SCTransform ncells param works (@test_preprocessing.R#316)
     5. Failure: SCTransform ncells param works (@test_preprocessing.R#317)

     Error: testthat unit tests failed
     Execution halted

• https://cran.r-project.org/web/checks/check_results_Seurat.html

Flavor	Version	Tinstall	Tcheck	Ttotal	Status	Flags
r-devel-linux-x86_64-debian-clang	3.2.1	125.38	397.80	523.18	NOTE
r-devel-linux-x86_64-debian-gcc	3.2.1	97.95	295.78	393.73	ERROR
r-devel-linux-x86_64-fedora-clang	3.2.1			794.67	ERROR
r-devel-linux-x86_64-fedora-gcc	3.2.1			773.47	ERROR
r-devel-windows-ix86+x86_64	3.2.1	273.00	614.00	887.00	NOTE
r-patched-linux-x86_64	3.2.1	110.85	391.09	501.94	ERROR
r-patched-solaris-x86	3.2.1			682.40	ERROR
r-release-linux-x86_64	3.2.1	111.73	392.98	504.71	NOTE
r-release-macos-x86_64	3.2.1				NOTE
r-release-windows-ix86+x86_64	3.2.1	267.00	744.00	1011.00	NOTE
r-oldrel-macos-x86_64	3.2.1				NOTE
r-oldrel-windows-ix86+x86_64	3.2.1	186.00	621.00	807.00	NOTE

• https://cran.r-project.org/web/checks/check_results_sctransform.html

Flavor	Version	Tinstall	Tcheck	Ttotal	Status	Flags
r-devel-linux-x86_64-debian-clang	0.2.1	30.36	89.87	120.23	OK
r-devel-linux-x86_64-debian-gcc	0.3	27.92	71.74	99.66	NOTE
r-devel-linux-x86_64-fedora-clang	0.3			178.29	NOTE
r-devel-linux-x86_64-fedora-gcc	0.3			156.54	NOTE
r-devel-windows-ix86+x86_64	0.2.1	81.00	174.00	255.00	OK
r-patched-linux-x86_64	0.3	27.92	87.96	115.88	NOTE
r-patched-solaris-x86	0.3			202.80	NOTE
r-release-linux-x86_64	0.2.1	26.47	85.89	112.36	OK
r-release-macos-x86_64	0.2.1				OK
r-release-windows-ix86+x86_64	0.3	67.00	144.00	211.00	OK
r-oldrel-macos-x86_64	0.3				NOTE
r-oldrel-windows-ix86+x86_64	0.2.1	53.00	155.00	208.00	OK

PS. I noticed this because CRAN's reverse-dependency checks on my future packages incorrectly linked it to my update.

cc/ @satijalab

[ChristophH]

Thanks for the FYI

I noticed that you also submitted new versions of your packages around the same time I did submit sctransform v0.3. Don't understand how the reverse dependency checks would link the Seurat errors to the wrong package - don't they test each package one-by-one?

In any case, Seurat developers and I are aware of the breaking changes and a fix is underway.

[HenrikBengtsson]

I re-submitted again yesterday, because there was another, real revdep error. In the new submission, they got these new errors, probably because sctransform 0.3.0 is rolling out right now. It's probably because the old version of future (1.18.0) was tested with sctransform 0.2.1 and the new future (1.19.1) with sctransfrom 0.3.0.

[HenrikBengtsson]

BTW, if you're not running https://github.com/r-lib/revdepcheck, it's pretty useful and quite straightforward.

[HenrikBengtsson]

Feel free to close or leave it open for others who might wonder.

[HenrikBengtsson]

Seems to be fixed in Seurat 3.2.2