bitwise.ia() degraded performance on windows

[zkamvar]

It appears that bitwise.ia() is not performing well on Windows as it seems to overflow the stack too early. This test in particular fails on both windows flavors:

https://github.com/grunwaldlab/poppr/blob/0ab416f30c78f08fe23fe7a079b1900449a54e94/tests/testthat/test-values.R#L340-L347

here's the error summary:

== Warnings ====================================================================
  -- Warning (test-values.R:345:3): bitwise.ia can handle large samples ----------

  An integer overflow has occured and the resulting index will not be accurate.
  Please consider using a smaller sample.
  Backtrace:
   1. poppr::bitwise.ia(x[, snps], thread = 1L) test-values.R:345:2

  == Failed tests ================================================================
  -- Failure (test-values.R:346:3): bitwise.ia can handle large samples ----------
  `res` not equal to 8.6296328853274e-06.
  1/1 mismatches
  [1] -2.21 - 8.63e-06 == -2.21

R CMD check output on winbuilder

``` * using log directory 'd:/RCompile/CRANguest/R-release/poppr.Rcheck' * using R version 4.0.3 (2020-10-10) * using platform: x86_64-w64-mingw32 (64-bit) * using session charset: ISO8859-1 * checking for file 'poppr/DESCRIPTION' ... OK * checking extension type ... Package * this is package 'poppr' version '2.8.7' * package encoding: UTF-8 * checking CRAN incoming feasibility ... WARNING Maintainer: 'Zhian N. Kamvar ' Insufficient package version (submitted: 2.8.7, existing: 2.8.7) * checking package namespace information ... OK * checking package dependencies ... OK * checking if this is a source package ... OK * checking if there is a namespace ... OK * checking for hidden files and directories ... OK * checking for portable file names ... OK * checking serialization versions ... OK * checking whether package 'poppr' can be installed ... OK * checking installed package size ... OK * checking package directory ... OK * checking for future file timestamps ... OK * checking 'build' directory ... OK * checking DESCRIPTION meta-information ... OK * checking top-level files ... OK * checking for left-over files ... OK * checking index information ... OK * checking package subdirectories ... OK * checking R files for non-ASCII characters ... OK * checking R files for syntax errors ... OK * loading checks for arch 'i386' ** checking whether the package can be loaded ... OK ** checking whether the package can be loaded with stated dependencies ... OK ** checking whether the package can be unloaded cleanly ... OK ** checking whether the namespace can be loaded with stated dependencies ... OK ** checking whether the namespace can be unloaded cleanly ... OK ** checking loading without being on the library search path ... OK ** checking use of S3 registration ... OK * loading checks for arch 'x64' ** checking whether the package can be loaded ... OK ** checking whether the package can be loaded with stated dependencies ... OK ** checking whether the package can be unloaded cleanly ... OK ** checking whether the namespace can be loaded with stated dependencies ... OK ** checking whether the namespace can be unloaded cleanly ... OK ** checking loading without being on the library search path ... OK ** checking use of S3 registration ... OK * checking dependencies in R code ... OK * checking S3 generic/method consistency ... OK * checking replacement functions ... OK * checking foreign function calls ... OK * checking R code for possible problems ... [46s] OK * checking Rd files ... OK * checking Rd metadata ... OK * checking Rd line widths ... OK * checking Rd cross-references ... OK * checking for missing documentation entries ... OK * checking for code/documentation mismatches ... OK * checking Rd \usage sections ... OK * checking Rd contents ... OK * checking for unstated dependencies in examples ... OK * checking contents of 'data' directory ... OK * checking data for non-ASCII characters ... OK * checking data for ASCII and uncompressed saves ... OK * checking line endings in C/C++/Fortran sources/headers ... OK * checking line endings in Makefiles ... OK * checking compilation flags in Makevars ... OK * checking for GNU extensions in Makefiles ... OK * checking for portable use of $(BLAS_LIBS) and $(LAPACK_LIBS) ... OK * checking use of PKG_*FLAGS in Makefiles ... OK * checking use of SHLIB_OPENMP_*FLAGS in Makefiles ... OK * checking pragmas in C/C++ headers and code ... OK * checking compiled code ... OK * checking sizes of PDF files under 'inst/doc' ... OK * checking installed files from 'inst/doc' ... OK * checking files in 'vignettes' ... OK * checking examples ... ** running examples for arch 'i386' ... [48s] OK ** running examples for arch 'x64' ... [58s] OK * checking for unstated dependencies in 'tests' ... OK * checking tests ... ** running tests for arch 'i386' ... [24s] ERROR Running 'test-all.R' [24s] Running the tests in 'tests/test-all.R' failed. Complete output: > library(testthat) > options(poppr.debug = TRUE) > test_check("poppr") Loading required package: poppr Loading required package: adegenet Loading required package: ade4 /// adegenet 2.1.3 is loaded //////////// > overview: '?adegenet' > tutorials/doc/questions: 'adegenetWeb()' > bug reports/feature requests: adegenetIssues() This is poppr version 2.8.7. To get started, type package?poppr OMP parallel support: available == Skipped tests =============================================================== * On CRAN (186) == Warnings ==================================================================== -- Warning (test-values.R:345:3): bitwise.ia can handle large samples ---------- An integer overflow has occured and the resulting index will not be accurate. Please consider using a smaller sample. Backtrace: 1. poppr::bitwise.ia(x[, snps], thread = 1L) test-values.R:345:2 == Failed tests ================================================================ -- Failure (test-values.R:346:3): bitwise.ia can handle large samples ---------- `res` not equal to 8.6296328853274e-06. 1/1 mismatches [1] -2.21 - 8.63e-06 == -2.21 [ FAIL 1 | WARN 1 | SKIP 186 | PASS 364 ] Error: Test failures Execution halted ** running tests for arch 'x64' ... [25s] ERROR Running 'test-all.R' [25s] Running the tests in 'tests/test-all.R' failed. Complete output: > library(testthat) > options(poppr.debug = TRUE) > test_check("poppr") Loading required package: poppr Loading required package: adegenet Loading required package: ade4 /// adegenet 2.1.3 is loaded //////////// > overview: '?adegenet' > tutorials/doc/questions: 'adegenetWeb()' > bug reports/feature requests: adegenetIssues() This is poppr version 2.8.7. To get started, type package?poppr OMP parallel support: available == Skipped tests =============================================================== * On CRAN (186) == Warnings ==================================================================== -- Warning (test-values.R:345:3): bitwise.ia can handle large samples ---------- An integer overflow has occured and the resulting index will not be accurate. Please consider using a smaller sample. Backtrace: 1. poppr::bitwise.ia(x[, snps], thread = 1L) test-values.R:345:2 == Failed tests ================================================================ -- Failure (test-values.R:346:3): bitwise.ia can handle large samples ---------- `res` not equal to 8.6296328853274e-06. 1/1 mismatches [1] -2.21 - 8.63e-06 == -2.21 [ FAIL 1 | WARN 1 | SKIP 186 | PASS 364 ] Error: Test failures Execution halted * checking for unstated dependencies in vignettes ... OK * checking package vignettes in 'inst/doc' ... OK * checking re-building of vignette outputs ... [26s] OK * checking PDF version of manual ... OK * checking for detritus in the temp directory ... OK * DONE Status: 2 ERRORs, 1 WARNING ```

[zkamvar]

N.B. I realized that I never tested bitwise.ia() on Windows because the NOT_CRAN was never assigned on Appveyor and so those tests were never run. Luckily, this one test with bitwise.ia() is the only one that fails. Here are the results after I comment out the not_cran() from bitwise.ia() tests:

== Warnings ====================================================================
  -- Warning (test-values.R:345:3): bitwise.ia can handle large samples ----------

  An integer overflow has occured and the resulting index will not be accurate.
  Please consider using a smaller sample.
  Backtrace:
   1. poppr::bitwise.ia(x[, snps], thread = 1L) test-values.R:345:2

  == Failed tests ================================================================
  -- Failure (test-values.R:346:3): bitwise.ia can handle large samples ----------
  `res` not equal to 8.6296328853274e-06.
  1/1 mismatches
  [1] -2.21 - 8.63e-06 == -2.21

  [ FAIL 1 | WARN 1 | SKIP 182 | PASS 376 ]

R CMD check output on winbuilder

``` * using log directory 'd:/RCompile/CRANguest/R-release/poppr.Rcheck' * using R version 4.0.3 (2020-10-10) * using platform: x86_64-w64-mingw32 (64-bit) * using session charset: ISO8859-1 * checking for file 'poppr/DESCRIPTION' ... OK * checking extension type ... Package * this is package 'poppr' version '2.8.7' * package encoding: UTF-8 * checking CRAN incoming feasibility ... WARNING Maintainer: 'Zhian N. Kamvar ' Insufficient package version (submitted: 2.8.7, existing: 2.8.7) * checking package namespace information ... OK * checking package dependencies ... OK * checking if this is a source package ... OK * checking if there is a namespace ... OK * checking for hidden files and directories ... OK * checking for portable file names ... OK * checking serialization versions ... OK * checking whether package 'poppr' can be installed ... OK * checking installed package size ... OK * checking package directory ... OK * checking for future file timestamps ... OK * checking 'build' directory ... OK * checking DESCRIPTION meta-information ... OK * checking top-level files ... OK * checking for left-over files ... OK * checking index information ... OK * checking package subdirectories ... OK * checking R files for non-ASCII characters ... OK * checking R files for syntax errors ... OK * loading checks for arch 'i386' ** checking whether the package can be loaded ... OK ** checking whether the package can be loaded with stated dependencies ... OK ** checking whether the package can be unloaded cleanly ... OK ** checking whether the namespace can be loaded with stated dependencies ... OK ** checking whether the namespace can be unloaded cleanly ... OK ** checking loading without being on the library search path ... OK ** checking use of S3 registration ... OK * loading checks for arch 'x64' ** checking whether the package can be loaded ... OK ** checking whether the package can be loaded with stated dependencies ... OK ** checking whether the package can be unloaded cleanly ... OK ** checking whether the namespace can be loaded with stated dependencies ... OK ** checking whether the namespace can be unloaded cleanly ... OK ** checking loading without being on the library search path ... OK ** checking use of S3 registration ... OK * checking dependencies in R code ... OK * checking S3 generic/method consistency ... OK * checking replacement functions ... OK * checking foreign function calls ... OK * checking R code for possible problems ... [25s] OK * checking Rd files ... OK * checking Rd metadata ... OK * checking Rd line widths ... OK * checking Rd cross-references ... OK * checking for missing documentation entries ... OK * checking for code/documentation mismatches ... OK * checking Rd \usage sections ... OK * checking Rd contents ... OK * checking for unstated dependencies in examples ... OK * checking contents of 'data' directory ... OK * checking data for non-ASCII characters ... OK * checking data for ASCII and uncompressed saves ... OK * checking line endings in C/C++/Fortran sources/headers ... OK * checking line endings in Makefiles ... OK * checking compilation flags in Makevars ... OK * checking for GNU extensions in Makefiles ... OK * checking for portable use of $(BLAS_LIBS) and $(LAPACK_LIBS) ... OK * checking use of PKG_*FLAGS in Makefiles ... OK * checking use of SHLIB_OPENMP_*FLAGS in Makefiles ... OK * checking pragmas in C/C++ headers and code ... OK * checking compiled code ... OK * checking sizes of PDF files under 'inst/doc' ... OK * checking installed files from 'inst/doc' ... OK * checking files in 'vignettes' ... OK * checking examples ... ** running examples for arch 'i386' ... [28s] OK ** running examples for arch 'x64' ... [32s] OK * checking for unstated dependencies in 'tests' ... OK * checking tests ... ** running tests for arch 'i386' ... [12s] ERROR Running 'test-all.R' [11s] Running the tests in 'tests/test-all.R' failed. Complete output: > library(testthat) > options(poppr.debug = TRUE) > test_check("poppr") Loading required package: poppr Loading required package: adegenet Loading required package: ade4 /// adegenet 2.1.3 is loaded //////////// > overview: '?adegenet' > tutorials/doc/questions: 'adegenetWeb()' > bug reports/feature requests: adegenetIssues() This is poppr version 2.8.7. To get started, type package?poppr OMP parallel support: available == Skipped tests =============================================================== * On CRAN (182) == Warnings ==================================================================== -- Warning (test-values.R:345:3): bitwise.ia can handle large samples ---------- An integer overflow has occured and the resulting index will not be accurate. Please consider using a smaller sample. Backtrace: 1. poppr::bitwise.ia(x[, snps], thread = 1L) test-values.R:345:2 == Failed tests ================================================================ -- Failure (test-values.R:346:3): bitwise.ia can handle large samples ---------- `res` not equal to 8.6296328853274e-06. 1/1 mismatches [1] -2.21 - 8.63e-06 == -2.21 [ FAIL 1 | WARN 1 | SKIP 182 | PASS 376 ] Error: Test failures Execution halted ** running tests for arch 'x64' ... [13s] ERROR Running 'test-all.R' [12s] Running the tests in 'tests/test-all.R' failed. Complete output: > library(testthat) > options(poppr.debug = TRUE) > test_check("poppr") Loading required package: poppr Loading required package: adegenet Loading required package: ade4 /// adegenet 2.1.3 is loaded //////////// > overview: '?adegenet' > tutorials/doc/questions: 'adegenetWeb()' > bug reports/feature requests: adegenetIssues() This is poppr version 2.8.7. To get started, type package?poppr OMP parallel support: available == Skipped tests =============================================================== * On CRAN (182) == Warnings ==================================================================== -- Warning (test-values.R:345:3): bitwise.ia can handle large samples ---------- An integer overflow has occured and the resulting index will not be accurate. Please consider using a smaller sample. Backtrace: 1. poppr::bitwise.ia(x[, snps], thread = 1L) test-values.R:345:2 == Failed tests ================================================================ -- Failure (test-values.R:346:3): bitwise.ia can handle large samples ---------- `res` not equal to 8.6296328853274e-06. 1/1 mismatches [1] -2.21 - 8.63e-06 == -2.21 [ FAIL 1 | WARN 1 | SKIP 182 | PASS 376 ] Error: Test failures Execution halted * checking for unstated dependencies in vignettes ... OK * checking package vignettes in 'inst/doc' ... OK * checking re-building of vignette outputs ... [15s] OK * checking PDF version of manual ... OK * checking for detritus in the temp directory ... OK * DONE Status: 2 ERRORs, 1 WARNING ```