search

elbamos / largeVis

An implementation of the largeVis algorithm for visualizing large, high-dimensional datasets, for R
340 stars 62 forks source link

testcfunctions.cpp fails (Win)... what is it supposed to do? #51

Closed meowcat closed 6 years ago

meowcat commented 7 years ago

On a github clone with R-3.4.1, the tests from testcfunctions.cpp fail:

testthat results ================================================================
OK: 147 SKIPPED: 1 FAILED: 1
1. Failure: Catch unit tests pass (@test-cpp.R#6)

and above:

testcfunctions.cpp:9
...............................................................................

testcfunctions.cpp:17: FAILED:
  CATCH_CHECK( testAlias() == 71 )
with expansion:
  83 == 71

testcfunctions.cpp:18: FAILED:
  CATCH_CHECK( testAlias() == 74 )
with expansion:
  97 == 74

testcfunctions.cpp:19: FAILED:
  CATCH_CHECK( testAlias() == 70 )
with expansion:
  68 == 70

testcfunctions.cpp:20: FAILED:
  CATCH_CHECK( testAlias() == 90 )
with expansion:
  88 == 90

etc. But I don't really get what these tests are supposed to test? Looks like you are setting up the RNG with a seed and expect a certain output?

Is this expected to fail on Windows possibly?

elbamos commented 7 years ago

That is testing the implementation of the alias algorithm, which is used to efficiently draw from a weighted distribution. It is not expected to fail, and should not unless your compiler or implementation of the C++ standard library is funky.

Can you try varying versions of R and your compiler and see what happens? I don't have a windows machine to test on.

On Jul 14, 2017, at 11:54 AM, meowcat notifications@github.com wrote:

On a github clone with R-3.4.1, the tests from testcfunctions.cpp fail:

testthat results ================================================================ OK: 147 SKIPPED: 1 FAILED: 1

  1. Failure: Catch unit tests pass (@test-cpp.R#6)

and above:

testcfunctions.cpp:9 ...............................................................................

testcfunctions.cpp:17: FAILED: CATCH_CHECK( testAlias() == 71 ) with expansion: 83 == 71

testcfunctions.cpp:18: FAILED: CATCH_CHECK( testAlias() == 74 ) with expansion: 97 == 74

testcfunctions.cpp:19: FAILED: CATCH_CHECK( testAlias() == 70 ) with expansion: 68 == 70

testcfunctions.cpp:20: FAILED: CATCH_CHECK( testAlias() == 90 ) with expansion: 88 == 90 etc. But I don't really get what these tests are supposed to test? Looks like you are setting up the RNG with a seed and expect a certain output?

Is this expected to fail on Windows possibly?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

meowcat commented 7 years ago

Seems not to be Windows related. I installed the package from CRAN on my Linux VM and get the same error.

R version 3.4.1 (2017-06-30) -- "Single Candle"
Copyright (C) 2017 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(largeVis)
Loading required package: Matrix
> library(testthat)
> expect_cpp_tests_pass("largeVis")
Error: C++ unit tests:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 is a Catch v1.2.1 host application.
Run with -? for options

-------------------------------------------------------------------------------
alias tests | testcfunctions.cpp
  alias int succeeds
-------------------------------------------------------------------------------
testcfunctions.cpp:9
...............................................................................

testcfunctions.cpp:17: FAILED:
  CATCH_CHECK( testAlias() == 71 )
with expansion:
  83 == 71

testcfunctions.cpp:18: FAILED:
  CATCH_CHECK( testAlias() == 74 )
with expansion:
  97 == 74

testcfunctions.cpp:19: FAILED:
  CATCH_CHECK( testAlias() == 70 )
with expansion:
  68 == 70

testcfunctions.cpp:20: FAILED:
  CATC
> sessionInfo()
R version 3.4.1 (2017-06-30)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 17.04

Matrix products: default
BLAS: /usr/lib/libblas/libblas.so.3.7.0
LAPACK: /usr/lib/lapack/liblapack.so.3.7.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
 [9] LC_ADDRESS=C               LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] testthat_1.0.2 largeVis_0.2.1 Matrix_1.2-10

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.11     lattice_0.20-35  crayon_1.3.2     grid_3.4.1
 [5] plyr_1.8.4       R6_2.2.2         gtable_0.2.0     magrittr_1.5
 [9] scales_0.4.1     ggplot2_2.2.1    rlang_0.1.1      lazyeval_0.2.0
[13] munsell_0.4.3    compiler_3.4.1   colorspace_1.3-2 tibble_1.3.3
>
meowcat commented 7 years ago

Windows, R 3.4.0, installed from CRAN binary (not compiled myself AFAIK)


R version 3.4.0 (2017-04-21) -- "You Stupid Darkness"
Copyright (C) 2017 The R Foundation for Statistical Computing
Platform: x86_64-w64-mingw32/x64 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(testthat)
> library(largeVis)
Lade nötiges Paket: Matrix
> expect_cpp_tests_pass("largeVis")
Error: C++ unit tests:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 is a Catch v1.2.1 host application.
Run with -? for options

-------------------------------------------------------------------------------
alias tests | testcfunctions.cpp
  alias int succeeds
-------------------------------------------------------------------------------
testcfunctions.cpp:9
...............................................................................

testcfunctions.cpp:17: FAILED:
  CATCH_CHECK( testAlias() == 71 )
with expansion:
  83 == 71

testcfunctions.cpp:18: FAILED:
  CATCH_CHECK( testAlias() == 74 )
with expansion:
  97 == 74

testcfunctions.cpp:19: FAILED:
  CATCH_CHECK( testAlias() == 70 )
with expansion:
  68 == 70

testcfunctions.cpp:20: FAILED:
  CATCH_CHECK( testAlias() == 90 )
with expansion:
  88 == 90

-------------------------------------------------------------------------------
alias tests | testcfunctions.cpp
  alias long succeeds
-------
> sessionInfo()
R version 3.4.0 (2017-04-21)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1

Matrix products: default

locale:
[1] LC_COLLATE=German_Switzerland.1252  LC_CTYPE=German_Switzerland.1252    LC_MONETARY=German_Switzerland.1252 LC_NUMERIC=C                       
[5] LC_TIME=German_Switzerland.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] largeVis_0.2.2 Matrix_1.2-10  testthat_1.0.2

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.11     lattice_0.20-35  crayon_1.3.2     grid_3.4.0       R6_2.2.2         plyr_1.8.4       gtable_0.2.0     magrittr_1.5     scales_0.4.1    
[10] ggplot2_2.2.1    rlang_0.1.1      lazyeval_0.2.0   tools_3.4.0      munsell_0.4.3    compiler_3.4.0   colorspace_1.3-2 tibble_1.3.3
meowcat commented 7 years ago

Windows, some devel version from Feb 2016 (sic!) largeVis from CRAN, compiled while R 3.3.3 was up-to-date Same error

R Under development (unstable) (2016-02-08 r70124) -- "Unsuffered Consequences"
Copyright (C) 2016 The R Foundation for Statistical Computing
Platform: x86_64-w64-mingw32/x64 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(largeVis)
Lade nötiges Paket: Matrix
Warning messages:
1: Paket ‘largeVis’ wurde unter R Version 3.3.3 erstellt 
2: Paket ‘Matrix’ wurde unter R Version 3.3.1 erstellt 
> library(testthat)
Warning message:
Paket ‘testthat’ wurde unter R Version 3.3.3 erstellt 
> expect_cpp_tests_pass("largeVis")
Error: C++ unit tests:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 is a Catch v1.2.1 host application.
Run with -? for options

-------------------------------------------------------------------------------
alias tests | testcfunctions.cpp
  alias int succeeds
-------------------------------------------------------------------------------
testcfunctions.cpp:9
...............................................................................

testcfunctions.cpp:17: FAILED:
  CATCH_CHECK( testAlias() == 71 )
with expansion:
  83 == 71

testcfunctions.cpp:18: FAILED:
  CATCH_CHECK( testAlias() == 74 )
with expansion:
  97 == 74

testcfunctions.cpp:19: FAILED:
  CATCH_CHECK( testAlias() == 70 )
with expansion:
  68 == 70

testcfunctions.cpp:20: FAILED:
  CATCH_CHECK( testAlias() == 90 )
with expansion:
  88 == 90

-------------------------------------------------------------------------------
alias tests | testcfunctions.cpp
  alias long succeeds
-------
> sessionInfo()
R Under development (unstable) (2016-02-08 r70124)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1

locale:
[1] LC_COLLATE=German_Switzerland.1252  LC_CTYPE=German_Switzerland.1252    LC_MONETARY=German_Switzerland.1252 LC_NUMERIC=C                       
[5] LC_TIME=German_Switzerland.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] testthat_1.0.2 largeVis_0.2.1 Matrix_1.2-7.1

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.8      lattice_0.20-34  crayon_1.3.2     assertthat_0.1   grid_3.3.0       plyr_1.8.4       R6_2.1.3         gtable_0.2.0    
 [9] magrittr_1.5     scales_0.4.1     ggplot2_2.2.0    lazyeval_0.2.0   tools_3.3.0      munsell_0.4.3    colorspace_1.2-6 tibble_1.2
elbamos commented 7 years ago

Do you see the same thing with 3.4?

On Jul 18, 2017, at 4:44 AM, meowcat notifications@github.com wrote:

Seems not to be Windows related. I installed the package from CRAN on my Linux VM and get the same error.

R version 3.4.1 (2017-06-30) -- "Single Candle" Copyright (C) 2017 The R Foundation for Statistical Computing Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details.

Natural language support but running in an English locale

R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R.

library(largeVis) Loading required package: Matrix library(testthat) expect_cpp_tests_pass("largeVis") Error: C++ unit tests:


is a Catch v1.2.1 host application.
Run with -? for options

alias tests | testcfunctions.cpp alias int succeeds

testcfunctions.cpp:9 ...............................................................................

testcfunctions.cpp:17: FAILED: CATCH_CHECK( testAlias() == 71 ) with expansion: 83 == 71

testcfunctions.cpp:18: FAILED: CATCH_CHECK( testAlias() == 74 ) with expansion: 97 == 74

testcfunctions.cpp:19: FAILED: CATCH_CHECK( testAlias() == 70 ) with expansion: 68 == 70

testcfunctions.cpp:20: FAILED: CATC

sessionInfo() R version 3.4.1 (2017-06-30) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 17.04

Matrix products: default BLAS: /usr/lib/libblas/libblas.so.3.7.0 LAPACK: /usr/lib/lapack/liblapack.so.3.7.0

locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 [7] LC_PAPER=en_US.UTF-8 LC_NAME=C [9] LC_ADDRESS=C LC_TELEPHONE=C [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] testthat_1.0.2 largeVis_0.2.1 Matrix_1.2-10

loaded via a namespace (and not attached): [1] Rcpp_0.12.11 lattice_0.20-35 crayon_1.3.2 grid_3.4.1 [5] plyr_1.8.4 R6_2.2.2 gtable_0.2.0 magrittr_1.5 [9] scales_0.4.1 ggplot2_2.2.1 rlang_0.1.1 lazyeval_0.2.0 [13] munsell_0.4.3 compiler_3.4.1 colorspace_1.3-2 tibble_1.3.3

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

elbamos commented 7 years ago

And what compiler are you using?

I mean, I have oodles of windows and Linux users, and I test each release on both platforms. The function that's failing is some very generic C++, in files that haven't changed in a year. So, either there's something very funky in your setup, or something has recently changed in a component that largevis relies on to compile.

On Jul 18, 2017, at 10:38 AM, Amos Elberg amos.elberg@me.com wrote:

Do you see the same thing with 3.4?

On Jul 18, 2017, at 4:44 AM, meowcat notifications@github.com wrote:

Seems not to be Windows related. I installed the package from CRAN on my Linux VM and get the same error.

R version 3.4.1 (2017-06-30) -- "Single Candle" Copyright (C) 2017 The R Foundation for Statistical Computing Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details.

Natural language support but running in an English locale

R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R.

library(largeVis) Loading required package: Matrix library(testthat) expect_cpp_tests_pass("largeVis") Error: C++ unit tests:


is a Catch v1.2.1 host application.
Run with -? for options

alias tests | testcfunctions.cpp alias int succeeds

testcfunctions.cpp:9 ...............................................................................

testcfunctions.cpp:17: FAILED: CATCH_CHECK( testAlias() == 71 ) with expansion: 83 == 71

testcfunctions.cpp:18: FAILED: CATCH_CHECK( testAlias() == 74 ) with expansion: 97 == 74

testcfunctions.cpp:19: FAILED: CATCH_CHECK( testAlias() == 70 ) with expansion: 68 == 70

testcfunctions.cpp:20: FAILED: CATC

sessionInfo() R version 3.4.1 (2017-06-30) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 17.04

Matrix products: default BLAS: /usr/lib/libblas/libblas.so.3.7.0 LAPACK: /usr/lib/lapack/liblapack.so.3.7.0

locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 [7] LC_PAPER=en_US.UTF-8 LC_NAME=C [9] LC_ADDRESS=C LC_TELEPHONE=C [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] testthat_1.0.2 largeVis_0.2.1 Matrix_1.2-10

loaded via a namespace (and not attached): [1] Rcpp_0.12.11 lattice_0.20-35 crayon_1.3.2 grid_3.4.1 [5] plyr_1.8.4 R6_2.2.2 gtable_0.2.0 magrittr_1.5 [9] scales_0.4.1 ggplot2_2.2.1 rlang_0.1.1 lazyeval_0.2.0 [13] munsell_0.4.3 compiler_3.4.1 colorspace_1.3-2 tibble_1.3.3

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

meowcat commented 7 years ago

I have no quick way to put R3.4.0 on Linux, but I have the error in

So this is absolutely vanilla setups (except for the last one) which never cause problems otherwise.

This is also on two different machines plus one VM.

I also have no idea what makes these tests fail, but I also have no understanding of why you expect the output (71, 74, 70, 90) so I cannot really help you dig deeper...

Does it matter at all that I am running the tests from R using devtools and/or directly testthat? I originally caught this when using devtools::check() on the package.

Let me know if I can help more.

context("C++")
test_that("Catch unit tests pass", {
        skip_on_cran()
        skip_on_travis()
        skip_on_appveyor()
    expect_cpp_tests_pass("largeVis")
})
elbamos commented 7 years ago

The reason I expect the output Is that the script sets the random generator seed before it runs those tests. What compilers are you using? Can you try using llvm (it may link a different version of the stdlib.)

On Jul 19, 2017, at 12:38 AM, meowcat notifications@github.com wrote:

I have no quick way to put R3.4.0 on Linux, but I have the error in

R 3.4.1 on Linux (Ubuntu 17.04 repository R with the current gcc etc) R 3.4.0 on Windows with gcc 4.9.3 (current Rtools) R 3.3 devel on Windows with the CRAN largeVis build from a year ago (sic! Not built by myself) So this is absolutely vanilla setups (except for the last one) which never cause problems otherwise.

This is also on two different machines plus one VM.

I also have no idea what makes these tests fail, but I also have no understanding of why you expect the output (71, 74, 70, 90) so I cannot really help you dig deeper...

Does it matter at all that I am running the tests from R using devtools and/or directly testthat? I originally caught this when using devtools::check() on the package.

Let me know if I can help more.

context("C++") test_that("Catch unit tests pass", { skip_on_cran() skip_on_travis() skip_on_appveyor() expect_cpp_tests_pass("largeVis") }) — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

meowcat commented 7 years ago

I don't know how to set up R on Windows or Linux with llvm as a compiler; that would probably take me a day of work. I just tried setting up largeVis on a Windows machine with an old toolchain (R 3.2 with gcc 4.6.3) and it is incompatible (which I guess is intended):

In file included from checkfunctions.cpp:1:0:
largeVis.h:7:2: error: #error largeVis is incompatible with gcc < 4.9. Upgrade gcc or use llvm.
meowcat commented 7 years ago

Again, my compilers are: gcc 4.9.3 from Rtools on Windows (https://cran.r-project.org/bin/windows/Rtools/) gcc 6.3.0 on Ubuntu 17.04

elbamos commented 7 years ago

Hmmm.... Is it possible for you to send me the complete output of a session where you start from a new, empty R session without largeVis installed, compile the package from current Master and get this result?

Thank you for reporting, its really very odd.

meowcat commented 7 years ago

On Windows or on Linux?

elbamos commented 7 years ago

Let's try Linux.

On Jul 25, 2017, at 2:57 AM, meowcat notifications@github.com wrote:

On Windows or on Linux?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

dchiu911 commented 7 years ago

Hello, I recently tried to use largeVis in my own package, but the Travis build failed at the same place. Here is a complete session of the build, hope it helps https://travis-ci.org/AlineTalhouk/diceR/jobs/273500644

elbamos commented 7 years ago

It doesn't appear that your build script attempted to load largeVis, but just declared it in imports. Seems to be an issue in how you've configured Travis.

On Sep 11, 2017, at 9:56 AM, Derek Chiu notifications@github.com wrote:

Hello, I recently tried to use largeVis in my own package, but the Travis build failed at the same place. Here is a complete session of the build, hope it helps https://travis-ci.org/AlineTalhouk/diceR/jobs/273500644

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

dchiu911 commented 7 years ago

Do you have any insights on how I may solve this problem? I tried adding

before_install:
 - Rscript -e 'install.packages("largeVis")'

to no avail. Thanks!

I'll try adding

sudo apt-get -qq install g++-4.9

first.

elbamos commented 7 years ago

Look in the guidance for the Travis R system. You can look in the Travis yaml in this repo for an example but I'm sure it's far more complex than you need.

On Sep 11, 2017, at 11:32 AM, Derek Chiu notifications@github.com wrote:

Do you have any insights on how I may solve this problem? I tried adding

before_install:

  • Rscript -e 'install.packages("largeVis")' to no avail. Thanks!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

dchiu911 commented 7 years ago

After some tedious explorations, I've pinpointed the important parts of the .travis.yml that are needed for largeVis incorporation in another package for anyone else interested:

sudo: required
dist: trusty

before_install: |1
     mkdir ~/.R
     cat <<EOF > ~/.R/Makevars
     CC=gcc-6
     CXX=g++-6
     CXX1X=g++-6
     CXX11=g++-6

r_packages:
  - RcppArmadillo
  - RcppProgress

addons:
  apt:
    sources:
      - ubuntu-toolchain-r-test
    packages:
      - gcc-6
      - g++-6
      - gfortran-6
elbamos commented 6 years ago

Thanks!