search

diego-urgell / BinSeg

GSOC 2021. R package that performs changepoint analysis using the Binary Segmentation algorithm. Supports several statistical distributions. The model is computed in C++ and then interfaced with R via the Rcpp package.
https://summerofcode.withgoogle.com/projects/5915303348797440
MIT License
4 stars 0 forks source link

Setting up Github Actions #11

Closed diego-urgell closed 3 years ago

diego-urgell commented 3 years ago

Try to set up Github Actions instead of Travis CI

diego-urgell commented 3 years ago

Note: It is necessary to manually modify the .Rd files after using devtools::document() in order to include the alias with only the function name in addition to name-class. Only in this way the MacOS tests are passed

diego-urgell commented 3 years ago

Hi @rkillick and @tdhock! I'm afraid there is a huge problem with the package. I am unable to pass the `R CMD check´ for any system other than MacOS (which is the one I used for the development).

The tests that use old R versions don't pass because of the C++ compiler. However, the tests with the release version of R should pass for every OS.

On Windows, the problem occurs when loading the compiled package (windows-latest-release): 

Error: Error: package or namespace load failed for 'BinSeg' in inDL(x, as.logical(local), as.logical(now), ...):
137
 unable to load shared object 'D:/a/BinSeg/BinSeg/check/BinSeg.Rcheck/00LOCK-BinSeg/00new/BinSeg/libs/x64/BinSeg.dll':
138
  LoadLibrary failure:  A dynamic link library (DLL) initialization routine failed.

On Ubuntu, he following problem occurs (ubuntu-18.04 (release)): 

** testing if installed package can be loaded from temporary location
80

81
 *** caught segfault ***
82
address 0x8, cause 'memory not mapped'
83

84
Traceback:
85
 1: dyn.load(file, DLLpath = DLLpath, ...)
86
 2: library.dynam(lib, package, package.lib)
87
 3: loadNamespace(package, lib.loc)
88
 4: doTryCatch(return(expr), name, parentenv, handler)
89
 5: tryCatchOne(expr, names, parentenv, handlers[[1L]])
90
 6: tryCatchList(expr, classes, parentenv, handlers)
91
 7: tryCatch({    attr(package, "LibPath") <- which.lib.loc    ns <- loadNamespace(package, lib.loc)    env <- attachNamespace(ns, pos = pos, deps, exclude, include.only)}, error = function(e) {    P <- if (!is.null(cc <- conditionCall(e)))         paste(" in", deparse(cc)[1L])    else ""    msg <- gettextf("package or namespace load failed for %s%s:\n %s",         sQuote(package), P, conditionMessage(e))    if (logical.return)         message(paste("Error:", msg), domain = NA)    else stop(msg, call. = FALSE, domain = NA)})
92
 8: library(pkg_name, lib.loc = lib, character.only = TRUE, logical.return = TRUE)
93
 9: withCallingHandlers(expr, packageStartupMessage = function(c) tryInvokeRestart("muffleMessage"))
94
10: suppressPackageStartupMessages(library(pkg_name, lib.loc = lib,     character.only = TRUE, logical.return = TRUE))
95
11: doTryCatch(return(expr), name, parentenv, handler)
96
12: tryCatchOne(expr, names, parentenv, handlers[[1L]])
97
13: tryCatchList(expr, classes, parentenv, handlers)
98
14: tryCatch(expr, error = function(e) {    call <- conditionCall(e)    if (!is.null(call)) {        if (identical(call[[1L]], quote(doTryCatch)))             call <- sys.call(-4L)        dcall <- deparse(call)[1L]        prefix <- paste("Error in", dcall, ": ")        LONG <- 75L        sm <- strsplit(conditionMessage(e), "\n")[[1L]]        w <- 14L + nchar(dcall, type = "w") + nchar(sm[1L], type = "w")        if (is.na(w))             w <- 14L + nchar(dcall, type = "b") + nchar(sm[1L],                 type = "b")        if (w > LONG)             prefix <- paste0(prefix, "\n  ")    }    else prefix <- "Error : "    msg <- paste0(prefix, conditionMessage(e), "\n")    .Internal(seterrmessage(msg[1L]))    if (!silent && isTRUE(getOption("show.error.messages"))) {        cat(msg, file = outFile)        .Internal(printDeferredWarnings())    }    invisible(structure(msg, class = "try-error", condition = e))})
99
15: try(suppressPackageStartupMessages(library(pkg_name, lib.loc = lib,     character.only = TRUE, logical.return = TRUE)))
100
16: tools:::.test_load_package("BinSeg", "/home/runner/work/BinSeg/BinSeg/check/BinSeg.Rcheck/00LOCK-BinSeg/00new")
101
An irrecoverable exception occurred. R is aborting now ...
102
Segmentation fault (core dumped)
103
ERROR: loading failed
104
* removing ‘/home/runner/work/BinSeg/BinSeg/check/BinSeg.Rcheck/BinSeg’
105

You can see the complete logs on the Github Actions checks on this branch. Do you have any suggestions for any of the OS? I am worried that it only runs on Mac.

tdhock commented 3 years ago

i'm not sure what the problem is. looks like the C++ code can not be loaded for some reason. you can try to debug by using rocker or r-hub docker images. https://r-hub.github.io/rhub/articles/local-debugging.html

rkillick commented 3 years ago

It is really hard when things work on your OS but not on others. I agree with Toby that using the docker images might help you to debug as you can repeat your workflow directly on the OS's to try to identify the issue. I would start with ubuntu as that is closer to the Mac setup than Windows is.

diego-urgell commented 3 years ago

Great, thanks @tdhock and @rkillick! I will debug with the docker image and I'll let you know how it goes 😥

diego-urgell commented 3 years ago

@tdhock @rkillick I performed the CRAN tests using r-hub, and I found something very interesting. The problem does not seem to be the Operating System, but rather the compiler. Here you can see that both Fedora and Debian checks pass, using clang as the compiler. MacOS also uses clang. However, all the other tests were failed because they used gcc.

Screen Shot 2021-08-11 at 13 02 10

Worst case scenario, we used some implementation specific compiler hacks on the C++ code that only work for clang. However, I will continue to debug this.

I have not been able to use the docker containers, because there is a problem when it is loading since the BioConductor package is tried to be installed but it fails. Then, the container initialization also fails.

I will setup an Azure virtual machine with Ubuntu, then clone my repository, and build and check the package there. This is no ideal but it is the most viable option right now.

tdhock commented 3 years ago

why doesn't it work with gcc? it needs to if we want to put it on CRAN.

tdhock commented 3 years ago

it is because the gcc version is too old to support the new C++ features you are using? if so that is an issue you should discuss on R-pkg-devel https://stat.ethz.ch/mailman/listinfo/r-package-devel

rkillick commented 3 years ago

I recall some recent discussion on the package developers list so might be worth a search in the history here: https://www.mail-archive.com/search?q=gcc&l=r-package-devel%40r-project.org

tdhock commented 3 years ago

I recall some recent discussion on the package developers list so might be worth a search in the history here: https://www.mail-archive.com/search?q=gcc&l=r-package-devel%40r-project.org

Right. before posting a new message on a list like this, you should definitely do your homework by searching the list for relevant keywords, in this case "gcc Makevars CXX_STD CXX17" or something like that. By doing this search, you may find the solution to your problem before you have to post. Likewise, by creating a minimal reproducible example that distills your problem into something simple enough for others to immediately reproduce on another computer, you are likely to get to the root of your problem, and perhaps find a solution. https://stackoverflow.com/help/minimal-reproducible-example If after doing all that you still haven't solved your issue yourself, please post.

tdhock commented 3 years ago

I confirm seeing the same error when trying to install on Ubuntu, with gcc-7.5.0 and R-4.1.0

(base) tdhock@maude-MacBookPro:~/R/BinSeg(main)$ R CMD INSTALL .
Loading required package: grDevices
* installing to library ‘/home/tdhock/lib/R/library’
* installing *source* package ‘BinSeg’ ...
** using staged installation
Warning in .write_description(db, file.path(outDir, "DESCRIPTION")) :
  Unknown encoding with non-ASCII data: converting to ASCII
** libs
g++ -std=gnu++17 -I"/home/tdhock/lib/R/include" -DNDEBUG  -I'/home/tdhock/lib/R/library/Rcpp/include' -I/usr/local/include   -fpic  -g -O2  -c AlgorithmInterface.cpp -o AlgorithmInterface.o
g++ -std=gnu++17 -I"/home/tdhock/lib/R/include" -DNDEBUG  -I'/home/tdhock/lib/R/library/Rcpp/include' -I/usr/local/include   -fpic  -g -O2  -c Algorithms.cpp -o Algorithms.o
g++ -std=gnu++17 -I"/home/tdhock/lib/R/include" -DNDEBUG  -I'/home/tdhock/lib/R/library/Rcpp/include' -I/usr/local/include   -fpic  -g -O2  -c Cumsum.cpp -o Cumsum.o
g++ -std=gnu++17 -I"/home/tdhock/lib/R/include" -DNDEBUG  -I'/home/tdhock/lib/R/library/Rcpp/include' -I/usr/local/include   -fpic  -g -O2  -c DistributionInterface.cpp -o DistributionInterface.o
g++ -std=gnu++17 -I"/home/tdhock/lib/R/include" -DNDEBUG  -I'/home/tdhock/lib/R/library/Rcpp/include' -I/usr/local/include   -fpic  -g -O2  -c Distributions.cpp -o Distributions.o
g++ -std=gnu++17 -I"/home/tdhock/lib/R/include" -DNDEBUG  -I'/home/tdhock/lib/R/library/Rcpp/include' -I/usr/local/include   -fpic  -g -O2  -c GenericFactory.cpp -o GenericFactory.o
g++ -std=gnu++17 -I"/home/tdhock/lib/R/include" -DNDEBUG  -I'/home/tdhock/lib/R/library/Rcpp/include' -I/usr/local/include   -fpic  -g -O2  -c RcppExports.cpp -o RcppExports.o
g++ -std=gnu++17 -I"/home/tdhock/lib/R/include" -DNDEBUG  -I'/home/tdhock/lib/R/library/Rcpp/include' -I/usr/local/include   -fpic  -g -O2  -c RcppInterface.cpp -o RcppInterface.o
g++ -std=gnu++17 -I"/home/tdhock/lib/R/include" -DNDEBUG  -I'/home/tdhock/lib/R/library/Rcpp/include' -I/usr/local/include   -fpic  -g -O2  -c Segment.cpp -o Segment.o
g++ -std=gnu++17 -shared -L/home/tdhock/lib/R/lib -L/usr/local/lib -o BinSeg.so AlgorithmInterface.o Algorithms.o Cumsum.o DistributionInterface.o Distributions.o GenericFactory.o RcppExports.o RcppInterface.o Segment.o -L/home/tdhock/lib/R/lib -lR
installing to /home/tdhock/lib/R/library/00LOCK-BinSeg/00new/BinSeg/libs
** R
** byte-compile and prepare package for lazy loading
Loading required package: grDevices
Creating a new generic function for ‘dist’ in package ‘BinSeg’
Creating a new generic function for ‘plot’ in package ‘BinSeg’
Creating a new generic function for ‘show’ in package ‘BinSeg’
** help
*** installing help indices
** building package indices
Loading required package: grDevices
** testing if installed package can be loaded from temporary location
Loading required package: grDevices

 *** caught segfault ***
address 0x8, cause 'memory not mapped'

Traceback:
 1: dyn.load(file, DLLpath = DLLpath, ...)
 2: library.dynam(lib, package, package.lib)
 3: loadNamespace(package, lib.loc)
 4: doTryCatch(return(expr), name, parentenv, handler)
 5: tryCatchOne(expr, names, parentenv, handlers[[1L]])
 6: tryCatchList(expr, classes, parentenv, handlers)
 7: tryCatch({    attr(package, "LibPath") <- which.lib.loc    ns <- loadNamespace(package, lib.loc)    env <- attachNamespace(ns, pos = pos, deps, exclude, include.only)}, error = function(e) {    P <- if (!is.null(cc <- conditionCall(e)))         paste(" in", deparse(cc)[1L])    else ""    msg <- gettextf("package or namespace load failed for %s%s:\n %s",         sQuote(package), P, conditionMessage(e))    if (logical.return)         message(paste("Error:", msg), domain = NA)    else stop(msg, call. = FALSE, domain = NA)})
 8: library(pkg_name, lib.loc = lib, character.only = TRUE, logical.return = TRUE)
 9: withCallingHandlers(expr, packageStartupMessage = function(c) tryInvokeRestart("muffleMessage"))
10: suppressPackageStartupMessages(library(pkg_name, lib.loc = lib,     character.only = TRUE, logical.return = TRUE))
11: doTryCatch(return(expr), name, parentenv, handler)
12: tryCatchOne(expr, names, parentenv, handlers[[1L]])
13: tryCatchList(expr, classes, parentenv, handlers)
14: tryCatch(expr, error = function(e) {    call <- conditionCall(e)    if (!is.null(call)) {        if (identical(call[[1L]], quote(doTryCatch)))             call <- sys.call(-4L)        dcall <- deparse(call)[1L]        prefix <- paste("Error in", dcall, ": ")        LONG <- 75L        sm <- strsplit(conditionMessage(e), "\n")[[1L]]        w <- 14L + nchar(dcall, type = "w") + nchar(sm[1L], type = "w")        if (is.na(w))             w <- 14L + nchar(dcall, type = "b") + nchar(sm[1L],                 type = "b")        if (w > LONG)             prefix <- paste0(prefix, "\n  ")    }    else prefix <- "Error : "    msg <- paste0(prefix, conditionMessage(e), "\n")    .Internal(seterrmessage(msg[1L]))    if (!silent && isTRUE(getOption("show.error.messages"))) {        cat(msg, file = outFile)        .Internal(printDeferredWarnings())    }    invisible(structure(msg, class = "try-error", condition = e))})
15: try(suppressPackageStartupMessages(library(pkg_name, lib.loc = lib,     character.only = TRUE, logical.return = TRUE)))
16: tools:::.test_load_package("BinSeg", "/home/tdhock/lib/R/library/00LOCK-BinSeg/00new")
An irrecoverable exception occurred. R is aborting now ...
Segmentation fault (core dumped)
ERROR: loading failed
* removing ‘/home/tdhock/lib/R/library/BinSeg’
(base) tdhock@maude-MacBookPro:~/R/BinSeg(main)$ gcc --version
gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

(base) tdhock@maude-MacBookPro:~/R/BinSeg(main)$
tdhock commented 3 years ago

same with R-4.1.1 and g++-8.4.0 

(base) tdhock@maude-MacBookPro:~/R/BinSeg(main*)$ rm -f src/*.o && R CMD INSTALL .
Loading required package: grDevices
* installing to library ‘/home/tdhock/lib/R/library’
* installing *source* package ‘BinSeg’ ...
** using staged installation
Warning in .write_description(db, file.path(outDir, "DESCRIPTION")) :
  Unknown encoding with non-ASCII data: converting to ASCII
** libs
g++-8 -std=gnu++17 -I"/home/tdhock/lib/R/include" -DNDEBUG  -I'/home/tdhock/lib/R/library/Rcpp/include' -I/usr/local/include   -fpic  -g -O2  -c AlgorithmInterface.cpp -o AlgorithmInterface.o
g++-8 -std=gnu++17 -I"/home/tdhock/lib/R/include" -DNDEBUG  -I'/home/tdhock/lib/R/library/Rcpp/include' -I/usr/local/include   -fpic  -g -O2  -c Algorithms.cpp -o Algorithms.o
g++-8 -std=gnu++17 -I"/home/tdhock/lib/R/include" -DNDEBUG  -I'/home/tdhock/lib/R/library/Rcpp/include' -I/usr/local/include   -fpic  -g -O2  -c Cumsum.cpp -o Cumsum.o
g++-8 -std=gnu++17 -I"/home/tdhock/lib/R/include" -DNDEBUG  -I'/home/tdhock/lib/R/library/Rcpp/include' -I/usr/local/include   -fpic  -g -O2  -c DistributionInterface.cpp -o DistributionInterface.o
g++-8 -std=gnu++17 -I"/home/tdhock/lib/R/include" -DNDEBUG  -I'/home/tdhock/lib/R/library/Rcpp/include' -I/usr/local/include   -fpic  -g -O2  -c Distributions.cpp -o Distributions.o
g++-8 -std=gnu++17 -I"/home/tdhock/lib/R/include" -DNDEBUG  -I'/home/tdhock/lib/R/library/Rcpp/include' -I/usr/local/include   -fpic  -g -O2  -c GenericFactory.cpp -o GenericFactory.o
g++-8 -std=gnu++17 -I"/home/tdhock/lib/R/include" -DNDEBUG  -I'/home/tdhock/lib/R/library/Rcpp/include' -I/usr/local/include   -fpic  -g -O2  -c RcppExports.cpp -o RcppExports.o
g++-8 -std=gnu++17 -I"/home/tdhock/lib/R/include" -DNDEBUG  -I'/home/tdhock/lib/R/library/Rcpp/include' -I/usr/local/include   -fpic  -g -O2  -c RcppInterface.cpp -o RcppInterface.o
g++-8 -std=gnu++17 -I"/home/tdhock/lib/R/include" -DNDEBUG  -I'/home/tdhock/lib/R/library/Rcpp/include' -I/usr/local/include   -fpic  -g -O2  -c Segment.cpp -o Segment.o
g++-8 -std=gnu++17 -shared -L/home/tdhock/lib/R/lib -L/usr/local/lib -o BinSeg.so AlgorithmInterface.o Algorithms.o Cumsum.o DistributionInterface.o Distributions.o GenericFactory.o RcppExports.o RcppInterface.o Segment.o -L/home/tdhock/lib/R/lib -lR
installing to /home/tdhock/lib/R/library/00LOCK-BinSeg/00new/BinSeg/libs
** R
** byte-compile and prepare package for lazy loading
Loading required package: grDevices
Creating a new generic function for ‘dist’ in package ‘BinSeg’
Creating a new generic function for ‘plot’ in package ‘BinSeg’
Creating a new generic function for ‘show’ in package ‘BinSeg’
** help
*** installing help indices
** building package indices
Loading required package: grDevices
** testing if installed package can be loaded from temporary location
Loading required package: grDevices

 *** caught segfault ***
address 0x8, cause 'memory not mapped'

Traceback:
 1: dyn.load(file, DLLpath = DLLpath, ...)
 2: library.dynam(lib, package, package.lib)
 3: loadNamespace(package, lib.loc)
 4: doTryCatch(return(expr), name, parentenv, handler)
 5: tryCatchOne(expr, names, parentenv, handlers[[1L]])
 6: tryCatchList(expr, classes, parentenv, handlers)
 7: tryCatch({    attr(package, "LibPath") <- which.lib.loc    ns <- loadNamespace(package, lib.loc)    env <- attachNamespace(ns, pos = pos, deps, exclude, include.only)}, error = function(e) {    P <- if (!is.null(cc <- conditionCall(e)))         paste(" in", deparse(cc)[1L])    else ""    msg <- gettextf("package or namespace load failed for %s%s:\n %s",         sQuote(package), P, conditionMessage(e))    if (logical.return && !quietly)         message(paste("Error:", msg), domain = NA)    else stop(msg, call. = FALSE, domain = NA)})
 8: library(pkg_name, lib.loc = lib, character.only = TRUE, logical.return = TRUE)
 9: withCallingHandlers(expr, packageStartupMessage = function(c) tryInvokeRestart("muffleMessage"))
10: suppressPackageStartupMessages(library(pkg_name, lib.loc = lib,     character.only = TRUE, logical.return = TRUE))
11: doTryCatch(return(expr), name, parentenv, handler)
12: tryCatchOne(expr, names, parentenv, handlers[[1L]])
13: tryCatchList(expr, classes, parentenv, handlers)
14: tryCatch(expr, error = function(e) {    call <- conditionCall(e)    if (!is.null(call)) {        if (identical(call[[1L]], quote(doTryCatch)))             call <- sys.call(-4L)        dcall <- deparse(call)[1L]        prefix <- paste("Error in", dcall, ": ")        LONG <- 75L        sm <- strsplit(conditionMessage(e), "\n")[[1L]]        w <- 14L + nchar(dcall, type = "w") + nchar(sm[1L], type = "w")        if (is.na(w))             w <- 14L + nchar(dcall, type = "b") + nchar(sm[1L],                 type = "b")        if (w > LONG)             prefix <- paste0(prefix, "\n  ")    }    else prefix <- "Error : "    msg <- paste0(prefix, conditionMessage(e), "\n")    .Internal(seterrmessage(msg[1L]))    if (!silent && isTRUE(getOption("show.error.messages"))) {        cat(msg, file = outFile)        .Internal(printDeferredWarnings())    }    invisible(structure(msg, class = "try-error", condition = e))})
15: try(suppressPackageStartupMessages(library(pkg_name, lib.loc = lib,     character.only = TRUE, logical.return = TRUE)))
16: tools:::.test_load_package("BinSeg", "/home/tdhock/lib/R/library/00LOCK-BinSeg/00new")
An irrecoverable exception occurred. R is aborting now ...
Segmentation fault (core dumped)
ERROR: loading failed
* removing ‘/home/tdhock/lib/R/library/BinSeg’
(base) tdhock@maude-MacBookPro:~/R/BinSeg(main*)$ g++-8 --version
g++-8 (Ubuntu 8.4.0-1ubuntu1~18.04) 8.4.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

(base) tdhock@maude-MacBookPro:~/R/BinSeg(main*)$ R --version
R version 4.1.1 (2021-08-10) -- "Kick Things"
Copyright (C) 2021 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under the terms of the
GNU General Public License versions 2 or 3.
For more information about these matters see
https://www.gnu.org/licenses/.

(base) tdhock@maude-MacBookPro:~/R/BinSeg(main*)$
diego-urgell commented 3 years ago

Yesterday I did remove the C++ 17 features on branch #13, so it is now C++11. The packages is successfully compiled on every platform, and the checks pass on all except one. (There is a strange bug in Windows with an old R version, I am looking into it).

This solution is definitely not ideal, since all the static variables must be initialized in the RcppInterface files it looks messy. However, it works. I will look into the R package devel list and the archives that you provided, and I will look for a better solution. If this is not possible right now, at least we know that there is a way to make the package run.

Thanks for trying to load it in your machine :)

tdhock commented 3 years ago

another thing you should try is https://git-scm.com/docs/git-bisect which runs R CMD INSTALL on your package for each commit in the history until it finds the commit after which it starts failing.

diego-urgell commented 3 years ago

Thanks @tdhock! Given the time left for GSOC, @rkillick and I agreed that the C++17 dependencies will be removed from the package. It already compiles when using only C++11. Its probably a matter of time before this is fixed with R on Ubuntu and Windows. Later on I will investigate more on this issue, but for now it is a better option to merge #13.

I will merge this branch first since it is the first point in the repository's history where the package passes the checks sucesfully using the clang compiler. I will create a tag so that later on it is easier to identify how to C++ code structure should look like.