Closed Hoohm closed 7 years ago
Hi, thanks for reporting this! Could you please provide an output of sessionInfo()
after running sc3 and getting this error?
Small update in the meantime. I tried it on my server and it worked. So here are both sessionsInfos, the first is my local computer, the second is the server. One main difference is the R version 3.3 VS 3.2
Local computer with error
R version 3.3.2 (2016-10-31)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Arch Linux
locale:
[1] LC_CTYPE=en_US.UTF-8
[2] LC_NUMERIC=C
[3] LC_TIME=fr_CH.UTF-8
[4] LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=fr_CH.UTF-8
[6] LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=fr_CH.UTF-8
[8] LC_NAME=C
[9] LC_ADDRESS=C
[10] LC_TELEPHONE=C
[11] LC_MEASUREMENT=fr_CH.UTF-8
[12] LC_IDENTIFICATION=C
attached base packages:
[1] parallel stats graphics grDevices utils
[6] datasets methods base
other attached packages:
[1] SC3_1.3.11 scater_1.3.35
[3] ggplot2_2.2.0 Biobase_2.34.0
[5] BiocGenerics_0.20.0
loaded via a namespace (and not attached):
[1] viridis_0.3.4 edgeR_3.16.5
[3] foreach_1.4.3 gtools_3.5.0
[5] shiny_0.14.2 assertthat_0.1
[7] stats4_3.3.2 doRNG_1.6
[9] vipor_0.4.4 robustbase_0.92-7
[11] RSQLite_1.1-1 lattice_0.20-34
[13] limma_3.30.7 digest_0.6.10
[15] RColorBrewer_1.1-2 colorspace_1.3-2
[17] htmltools_0.3.5 httpuv_1.3.3
[19] Matrix_1.2-7.1 plyr_1.8.4
[21] pcaPP_1.9-61 WriteXLS_4.0.0
[23] XML_3.98-1.5 pheatmap_1.0.8
[25] biomaRt_2.30.0 zlibbioc_1.20.0
[27] mvtnorm_1.0-5 xtable_1.8-2
[29] scales_0.4.1 gdata_2.17.0
[31] tibble_1.2 pkgmaker_0.22
[33] IRanges_2.8.1 ROCR_1.0-7
[35] lazyeval_0.2.0 magrittr_1.5
[37] mime_0.5 memoise_1.0.0
[39] doParallel_1.0.10 gplots_3.0.1
[41] class_7.3-14 beeswarm_0.2.3
[43] shinydashboard_0.5.3 tools_3.3.2
[45] registry_0.3 data.table_1.10.0
[47] matrixStats_0.51.0 stringr_1.1.0
[49] S4Vectors_0.12.1 munsell_0.4.3
[51] locfit_1.5-9.1 cluster_2.0.5
[53] rngtools_1.2.4 AnnotationDbi_1.36.0
[55] compiler_3.3.2 e1071_1.6-7
[57] caTools_1.17.1 rhdf5_2.18.0
[59] grid_3.3.2 RCurl_1.95-4.8
[61] iterators_1.0.8 tximport_1.2.0
[63] rjson_0.2.15 bitops_1.0-6
[65] gtable_0.2.0 codetools_0.2-15
[67] DBI_0.5-1 rrcov_1.4-3
[69] reshape2_1.4.2 R6_2.2.0
[71] gridExtra_2.2.1 dplyr_0.5.0
[73] KernSmooth_2.23-15 stringi_1.1.2
[75] ggbeeswarm_0.5.3 Rcpp_0.12.8
[77] DEoptimR_1.0-8
Server without error
R version 3.3.1 (2016-06-21)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 14.04.5 LTS
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=de_DE.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=de_DE.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=de_DE.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=de_DE.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] parallel stats graphics grDevices utils datasets methods
[8] base
other attached packages:
[1] SC3_1.3.11 scater_1.3.35 ggplot2_2.2.0
[4] Biobase_2.32.0 BiocGenerics_0.18.0
loaded via a namespace (and not attached):
[1] viridis_0.3.4 edgeR_3.14.0
[3] foreach_1.4.3 gtools_3.5.0
[5] shiny_0.14.2 assertthat_0.1
[7] stats4_3.3.1 doRNG_1.6
[9] vipor_0.4.4 robustbase_0.92-7
[11] RSQLite_1.1-1 lattice_0.20-34
[13] limma_3.28.21 digest_0.6.10
[15] RColorBrewer_1.1-2 GenomicRanges_1.24.3
[17] XVector_0.12.1 colorspace_1.3-2
[19] htmltools_0.3.5 httpuv_1.3.3
[21] Matrix_1.2-7.1 plyr_1.8.4
[23] pcaPP_1.9-61 WriteXLS_4.0.0
[25] XML_3.98-1.5 pheatmap_1.0.8
[27] biomaRt_2.28.0 zlibbioc_1.18.0
[29] mvtnorm_1.0-5 xtable_1.8-2
[31] scales_0.4.1 gdata_2.17.0
[33] tibble_1.2 pkgmaker_0.22
[35] IRanges_2.6.1 SummarizedExperiment_1.2.3
[37] ROCR_1.0-7 lazyeval_0.2.0
[39] magrittr_1.5 mime_0.5
[41] memoise_1.0.0 doParallel_1.0.10
[43] gplots_3.0.1 class_7.3-14
[45] beeswarm_0.2.3 shinydashboard_0.5.3
[47] tools_3.3.1 registry_0.3
[49] data.table_1.10.0 matrixStats_0.51.0
[51] stringr_1.1.0 S4Vectors_0.10.3
[53] munsell_0.4.3 cluster_2.0.5
[55] rngtools_1.2.4 AnnotationDbi_1.34.4
[57] compiler_3.3.1 e1071_1.6-7
[59] GenomeInfoDb_1.8.7 caTools_1.17.1
[61] rhdf5_2.16.0 grid_3.3.1
[63] RCurl_1.95-4.8 iterators_1.0.8
[65] tximport_1.0.3 rjson_0.2.15
[67] labeling_0.3 bitops_1.0-6
[69] gtable_0.2.0 codetools_0.2-15
[71] DBI_0.5-1 rrcov_1.4-3
[73] reshape2_1.4.2 R6_2.2.0
[75] gridExtra_2.2.1 dplyr_0.5.0
[77] KernSmooth_2.23-15 stringi_1.1.2
[79] ggbeeswarm_0.5.3 Rcpp_0.12.8
[81] DEoptimR_1.0-8
Hmm, that's a hard error to catch... Looks like some memory issues... R version should not be a problem here.
Did you run it on the example dataset? If not, could you please share your dataset with me? Could you please also provide a full sequence of commands that you run after starting an R session?
Cheers, Vlad
I used the example you provided and my dataset, same error.
here is the sequence of commands with my dataset:
library(scater)
library(SC3)
#Load the data
Mouse = read.table('gene_name_matrix.tsv')
phenoData_matrix = data.frame(Cell = colnames(Mouse), Species=rep('Mouse', ncol(Mouse)))
phenoData <- new("AnnotatedDataFrame", data=phenoData_matrix)
rownames(phenoData) = phenoData_matrix$Cell
mouse = newSCESet(countData = Mouse, phenoData = phenoData)
keep_feature <- rowSums(exprs(mouse) > 0) > 0
mouse <- mouse[keep_feature,]
mouse <- calculateQCMetrics(mouse)
mouse <- sc3(mouse, ks = 2:4)
Could you share your dataset with me?
On Wed, Dec 21, 2016 at 12:29 PM Patrick Roelli notifications@github.com wrote:
I used the example you provided and my dataset, same error.
here is the sequence of commands with my dataset:
library(scater) library(SC3)
Load the data
Mouse = read.table('gene_name_matrix.tsv') phenoData_matrix = data.frame(Cell = colnames(Mouse), Species=rep('Mouse', ncol(Mouse))) phenoData <- new("AnnotatedDataFrame", data=phenoData_matrix) rownames(phenoData) = phenoData_matrix$Cell mouse = newSCESet(countData = Mouse, phenoData = phenoData) keep_feature <- rowSums(exprs(mouse) > 0) > 0 mouse <- mouse[keep_feature,] mouse <- calculateQCMetrics(mouse) mouse <- sc3(mouse, ks = 2:4)
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/hemberg-lab/SC3/issues/33#issuecomment-268511456, or mute the thread https://github.com/notifications/unsubscribe-auth/ACO-zmxXlBJTlbRKlmgQlKjw2n1J3Ey2ks5rKRu4gaJpZM4LSTCs .
sent via email
Hi Patrick,
Unfortunately, I can't reproduce your error using your data. Everything works fine for me.
However, it looks like the C++ call to norm_laplacian()
function of sc3 throws an error. So could you please try running this code (after starting a new R session) and let me know if you still get the error?
SC3::norm_laplacian(matrix(runif(100), nrow = 10))
Cheers, Vlad
Hello,
just tried on both platforms, same error on the local computer, works on the server:
SC3::norm_laplacian(matrix(runif(100), nrow = 10))
*** caught segfault ***
address 0x7ffdc981e000, cause 'memory not mapped'
Maybe it has something to do with armadillo?
Yes, most probably you are right, since I mostly call armadillo functions in norm_laplacian()
. Could you please reinstall both Rccparmadillo and SC3 on your local computer and try again? And could you please also check whether you have different versions of gcc compilers on your local computer and the server? Maybe the problem is in the compiler...
I am getting the same error. Here is my session info:
> sessionInfo()
R version 3.3.2 (2016-10-31)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.10
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] parallel stats graphics grDevices utils datasets methods
[8] base
other attached packages:
[1] SC3_1.3.12 scater_1.2.0 ggplot2_2.2.1
[4] Biobase_2.34.0 BiocGenerics_0.20.0
loaded via a namespace (and not attached):
[1] viridis_0.3.4 edgeR_3.16.5 foreach_1.4.3
[4] gtools_3.5.0 shiny_0.14.2 assertthat_0.1
[7] stats4_3.3.2 doRNG_1.6 vipor_0.4.4
[10] robustbase_0.92-7 RSQLite_1.1-1 lattice_0.20-34
[13] limma_3.30.7 digest_0.6.11 RColorBrewer_1.1-2
[16] colorspace_1.3-2 htmltools_0.3.5 httpuv_1.3.3
[19] plyr_1.8.4 pcaPP_1.9-61 WriteXLS_4.0.0
[22] XML_3.98-1.5 pheatmap_1.0.8 biomaRt_2.30.0
[25] zlibbioc_1.20.0 mvtnorm_1.0-5 xtable_1.8-2
[28] scales_0.4.1 gdata_2.17.0 Rtsne_0.11
[31] tibble_1.2 pkgmaker_0.22 IRanges_2.8.1
[34] ROCR_1.0-7 lazyeval_0.2.0 magrittr_1.5
[37] mime_0.5 memoise_1.0.0 doParallel_1.0.10
[40] gplots_3.0.1 class_7.3-14 beeswarm_0.2.3
[43] shinydashboard_0.5.3 tools_3.3.2 registry_0.3
[46] data.table_1.10.0 matrixStats_0.51.0 stringr_1.1.0
[49] S4Vectors_0.12.1 munsell_0.4.3 locfit_1.5-9.1
[52] cluster_2.0.5 rngtools_1.2.4 AnnotationDbi_1.36.0
[55] compiler_3.3.2 e1071_1.6-7 caTools_1.17.1
[58] rhdf5_2.18.0 grid_3.3.2 RCurl_1.95-4.8
[61] iterators_1.0.8 tximport_1.2.0 rjson_0.2.15
[64] bitops_1.0-6 labeling_0.3 gtable_0.2.0
[67] codetools_0.2-15 DBI_0.5-1 rrcov_1.4-3
[70] reshape2_1.4.2 R6_2.2.0 gridExtra_2.2.1
[73] dplyr_0.5.0 KernSmooth_2.23-15 stringi_1.1.2
[76] ggbeeswarm_0.5.3 Rcpp_0.12.8 DEoptimR_1.0-8
Thanks for your message, Jake! Hmm, now I think it maybe the R version problem (3.3.2). Would you be able to test on a different version of R, say 3.3.1?
Another option is Ubuntu. In Patrick's message above it worked under Ubuntu 14.04. In your case it is Ubuntu 16.10.
I can't really understand what the current problem in norm_laplacian() can be, maybe you guys have some ideas:
arma::mat norm_laplacian(arma::mat A) {
A = exp(-A/A.max());
arma::rowvec D_row = pow(sum(A), -0.5);
A.each_row() %= D_row;
colvec D_col = conv_to< colvec >::from(D_row);
A.each_col() %= D_col;
arma::mat res = eye(A.n_cols, A.n_cols) - A;
return(res);
}
If I don't find problems in this C++ code, I suppose the most reasonable thing I could do is to rewrite norm_laplacian() using only R functions...
Interestingly, Bioconductor builds for SC3 are all ok and the vignettes are built ok - http://bioconductor.org/packages/3.4/bioc/vignettes/SC3/inst/doc/my-vignette.html
All Bioconductor builds are made on R 3.3.2: R version 3.3.2 (2016-10-31) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 16.04.1 LTS
So I really have no idea what the problem is... I have posted a question on Biocoductor devel mailing list. I am copying it here (will let you know if anybody answers):
R version-dependent segfault
My package (SC3 - http://bioconductor.org/packages/3.4/bioc/html/SC3.html) has a function that causes R version/platform-dependent seqfault. Here is the function (it's in C++ using RccpArmadillo):
arma::mat norm_laplacian(arma::mat A) {
A = exp(-A/A.max());
arma::rowvec D_row = pow(sum(A), -0.5);
A.each_row() %= D_row;
colvec D_col = conv_to< colvec >::from(D_row);
A.each_col() %= D_col;
arma::mat res = eye(A.n_cols, A.n_cols) - A;
return(res);
}
The test code that provides a segfault on some R versions/platforms:
SC3::norm_laplacian(matrix(runif(100), nrow = 10))
The segfault usually looks like this:
*** caught segfault ***
address 0x7ffdc981e000, cause 'memory not mapped'
(where address can be a different sequence)
So far by a collaborative effort (me and some users of the package) we figured out configurations that cause or do not cause a segfault:
R version 3.3.2 (2016-10-31) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Arch Linux
R version 3.3.2 (2016-10-31) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 16.10
R version 3.3.2 (2016-10-31) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 16.04.1 LTS
R version 3.3.1 (2016-06-21) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 14.04.5 LTS
R version 3.3.0 (2016-05-03) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu precise (12.04.5 LTS)
R Under development (unstable) (2016-10-20 r71540) Platform: x86_64-apple-darwin13.4.0 (64-bit) Running under: OS X Yosemite 10.10.5
More details on our discussion can be found here: https://github.com/hemberg-lab/SC3/issues/33
Has anybody had a similar issue? Do you have any suggestions on how to fix this, except rewriting the function in R? Or maybe there already exists a normalised Laplacian function written in C++?
Many thanks, Cheers, Vladimir
I resintalled Rccparmadillo and SC3 on the local comp, still not working. I even tried to "cheat" and hardcopied the compiled version of the server onto local libs, still same error.
2016-12-21 21:12 GMT+01:00 Vladimir Kiselev notifications@github.com:
Yes, most probably you are right, since I mostly call armadillo functions in norm_laplacian(). Could you please reinstall both Rccparmadillo and SC3 on your local computer and try again? And could you please also check whether you have different versions of gcc compilers on your local computer and the server? Maybe the problem is in the compiler...
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/hemberg-lab/SC3/issues/33#issuecomment-268626819, or mute the thread https://github.com/notifications/unsubscribe-auth/ABNXaMejUMnnHsNJ1sDbARNBQl_mp_37ks5rKYgagaJpZM4LSTCs .
Hi Patrick, thanks for the updates. Please see my posts above.
Here is a reply from Martin Morgan at Bioconductor. Could you please follow his steps, guys, and let me know about your output? Many thanks in advance!
The first line of attack is to simplify the problem as much as possible. I did this by writing a C++ file norm_laplacian.cpp
#include <RcppArmadillo.h>
using namespace arma;
// [[Rcpp::depends(RcppArmadillo)]]
// [[Rcpp::export]]
arma::mat norm_laplacian(arma::mat A) {
A = exp(-A/A.max());
arma::rowvec D_row = pow(sum(A), -0.5);
A.each_row() %= D_row;
colvec D_col = conv_to< colvec >::from(D_row);
A.each_col() %= D_col;
arma::mat res = eye(A.n_cols, A.n_cols) - A;
return(res);
}
and then in R, e.g., norm_laplacian.R
library(Rcpp)
sourceCpp("norm_laplacian.cpp", showOutput=TRUE)
xx <- norm_laplacian(matrix(runif(100), nrow = 10))
sessionInfo()
It would be helpful to use set.seed() to make the example more reproducible. One would hope that
R -f norm_laplacian.R
would produce a segfault. Unfortunately not for me. My next step was to run this code under valgrind to look for invalid memory access
R -d valgrind -f norm_laplacian.R
again hoping for a report of 'invalid write' or 'invalid read', but again no luck for me.
You could see if your collaborators are able to generate segfaults with this simpler code. If R -f norm_laplacian.R is sufficient, the next step would be to run it under a C-level debugger like gdb, with some hints at http://bioconductor.org/developers/how-to/c-debugging/
Here's my output; it's also useful to know information about the compiler, and to pay attention to the compiler options (especially optimization level -O0 for me)
$ g++ --version|head -n1
g++ (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
$ R --vanilla -f norm_laplacian.R
> library(Rcpp)
> sourceCpp("norm_laplacian.cpp", showOutput=TRUE)
/home/mtmorgan/bin/R-devel/bin/R CMD SHLIB -o 'sourceCpp_2.so'
'norm_laplacian.cpp'
g++ -I/home/mtmorgan/bin/R-devel/include -DNDEBUG -I/usr/local/include
-I"/home/mtmorgan/R/x86_64-pc-linux-gnu-library/3.4-Bioc-3.5/Rcpp/include"
-I"/home/mtmorgan/R/x86_64-pc-linux-gnu-library/3.4-Bioc-3.5/RcppArmadillo/include"
-I"/tmp" -fpic -g -O0 -c norm_laplacian.cpp -o norm_laplacian.o
g++ -shared -L/home/mtmorgan/bin/R-devel/lib -L/usr/local/lib -o
sourceCpp_2.so norm_laplacian.o -L/home/mtmorgan/bin/R-devel/lib
-lRlapack -L/home/mtmorgan/bin/R-devel/lib -lRblas -lgfortran -lm
-lquadmath -L/home/mtmorgan/bin/R-devel/lib -lR
> xx <- norm_laplacian(matrix(runif(100), nrow = 10))
> sessionInfo()
R Under development (unstable) (2016-12-20 r71827)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.1 LTS
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] Rcpp_0.12.8.3
loaded via a namespace (and not attached):
[1] compiler_3.4.0 tools_3.4.0
[3] RcppArmadillo_0.7.600.1.0
if the segfault does not occur with the simpler code, then one could try gdb / valgrind with SC3::norm_laplacian(matrix(runif(100), nrow = 10))
Martin
Hi Vladimir
I know you have moved past the R version, but I was also getting the segfault on 3.3.1.
I ran the simplified code suggested above and did not receive the segfault. Below is the output
Rscript norm_laplacian.R
/home/jake/miniconda3/lib/R/bin/R CMD SHLIB -o 'sourceCpp_2.so' 'norm_laplacian.cpp'
g++ -I/home/jake/miniconda3/lib/R/include -DNDEBUG -I/home/jake/miniconda3/include -I"/home/jake/R/x86_64-pc-linux-gnu-library/3.3/Rcpp/include" -I"/home/jake/R/x86_64-pc-linux-gnu-library/3.3/RcppArmadillo/include" -I"/home/jake/Documents" -fpic -I/home/jake/miniconda3/include -c norm_laplacian.cpp -o norm_laplacian.o
g++ -shared -L/home/jake/miniconda3/lib/R/lib -L/home/jake/miniconda3/lib -lgfortran -o sourceCpp_2.so norm_laplacian.o -L/home/jake/miniconda3/lib/R/lib -lRlapack -L/home/jake/miniconda3/lib/R/lib -lRblas -lgfortran -lm -lquadmath -L/home/jake/miniconda3/lib/R/lib -lR
R version 3.3.2 (2016-10-31)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.10
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets base
other attached packages:
[1] Rcpp_0.12.8
loaded via a namespace (and not attached):
[1] tools_3.3.2 RcppArmadillo_0.7.600.1.0
$ g++ --version|head -n1
g++ (Ubuntu 6.2.0-5ubuntu12) 6.2.0 20161005
I can try to work with valgrind a bit later, thanks for the help.
Here is the output from valgrind.
$ R -d valgrind -f norm_laplacian.R
==7046== Memcheck, a memory error detector
==7046== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==7046== Using Valgrind-3.12.0.SVN and LibVEX; rerun with -h for copyright info
==7046== Command: /home/jake/miniconda3/lib/R/bin/exec/R -f norm_laplacian.R
==7046==
R version 3.3.2 (2016-10-31) -- "Sincere Pumpkin Patch"
Copyright (C) 2016 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
> library(Rcpp)
> sourceCpp("norm_laplacian.cpp", showOutput=TRUE)
/home/jake/miniconda3/lib/R/bin/R CMD SHLIB -o 'sourceCpp_2.so' 'norm_laplacian.cpp'
g++ -I/home/jake/miniconda3/lib/R/include -DNDEBUG -I/home/jake/miniconda3/include -I"/home/jake/R/x86_64-pc-linux-gnu-library/3.3/Rcpp/include" -I"/home/jake/R/x86_64-pc-linux-gnu-library/3.3/RcppArmadillo/include" -I"/home/jake/Documents" -fpic -I/home/jake/miniconda3/include -c norm_laplacian.cpp -o norm_laplacian.o
g++ -shared -L/home/jake/miniconda3/lib/R/lib -L/home/jake/miniconda3/lib -lgfortran -o sourceCpp_2.so norm_laplacian.o -L/home/jake/miniconda3/lib/R/lib -lRlapack -L/home/jake/miniconda3/lib/R/lib -lRblas -lgfortran -lm -lquadmath -L/home/jake/miniconda3/lib/R/lib -lR
> xx <- norm_laplacian(matrix(runif(100), nrow = 10))
> SC3::norm_laplacian(matrix(runif(100), nrow = 10))
==7046== Use of uninitialised value of size 8
==7046== at 0x213645B8: direct_max<double> (op_max_meat.hpp:362)
==7046== by 0x213645B8: max (Mat_meat.hpp:6801)
==7046== by 0x213645B8: norm_laplacian(arma::Mat<double>) (cppFunctions.cpp:87)
==7046== by 0x21360E8C: SC3_norm_laplacian (RcppExports.cpp:49)
==7046== by 0x521DE81: R_doDotCall (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x522BD99: do_dotcall (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x5267AAC: Rf_eval (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x526B0A7: do_begin (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x526789E: Rf_eval (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x5268B8C: Rf_applyClosure (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x5267BBF: Rf_eval (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x52A6C1E: Rf_ReplIteration (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x52A6DD1: R_ReplConsole (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x52A8758: run_Rmainloop (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046==
==7046== Invalid read of size 8
==7046== at 0x213645B8: direct_max<double> (op_max_meat.hpp:362)
==7046== by 0x213645B8: max (Mat_meat.hpp:6801)
==7046== by 0x213645B8: norm_laplacian(arma::Mat<double>) (cppFunctions.cpp:87)
==7046== by 0x21360E8C: SC3_norm_laplacian (RcppExports.cpp:49)
==7046== by 0x521DE81: R_doDotCall (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x522BD99: do_dotcall (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x5267AAC: Rf_eval (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x526B0A7: do_begin (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x526789E: Rf_eval (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x5268B8C: Rf_applyClosure (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x5267BBF: Rf_eval (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x52A6C1E: Rf_ReplIteration (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x52A6DD1: R_ReplConsole (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x52A8758: run_Rmainloop (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== Address 0xa0000000a is not stack'd, malloc'd or (recently) free'd
==7046==
*** caught segfault ***
address 0xa0000000a, cause 'memory not mapped'
Traceback:
1: .Call("SC3_norm_laplacian", PACKAGE = "SC3", A)
2: SC3::norm_laplacian(matrix(runif(100), nrow = 10))
An irrecoverable exception occurred. R is aborting now ...
==7046==
==7046== Process terminating with default action of signal 11 (SIGSEGV)
==7046== at 0x5C4C4DD: raise (raise.c:53)
==7046== by 0x52A76C4: sigactionSegv (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x5C4C62F: ??? (in /lib/x86_64-linux-gnu/libpthread-2.24.so)
==7046== by 0x213645B7: direct_max<double> (op_max_meat.hpp:360)
==7046== by 0x213645B7: max (Mat_meat.hpp:6801)
==7046== by 0x213645B7: norm_laplacian(arma::Mat<double>) (cppFunctions.cpp:87)
==7046==
==7046== HEAP SUMMARY:
==7046== in use at exit: 149,681,035 bytes in 81,447 blocks
==7046== total heap usage: 275,293 allocs, 193,846 frees, 410,601,938 bytes allocated
==7046==
==7046== LEAK SUMMARY:
==7046== definitely lost: 0 bytes in 0 blocks
==7046== indirectly lost: 0 bytes in 0 blocks
==7046== possibly lost: 0 bytes in 0 blocks
==7046== still reachable: 149,681,035 bytes in 81,447 blocks
==7046== of which reachable via heuristic:
==7046== newarray : 7,152 bytes in 1 blocks
==7046== suppressed: 0 bytes in 0 blocks
==7046== Rerun with --leak-check=full to see details of leaked memory
==7046==
==7046== For counts of detected and suppressed errors, rerun with: -v
==7046== Use --track-origins=yes to see where uninitialised values come from
==7046== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)
Segmentation fault (core dumped)
Hi Jake, many thanks for your help! I've sent the following message back to Martin at Bioconductor. Will let you know when he replies.
Dear Martin,
Many thanks for your reply, it was really helpful. My collaborator ran the commands you suggested and got the following output:
$ Rscript norm_laplacian.R
/home/jake/miniconda3/lib/R/bin/R CMD SHLIB -o 'sourceCpp_2.so' 'norm_laplacian.cpp'
g++ -I/home/jake/miniconda3/lib/R/include -DNDEBUG -I/home/jake/miniconda3/include -I"/home/jake/R/x86_64-pc-linux-gnu-library/3.3/Rcpp/include" -I"/home/jake/R/x86_64-pc-linux-gnu-library/3.3/RcppArmadillo/include" -I"/home/jake/Documents" -fpic -I/home/jake/miniconda3/include -c norm_laplacian.cpp -o norm_laplacian.o
g++ -shared -L/home/jake/miniconda3/lib/R/lib -L/home/jake/miniconda3/lib -lgfortran -o sourceCpp_2.so norm_laplacian.o -L/home/jake/miniconda3/lib/R/lib -lRlapack -L/home/jake/miniconda3/lib/R/lib -lRblas -lgfortran -lm -lquadmath -L/home/jake/miniconda3/lib/R/lib -lR
R version 3.3.2 (2016-10-31)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.10
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets base
other attached packages:
[1] Rcpp_0.12.8
loaded via a namespace (and not attached):
[1] tools_3.3.2 RcppArmadillo_0.7.600.1.0
$ g++ --version|head -n1
g++ (Ubuntu 6.2.0-5ubuntu12) 6.2.0 20161005
So running the simplified code did not produce a segfault and suggested that the problem was in SC3::norm_laplacian(). And indeed, running valgrind with SC3::norm_laplacian(matrix(runif(100), nrow = 10)) did catch the error:
$ R -d valgrind -f norm_laplacian.R
==7046== Memcheck, a memory error detector
==7046== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==7046== Using Valgrind-3.12.0.SVN and LibVEX; rerun with -h for copyright info
==7046== Command: /home/jake/miniconda3/lib/R/bin/exec/R -f norm_laplacian.R
==7046==
R version 3.3.2 (2016-10-31) -- "Sincere Pumpkin Patch"
Copyright (C) 2016 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
> library(Rcpp)
> sourceCpp("norm_laplacian.cpp", showOutput=TRUE)
/home/jake/miniconda3/lib/R/bin/R CMD SHLIB -o 'sourceCpp_2.so' 'norm_laplacian.cpp'
g++ -I/home/jake/miniconda3/lib/R/include -DNDEBUG -I/home/jake/miniconda3/include -I"/home/jake/R/x86_64-pc-linux-gnu-library/3.3/Rcpp/include" -I"/home/jake/R/x86_64-pc-linux-gnu-library/3.3/RcppArmadillo/include" -I"/home/jake/Documents" -fpic -I/home/jake/miniconda3/include -c norm_laplacian.cpp -o norm_laplacian.o
g++ -shared -L/home/jake/miniconda3/lib/R/lib -L/home/jake/miniconda3/lib -lgfortran -o sourceCpp_2.so norm_laplacian.o -L/home/jake/miniconda3/lib/R/lib -lRlapack -L/home/jake/miniconda3/lib/R/lib -lRblas -lgfortran -lm -lquadmath -L/home/jake/miniconda3/lib/R/lib -lR
> xx <- norm_laplacian(matrix(runif(100), nrow = 10))
> SC3::norm_laplacian(matrix(runif(100), nrow = 10))
==7046== Use of uninitialised value of size 8
==7046== at 0x213645B8: direct_max<double> (op_max_meat.hpp:362)
==7046== by 0x213645B8: max (Mat_meat.hpp:6801)
==7046== by 0x213645B8: norm_laplacian(arma::Mat<double>) (cppFunctions.cpp:87)
==7046== by 0x21360E8C: SC3_norm_laplacian (RcppExports.cpp:49)
==7046== by 0x521DE81: R_doDotCall (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x522BD99: do_dotcall (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x5267AAC: Rf_eval (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x526B0A7: do_begin (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x526789E: Rf_eval (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x5268B8C: Rf_applyClosure (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x5267BBF: Rf_eval (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x52A6C1E: Rf_ReplIteration (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x52A6DD1: R_ReplConsole (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x52A8758: run_Rmainloop (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046==
==7046== Invalid read of size 8
==7046== at 0x213645B8: direct_max<double> (op_max_meat.hpp:362)
==7046== by 0x213645B8: max (Mat_meat.hpp:6801)
==7046== by 0x213645B8: norm_laplacian(arma::Mat<double>) (cppFunctions.cpp:87)
==7046== by 0x21360E8C: SC3_norm_laplacian (RcppExports.cpp:49)
==7046== by 0x521DE81: R_doDotCall (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x522BD99: do_dotcall (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x5267AAC: Rf_eval (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x526B0A7: do_begin (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x526789E: Rf_eval (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x5268B8C: Rf_applyClosure (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x5267BBF: Rf_eval (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x52A6C1E: Rf_ReplIteration (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x52A6DD1: R_ReplConsole (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x52A8758: run_Rmainloop (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== Address 0xa0000000a is not stack'd, malloc'd or (recently) free'd
==7046==
*** caught segfault ***
address 0xa0000000a, cause 'memory not mapped'
Traceback:
1: .Call("SC3_norm_laplacian", PACKAGE = "SC3", A)
2: SC3::norm_laplacian(matrix(runif(100), nrow = 10))
An irrecoverable exception occurred. R is aborting now ...
==7046==
==7046== Process terminating with default action of signal 11 (SIGSEGV)
==7046== at 0x5C4C4DD: raise (raise.c:53)
==7046== by 0x52A76C4: sigactionSegv (in /home/jake/miniconda3/lib/R/lib/libR.so)
==7046== by 0x5C4C62F: ??? (in /lib/x86_64-linux-gnu/libpthread-2.24.so)
==7046== by 0x213645B7: direct_max<double> (op_max_meat.hpp:360)
==7046== by 0x213645B7: max (Mat_meat.hpp:6801)
==7046== by 0x213645B7: norm_laplacian(arma::Mat<double>) (cppFunctions.cpp:87)
==7046==
==7046== HEAP SUMMARY:
==7046== in use at exit: 149,681,035 bytes in 81,447 blocks
==7046== total heap usage: 275,293 allocs, 193,846 frees, 410,601,938 bytes allocated
==7046==
==7046== LEAK SUMMARY:
==7046== definitely lost: 0 bytes in 0 blocks
==7046== indirectly lost: 0 bytes in 0 blocks
==7046== possibly lost: 0 bytes in 0 blocks
==7046== still reachable: 149,681,035 bytes in 81,447 blocks
==7046== of which reachable via heuristic:
==7046== newarray : 7,152 bytes in 1 blocks
==7046== suppressed: 0 bytes in 0 blocks
==7046== Rerun with --leak-check=full to see details of leaked memory
==7046==
==7046== For counts of detected and suppressed errors, rerun with: -v
==7046== Use --track-origins=yes to see where uninitialised values come from
==7046== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)
Segmentation fault (core dumped)
So, it looks like the problem is in finding the max of the input matrix A, specifically in the direct_max() function of Armadillo library. But the segfault only appears when running SC3::norm_laplacian() and not the simplified code. Does it suggest that I have some linking or exporting problems in the package? Can you think of any solution to this problem?
In case this can help, the SC3::norm_laplacian() function is stored in the file with the following header:
#include <armadillo>
#include <RcppArmadillo.h>
using namespace arma;
using namespace Rcpp;
Many thanks in advance, Kind regards, Vladimir
Hi guys, I am trying to make a simple fix, not sure whether it will work but worth trying. Could you please reinstall SC3 from GitHub:
install.packages("devtools")
devtools::install_github("hemberg-lab/SC3")
And then run this command again a new R session?
SC3::norm_laplacian(matrix(runif(100), nrow = 10))
It worked for me!
2017-01-06 12:44 GMT+01:00 Vladimir Kiselev notifications@github.com:
Hi guys, I am trying to make a simple fix, not sure whether it will work but worth trying. Could you please reinstall SC3 from GitHub:
install.packages("devtools") devtools::install_github("hemberg-lab/SC3")
And then run this command again a new R session?
SC3::norm_laplacian(matrix(runif(100), nrow = 10))
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/hemberg-lab/SC3/issues/33#issuecomment-270888331, or mute the thread https://github.com/notifications/unsubscribe-auth/ABNXaOmdiWZJXPo8K9uaaxElYT-AUl21ks5rPikcgaJpZM4LSTCs .
It also worked for me. Let me know if you need anymore testing. Thanks for the help
Wow, this is great, guys! At the end it was a simple conflict in namespaces between RcppArmadillo and Rccp. By removing one of them from the header the problem was solved. Thanks so much for testing it for me!
Cheers, Vlad
I am also copying a reply I received from Bioconductor's Martin Morgan without whom I would have never solved this problem. Thanks to him as well!
I don't think #include
I should have added --vanilla to the incantation
$ R --vanilla -d valgrind -f norm_laplacian.R
Using gdb seems more intimidating than it actually is. One would need to make sure ~/.R/Makevars had lines
CFLAGS = -g -O0
CXXFLAGS = -g -O0
install RcppArmadillo and SC3 (the compiler should report flags -g -O0) and to confirm that the segfault still occurs (it might not -- it could be the result of a subtle bug introduced by optimization). One would then
R --vanilla -d gdb
(gdb) r
> library(SC3)
> SC3::norm_laplacian(matrix(runif(100), nrow = 10))
probably ending up in the debugger where you could do
(gdb) up
to go up the call stack unitil you see something like
(gdb) up
#1 0x00007fffe4118ac5 in arma::Mat<double>::max (this=0x7fffffffb490)
at
/home/mtmorgan/R/x86_64-pc-linux-gnu-library/3.4-Bioc-3.5/RcppArmadillo/include/armadillo_bits/Mat_meat.hpp:6801
6801 return op_max::direct_max(memptr(), n_elem);
then down to the line that fails
(gdb) do
#0 arma::op_max::direct_max<double> (X=0xfc73dc0, n_elem=100)
at
/home/mtmorgan/R/x86_64-pc-linux-gnu-library/3.4-Bioc-3.5/RcppArmadillo/include/armadillo_bits/op_max_meat.hpp:362
362 const eT X_i = X[i];
then take a peak at the source code
(gdb) l
357 eT max_val = priv::most_neg<eT>();
358
359 uword i,j;
360 for(i=0, j=1; j<n_elem; i+=2, j+=2)
361 {
362 const eT X_i = X[i];
363 const eT X_j = X[j];
364
365 if(X_i > max_val) { max_val = X_i; }
366 if(X_j > max_val) { max_val = X_j; }
and check out things to make sure that they make sense
(gdb) p n_elem
$1 = 100
(gdb) p i
$2 = 0
(gdb) p j
$3 = 1
(gdb) call X[i]
$4 = 0.90036606369540095
etc. One might go up the call stack, verifying along the way
(gdb) up
#1 0x00007fffe4118ac5 in arma::Mat<double>::max (this=0x7fffffffb490)
at
/home/mtmorgan/R/x86_64-pc-linux-gnu-library/3.4-Bioc-3.5/RcppArmadillo/include/armadillo_bits/Mat_meat.hpp:6801
6801 return op_max::direct_max(memptr(), n_elem);
(gdb)
#2 0x00007fffe4116cae in norm_laplacian (A=...) at cppFunctions.cpp:86
86 A = exp(-A/A.max());
(gdb) print A
$6 = {<arma::Base<double, arma::Mat<double> >> =
{<arma::Base_inv_yes<arma::Mat<double> >> = {<No data fields>},
<arma::Base_eval_Mat<double, arma::Mat<double> >> = {<No data fields>},
<arma::Base_trans_default<arma::Mat<double> >> = {<No data fields>}, <No
data fields>}, n_rows = 10, n_cols = 10, n_elem = 100,
vec_state = 0, mem_state = 0, mem = 0xfc73dc0, mem_local = {
6.9533484229745318e-310, 6.9533490667683032e-310,
6.8279872255260272e-321,
6.9533484229745318e-310, 6.9533491664832504e-310,
6.9533558068923271e-310,
6.9533558068921294e-310, 6.9533490666871776e-310,
4.9406564584124654e-324,
-3.9321179452085543e+234, 6.9533480320859982e-310,
1.1680649801909133e-315, 6.9533558069148565e-310,
6.9533487849390297e-310,
1.1680649801909133e-315, 6.0259171035421809e-317},
static is_col = <optimized out>, static is_row = <optimized out>}
My next stop would be the Rcpp mailing list http://lists.r-forge.r-project.org/mailman/listinfo/rcpp-devel. They will likely either have something useful to say or be reluctant to install the 80+ packages that SC3 depends on.
Hope that helps somehow...
Martin
Hello,
I was trying out your package but I get a pretty long error list when I run sc3() I used the example your provide.
Any idea?