R Crashes When Running SPADE

[jkrzyspiak]

Hello,

I am using the example data to try out SPADE, but am getting the following error:

* caught segfault * address 0x0, cause 'memory not mapped'

Traceback: 1: .Call("hclust_from_data", data = x, link = as.integer(method), dist = as.integer(distance), p = as.numeric(p), DUP = FALSE, NAOK = FALSE, PACKAGE = "Rclusterpp") 2: Rclusterpp.hclust(tbl) 3: SPADE.cluster(SPADE.transform.matrix(data, transforms), k) 4: SPADE.FCSToTree(sampled_files, cells_file, graph_file, clust_file, cols = cluster_cols, transforms = transforms, k = k, desired_samples = clustering_samples, comp = comp) 5: SPADE.driver("Bendall_et_al_Science_2011_Marrow_1_SurfacePanel_Live_CD44pos_Singlets.fcs", out_dir = "output", cluster_cols = markers, panels = PANELS, transforms = flowCore::arcsinhTransform(a = 0, b = 0.2), layout = SPADE.layout.arch, downsampling_target_percent = 0.1, downsampling_target_number = NULL, downsampling_target_pctile = NULL, downsampling_exclude_pctile = 0.01, k = 200, clustering_samples = 50000)

When I try it in CytoSPADE I also still get the "unused argument (downsampling_samples = DOWNSAMPLED_EVENTS)" error even with the updates.

I am using R 3.2, SPADE 1.16 and OSX Yosemite 10.10.3. Any help would be greatly appreciated.

Thanks!

Joanna

[spholmes]

Hi, I wondered if anyone had fixed this issue, I am running a more recent version of R: R version 3.2.3 and on El capitan, but get exactly the same segmentation fault even on the simple simulation data example from the vignette.

` Targeting 4925 events for /var/folders/hp/jl3_2mhn04s572lqnvxnfv700000gp/T//RtmpVIjTqI/SimulatedRawData.fcs.density.fcs Clustering files...

* caught segfault * address 0x0, cause 'memory not mapped'

Traceback: 1: .Call("hclust_from_data", data = x, link = as.integer(method), dist = as.integer(distance), p = as.numeric(p), DUP = FALSE, NAOK = FALSE, PACKAGE = "Rclusterpp") 2: Rclusterpp.hclust(tbl) 3: SPADE.cluster(SPADE.transform.matrix(data, transforms), k) 4: SPADE.FCSToTree(sampled_files, cells_file, graph_file, clust_file, cols = cluster_cols, transforms = transforms, k = k, desired_samples = clustering_samples, comp = comp) 5: SPADE.driver(data_file_path, out_dir = output_dir, cluster_cols = c("marker1", "marker2")) `

Any progress on this would be greatly appreciated?

Thanks Susan

[dm319]

You both seem to have this problem on OS-X. I can't get it working on Linux, but the problem is Issue #104 . I wonder if it works in Windows?

[spholmes]

Actually no, it's not the igraph problem, I have no problem with the igraph, I solved the issue by installing Rclusterpp from github instead of using the cran version, it was the old version 0.2.3 of Rclusterpp (which is the one on CRAN right now) which had an issue. Installint version 0.2.4 of Rclusterpp works for me (at least I was able to reproduce the vignette), we'll see what happens next when I try to do some real data.

Thanks fro getting back to me so quickly, I'd like to include a spade example in a book I am writing but I won't unless it is pretty robust.

Best Susan

On Sun, Mar 13, 2016 at 12:01 PM, dm319 notifications@github.com wrote:

You both seem to have this problem on OS-X. I can't get it working on Linux, but the problem is Issue #104 https://github.com/nolanlab/spade/issues/104 . I wonder if it works in Windows?

— Reply to this email directly or view it on GitHub https://github.com/nolanlab/spade/issues/115#issuecomment-196021234.

Susan Holmes Professor, Statistics and BioX John Henry Fellow in Undergraduate Education Sequoia Hall, 390 Serra Mall Stanford, CA 94305 http://www-stat.stanford.edu/~susan/

[SamGG]

When the layout is modified, the demo is working under Windows R3.2.3. See #104. Thanks dm319 for pointing to the right issue.

[spholmes]

@SamGG

Actually the issue I had originally was not #104 https://github.com/nolanlab/spade/issues/104 but a different one due to Rclusterpp not being up to date on cran: can someone from the Nolan lab make sure the right version of Rclusterpp is put up in cran. Also, this needs to be fixed so the Biocoductor versions passes the tests, otherwise the BioC people will probably have to take it down, it is currently marked as deprecated.

Thanks Susan

On Sun, Mar 13, 2016 at 1:10 PM, SamGG notifications@github.com wrote:

When the layout is modified, the demo is working under Windows R3.2.3. See

104 https://github.com/nolanlab/spade/issues/104. Thanks dm319 for

pointing to the right issue.

— Reply to this email directly or view it on GitHub https://github.com/nolanlab/spade/issues/115#issuecomment-196044547.

Susan Holmes Professor, Statistics and BioX John Henry Fellow in Undergraduate Education Sequoia Hall, 390 Serra Mall Stanford, CA 94305 http://www-stat.stanford.edu/~susan/

[dm319]

I'm glad you got it working, Susan. I'm using Rclusterpp v0.2.3 on linux - it seems to work if I change the layout call.

[zbjornson]

Thanks everyone for helping out.

We have been trying to get Bioconductor to end-of-life SPADE from their repository for about a year now, in favor of just doing the easier github install method, but the process keeps getting stuck. That's why it's showing up as deprecated.

Pull requests are welcome for any fixes to improve stability...

[SamGG]

Under Windows and a fresh R 3.2.3, the package I have just installed is Rclusterpp_0.2.3 although it stated to be complied with R 3.2.4. No problem to get SPADE working under R 3.2.3.

Susan, you told you follow the installation pointed out by the wiki doc on GitHub. I did exactly so, but it ends up by installing the Rclusterpp_0.2.3 from source.

In order to get the 0.2.4 installed from Github, I did the following:

sessionInfo()
install.packages("devtools")
install.packages("curl")
devtools::install_github("nolanlab/Rclusterpp")
sessionInfo()

Susan, did you intall the same way? Zach, do you agree the install procedure of the Wiki does install from CRAN, not GitHub, or I missed something?

[zbjornson]

Yep, that's pulling from CRAN (which has Rclusterpp 0.2.3) as far as I see. I'm hoping to update the wiki later today to just list the install_github method instead as it's much easier for us to deal with versioning than CRAN or Bioconductor. I'm behind on issues -- I'm not sure if everything works using the latest version from github or not at the moment.

[spholmes]

@SamGG, Yes I used the exact install from github instructions you give from github for the up to date version of Rclusterpp_0.2.4. I had actually tried hacking the Rclusterpp.hclust function to make it work by replacing the hierarchical clustering function from the fastcluster package because the crash report showed:

.Call("hclust_from_data", data = x, link = as.integer(method), dist = as.integer(distance), p = as.numeric(p), DUP = FALSE, NAOK = FALSE, PACKAGE = "Rclusterpp")

and I think the DUP=FALSE is what is wrong.

I will try the updated procedure from github in April after the complete BioC update comes through the pipeline.

Susan

[zbjornson]

I can't reproduce this issue on Debian (Jessie) with R 3.1.1 or R 3.2.3 and Rclusterpp 0.2.4. Am I correct in understanding that this is only failing on Mac OSX?

Another one of the authors pointed out that the stock hclust function could be used, although it is very slow. Replace this line with: cluster = hclust(dist(tbl), method="ward.D"))

[rbruggner]

Yes, this is only a problem on Mac OS X and I think only a problem with Rclusterpp 0.2.3. I think it might be fixed in 0.2.4.

[spholmes]

Indeed only a problem with the CRAN version of Rclusterpp which is 0.2.3, if you upload the new version to CRAN, all should be fine, except they complain about DUP. Thanks for the quick turnaround, Susan

On Sun, Mar 13, 2016 at 10:46 PM, Robert Bruggner notifications@github.com wrote:

Yes, this is only a problem on Mac OS X and I think only a problem with Rclusterpp 0.2.3. I think it might be fixed in 0.2.4.

— Reply to this email directly or view it on GitHub https://github.com/nolanlab/spade/issues/115#issuecomment-196153204.

Susan Holmes Professor, Statistics and BioX John Henry Fellow in Undergraduate Education Sequoia Hall, 390 Serra Mall Stanford, CA 94305 http://www-stat.stanford.edu/~susan/

[zbjornson]

Thanks for confirming. Working on pushing it to CRAN this week.

[oneillkza]

I also got a segfault on OSX (El Capitan) using the Bioconductor version of SPADE. This occurred when running example(SPADE.driver) as per the current BioC build report: http://bioconductor.org/checkResults/release/bioc-LATEST/spade/oaxaca-checksrc.html

However, when I updated to the current github version of SPADE this morning, I could run SPADE.driver without a segfault. (PS: I checked and I have the 0.2.3 version of Rclusterpp).

[MissIslander]

I'm run devtools::install_github("nolanlab/Rclusterpp") on R 3.3.1 and OS X EI Capitan 10.11.6. But something was wrong I got the following error message:

Error in curl::curl_fetch_memory(url, handle = handle) : Timeout was reached

So I couldn't update Rclusterpp to version 0.2.4. I'm depressed and wondering that how to update Rclusterpp then avoid the crashes problem.

[MissIslander]

@oneillkza May I ask you some details? My computer system is the same as you. I update the spade to version 1.10.4, but the crashes problem wasn't solved. Did you finish the steps suggested in github/cran/Rclusterpp?

Thank you for your time!

[oneillkza]

@MissIslander, you could also install Rclusterpp directly from cran. I'm afraid I can't remember the steps I took to get it working, but I'd expect it was something like the following:

install.packages('Rclusterpp')
install_github("nolanlab/spade")

I'm on OSX 10.11.3 (15D21) and using R 3.2.3, for what it's worth.

[zbjornson]

@MissIslander "Timeout was reached" from that curl function means that there was a timeout downloading the package. Make sure your internet connection is stable and try again... this can be pokey and hard to troubleshoot.

One of the Rclusterpp maintainers prepared some fixes for that package, I just need to get it submitted to CRAN then you won't have to use the install_github route.

[MissIslander]

@zbjornson @oneillkza Thank you for getting back to me so quickly! I tried again this morning(Beijing Time Zone) with much faster Internet and downloaded successfully the Rclusterapp of version 0.2.4. Then it's totally fine for testing the spade with example under the suggestion in READ ME.

Thank you again for your work!