scde dqrls on Windows

[RaiaN]

Hi. I installed scde from your lab website and I cannot run the following command on Windows (R version 3.2.2):

o.ifm <- scde.error.models(counts = data, n.cores = 1, threshold.segmentation = FALSE, save.crossfit.plots = FALSE, save.model.plots = FALSE, min.nonfailed = 500, min.size.entries = 500, verbose = 1)

Error: Error in .Fortran("dqrls", qr = x[good, ] * w, n = ngoodobs, p = nvars, : "dqrls" not resolved from current namespace (scde)

Data is gene expression matrix (rows = genes, columns = samples). It works on Linux (the same versions of R and scde package).

Is there some way to run this script on Windows?

According to this https://cran.r-project.org/bin/windows/base/old/3.0.2/NEWS.R-3.0.2.html it is not possible to use dqrls on R since version 2.15.1. What is about Linux then?

P.S I simply want to calculate dist object (distances between cells) from gene expression dataset using scde.error.models if it helps...

[JEFworks]

Hi Peter,

This looks like a similar to issue to what some have encountered when installing on Mac. The issue seems to stem from fortran compilers no longer being installed by default on Windows and newer versions of Mac OS. Can you check if you have a fortran compiler installed on your Windows machine?

Best, Jean

[RaiaN]

Hi Jean, thank you very much for fast response! I've installed all available Cygwin fortran compilers for Win 8.1 and added them to PATH. I don't exactly know how to solve such problems because I am not very familiar with R programming and packages.

I suppose R should run some Fortran stuff or compile and run(?) ?

Best, Peter

[JEFworks]

Hi Peter,

Unfortunately, I'm not very familiar with Windows. Can you try installing the binary version from our BioConductor development version: http://www.bioconductor.org/packages/devel/bioc/html/scde.html#archives That will help me figure out whether this is some compiler issue or other issue.

Thanks.

Best, Jean

[RaiaN]

Hi Jean, It works! Thank you! I've installed Bioconductor development version and scde using biocLite() and it works. Could you, please, share thoughts what potentially was the source of this problem?

Best wishes, Peter

[JEFworks]

Hi Peter,

Great! When you installed using biocLite(), did you get any messages saying "Package which is only available in source form, and may need compilation of C/C++/Fortran”? If not, then you installed the pre-compiled Windows binary. So your previous source of problems was likely a compiler related issue. I’ve heard of people using RTools (https://cran.r-project.org/bin/windows/Rtools/) for building packages for R under Microsoft Windows but have not personally tried it.

Best, Jean

On Nov 20, 2015, at 4:45 PM, Peter notifications@github.com<mailto:notifications@github.com> wrote:

Hi Jean, It works! Thank you! I've installed Bioconductor development version and scde using biocLite() and it works. Could you, please, share thoughts what potentially was the source of this problem?

Best wishes, Peter

— Reply to this email directly or view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_hms-2Ddbmi_scde_issues_21-23issuecomment-2D158533499&d=CwMCaQ&c=WO-RGvefibhHBZq3fL85hQ&r=2gb0vmLv11Vi98WTAqlCXyDkhi11d9lKeGWDXEU-qNw&m=Fh9EtfRXIKco7BMQhakWhxZpyod09Tb2UAb8YTuioRs&s=lcqXKh831mRzZoZ6p5oF_K23txo_TTLYlCYyzz8cKOY&e=.

[RaiaN]

I had Rtools installed before trying Bioconductor development version. It seems that necessary fortran stuff came from R development version. I am not absolutely sure about the compilation procedure but there definitely were something like that.

Neverthelesss, I'm trying to use (still on Windows) n.cores = 1 and I get the error about necessity to use SnowParam instead of MulticoreParam (in biocParallel I suppose). It seems that something inside scde.error.models forces to use multithreaded stuff on Windows which is not available according to biocParallel

[JEFworks]

Hi Peter,

Humm, we actually switched from parallel to biocParallel because we knew Windows could not handle mclapply with mc.cores > 1, as this require forking. BiocParallel was originally created to circumvent some of these interface limitations and handle this issue automatically.

Could you provide a traceback() or some other log to help point me to where scde.error.models is trying to use multithreading even with n.cores=1?

Thanks.

Best, Jean

[RaiaN]

Hi Jean, I was trying to solve the problem. I'm sorry I've noticed that there is only warning about using SnowParam instead of MulticoreParam on Windows. However, I get the following error:

Error in tapply(unlist(x, recursive = FALSE), factor(unlist(lapply(x, : arguments should have the same length

when I run the command: o.ifm <- scde.error.models(counts = data, n.cores = 1, threshold.segmentation = FALSE, save.crossfit.plots = FALSE, save.model.plots = FALSE, min.size.entries = 100, verbose = 1)

Everything is OK when I do threshold.segmentation = FALSE. What I want is to avoid using threshold.segmentation = TRUE - because it works much slowly...

data is data.frame if it helps.

Best, Peter

[Atilz]

Hi,

I get the exactly same Error: "Error in .Fortran("dqrls", qr = x[good, ] * w, n = ngoodobs, p = nvars, : "dqrls" not resolved from current namespace (scde)" when I try to use "scde.error.models"! I'm working on Windows 10.

I have 10938 genes and 263 cells after clean count. This is the function i'm trying to use: o.ifm <- scde.error.models(counts = LSK.mat.cd, groups = sg, min.nonfailed = 20, min.count.threshold = 1, min.size.entries = 3000, n.cores = 1, min.pairs.per.cell = 60, linear.fit = T, threshold.segmentation = TRUE, save.crossfit.plots = FALSE, save.model.plots = FALSE, verbose = 1)

Here when I use linear.fit = T, the code is running with no errors but I get the "corr.a" values all 1. I played a lot with the parameters but I always get 1. also the "scde.failure.probability" plots for assessing the quality of the measurements are not look fine! So I tried to test linear.fit = F, but this time I get the above error about Fortran!

I tried all the things suggested above, checking the installed compilers and the path, installing Rtools, installing the "scde" package from BioConductor development version, but nothing works! When the linear.fit = False, it doesnt work!

I would be so grateful of you can help me with this.

Regards,

Atefeh

[JEFworks]

Hi Atefeh,

Yes, when linear.fit=T, the error models use a linear-scale fit with expression-dependent negative binomial overdispersion fit, as detailed in Fan et al. Under this linear-scale fit model, corr.a will either be 1 or -1 (if your data is really poor) but the measurement quality information will be captured in the overdispersion term.

When you set linear.fit = F, then you are using the original modeling procedures from Kharchenko et al, which depends on dqrls.

dqrls is a Fortran module that was removed at some point from the core R components but then added back in, then removed again. So the error may have to do with the version of R you're using and whether that version has it or not.

We originally tried to remedy this inconsistency by adding the needed .f files directly to the src/ folder and recompiling. But we had to remove the files for Bioconductor submission.

Anyways, I just pushed the relevant .f files back into src/. Can you please try reinstalling using

require(devtools)
devtools::install_github('hms-dbmi/scde', build_vignettes = FALSE)

I was able to fun the differential expression tutorial but with

o.ifm <- scde.error.models(counts = cd, groups = sg, n.cores = 1, threshold.segmentation = TRUE, save.crossfit.plots = FALSE, save.model.plots = FALSE, verbose = 1, linear.fit=FALSE)

without any errors.

Let me know if this helps. Thanks, Atefeh!

Best, Jean

[Atilz]

Dear Jean,

Great! It works now! Thank you very much..

Best,

Atefeh