biostars / biostar-handbook

Issue tracker for the Biostar Handbook
57 stars 12 forks source link

Issues with running the DE analysis in the step-by-step tutorial (chapter 6) #162

Closed EMDLvdHeijden closed 2 years ago

EMDLvdHeijden commented 2 years ago

I have been following the step-by-step tutorial for RNA-seq, which thus far has been great. Thanks so much for creating this content! It's very useful.

I've now reached the chapter where the differential expression analysis is done using either DESeq2 or EdgeR using these commands:

cat counts.txt | deseq2.r 3x3 > deseq2-results.csv

or

cat counts.txt | edger.r 3x3 > edger-results.csv

However, I am unable to do either. I get the following errors:

Error: package or namespace load failed for 'S4Vectors' in library.dynam(lib, package, package.lib): shared object 'S4Vectors.dylib' not found Error: package 'S4Vectors' could not be loaded Execution halted (stats)

running deseq2.r

or

Error: package or namespace load failed for 'limma': .onLoad failed in loadNamespace() for 'limma', details: call: library.dynam("limma", pkgname, libname) error: shared object 'limma.dylib' not found Error: package 'limma' could not be loaded Execution halted (stats)

running edger.r

I checked various things.

First: am I in the right environment? Yes - I'm in stats. Second: are limma and S4vectors installed? Yes - running conda list in the stats environment tells me they are. Third: maybe there is an issue with the basic set-up? Running doctor.py in the bioinfo environment tells me all is OK. Fourth: total re-install of the entire set-up: rm -rf ~/miniconda3 followed by curl http://data.biostarhandbook.com/install.sh | bash. Ran doctor.py again in the bioinfo environment (all OK) and a re-install of the stats environment using conda create --name stats python=3.8 -y, conda activate stats and URL=http://data.biostarhandbook.com/books/rnaseq/code/rnaseq-conda.txt curl $URL | xargs mamba install -y. I checked whether the installation was correct by just running deseq2.r as suggested and that gives the expected error. Everything looks good up to this point. But I still I get the same issue. So I'm stuck.

I'm not really sure what else I can do? Am I missing something or doing something wrong?

I am on MacOSX running Big Sur 11.6.

Thanks in advance.

ialbert commented 2 years ago

I am running this exact same code today (a few minutes ago actually, as I am preparing for a lecture) and I am using BigSur 11.6 and did work,

alas but I do know that sometimes things can get out of sync if other packages get installed.

I also tried the following as an alternative and perhaps you can try it yourself:


conda create -n stats -y
conda activate stats
mamba install bioconductor-edger bioconductor-deseq2 r-gplots -y

cat counts.txt | deseq2.r 3x3 > deseq2-results.csv

see if that helps, a brand new environment with minimal dependencies, not even python

ialbert commented 2 years ago

if the above does not work, paste here your

conda list 

output, I think one of the packages became out of sync and needs updating, ... that would be conda's job actually

EMDLvdHeijden commented 2 years ago

Thanks so much for your super quick reply!

Sorry - should have mentioned that I had tried that as well, prior to doing the entire re-install. I tried it again now, but same result.

Here is the output from conda list in stats:

$ conda list
# packages in environment at /Users/emdlvanderheijden/miniconda3/envs/stats:
#
# Name                    Version                   Build  Channel
_r-mutex                  1.0.1               anacondar_1    conda-forge
bioconductor-annotate     1.72.0            r41hdfd78af_0    bioconda
bioconductor-annotationdbi 1.56.1            r41hdfd78af_0    bioconda
bioconductor-biobase      2.54.0            r41h68a2ddb_0    bioconda
bioconductor-biocgenerics 0.40.0            r41hdfd78af_0    bioconda
bioconductor-biocparallel 1.28.0            r41h832b226_0    bioconda
bioconductor-biostrings   2.62.0            r41h68a2ddb_0    bioconda
bioconductor-delayedarray 0.20.0            r41h68a2ddb_0    bioconda
bioconductor-deseq2       1.34.0            r41h832b226_0    bioconda
bioconductor-edger        3.36.0            r41h832b226_0    bioconda
bioconductor-genefilter   1.76.0            r41hcf7da7f_0    bioconda
bioconductor-geneplotter  1.72.0            r41hdfd78af_0    bioconda
bioconductor-genomeinfodb 1.30.0            r41hdfd78af_0    bioconda
bioconductor-genomeinfodbdata 1.2.7             r41hdfd78af_1    bioconda
bioconductor-genomicranges 1.46.0            r41h68a2ddb_0    bioconda
bioconductor-iranges      2.28.0            r41h68a2ddb_0    bioconda
bioconductor-keggrest     1.34.0            r41hdfd78af_0    bioconda
bioconductor-limma        3.50.0            r41h68a2ddb_0    bioconda
bioconductor-matrixgenerics 1.6.0             r41hdfd78af_0    bioconda
bioconductor-s4vectors    0.32.0            r41h68a2ddb_0    bioconda
bioconductor-summarizedexperiment 1.24.0            r41hdfd78af_0    bioconda
bioconductor-xvector      0.34.0            r41h68a2ddb_0    bioconda
bioconductor-zlibbioc     1.40.0            r41h68a2ddb_0    bioconda
bwidget                   1.9.14               h694c41f_1    conda-forge
bzip2                     1.0.8                h0d85af4_4    conda-forge
c-ares                    1.18.1               h0d85af4_0    conda-forge
ca-certificates           2021.10.8            h033912b_0    conda-forge
cairo                     1.16.0            he01c77b_1009    conda-forge
cctools_osx-64            973.0.1              h3e07e27_2    conda-forge
clang                     12.0.1               h694c41f_4    conda-forge
clang-12                  12.0.1          default_he082bbe_4    conda-forge
clang_osx-64              12.0.1               h633439f_5    conda-forge
clangxx                   12.0.1          default_he082bbe_4    conda-forge
clangxx_osx-64            12.0.1               hdb584c0_5    conda-forge
compiler-rt               12.0.1               he01351e_0    conda-forge
compiler-rt_osx-64        12.0.1               hd3f61c9_0    conda-forge
curl                      7.80.0               h97da3c1_1    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
fontconfig                2.13.1            h10f422b_1005    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
freetype                  2.10.4               h4cff582_1    conda-forge
fribidi                   1.0.10               hbcb3906_0    conda-forge
gettext                   0.19.8.1          hd1a6beb_1008    conda-forge
gfortran_impl_osx-64      9.3.0               h9cc0e5e_23    conda-forge
gfortran_osx-64           9.3.0               h18f7dce_15    conda-forge
gmp                       6.2.1                h2e338ed_0    conda-forge
graphite2                 1.3.13            h2e338ed_1001    conda-forge
gsl                       2.7                  h93259b0_0    conda-forge
harfbuzz                  3.1.2                h447b35c_0    conda-forge
icu                       69.1                 he49afe7_0    conda-forge
isl                       0.22.1               hb1e8313_2    conda-forge
jbig                      2.1               h0d85af4_2003    conda-forge
jpeg                      9d                   hbcb3906_0    conda-forge
krb5                      1.19.2               h289aae4_3    conda-forge
ld64_osx-64               609                  h2487922_2    conda-forge
ldid                      2.1.2                h6a69015_3    conda-forge
lerc                      3.0                  he49afe7_0    conda-forge
libblas                   3.9.0           12_osx64_openblas    conda-forge
libcblas                  3.9.0           12_osx64_openblas    conda-forge
libclang-cpp12            12.0.1          default_he082bbe_4    conda-forge
libcurl                   7.80.0               h97da3c1_1    conda-forge
libcxx                    12.0.1               habf9029_0    conda-forge
libdeflate                1.8                  h0d85af4_0    conda-forge
libedit                   3.1.20191231         h0678c8f_2    conda-forge
libev                     4.33                 haf1e3a3_1    conda-forge
libffi                    3.4.2                h0d85af4_5    conda-forge
libgfortran               5.0.0           9_3_0_h6c81a4c_23    conda-forge
libgfortran-devel_osx-64  9.3.0               h6c81a4c_23    conda-forge
libgfortran5              9.3.0               h6c81a4c_23    conda-forge
libglib                   2.70.1               hf1fb8c0_0    conda-forge
libiconv                  1.16                 haf1e3a3_0    conda-forge
liblapack                 3.9.0           12_osx64_openblas    conda-forge
libllvm12                 12.0.1               hd011deb_2    conda-forge
libnghttp2                1.43.0               hfd382f3_1    conda-forge
libopenblas               0.3.18          openmp_h3351f45_0    conda-forge
libpng                    1.6.37               h7cec526_2    conda-forge
libssh2                   1.10.0               hd3787cc_2    conda-forge
libtiff                   4.3.0                hd146c10_2    conda-forge
libwebp-base              1.2.1                h0d85af4_0    conda-forge
libxml2                   2.9.12               h7e28ab6_1    conda-forge
libzlib                   1.2.11            h9173be1_1013    conda-forge
llvm-openmp               12.0.1               hda6cdc1_1    conda-forge
llvm-tools                12.0.1               hd011deb_2    conda-forge
lz4-c                     1.9.3                he49afe7_1    conda-forge
make                      4.3                  h22f3db7_1    conda-forge
mpc                       1.2.1                hbb51d92_0    conda-forge
mpfr                      4.1.0                h0f52abe_1    conda-forge
ncurses                   6.2                  h2e338ed_4    conda-forge
openssl                   3.0.0                h0d85af4_2    conda-forge
pango                     1.48.10              h056538c_2    conda-forge
pcre                      8.45                 he49afe7_0    conda-forge
pcre2                     10.37                ha16e1b2_0    conda-forge
pixman                    0.40.0               hbcb3906_0    conda-forge
r-askpass                 1.1               r41h28b5c78_2    conda-forge
r-assertthat              0.2.1             r41hc72bb7e_2    conda-forge
r-backports               1.4.0             r41h28b5c78_0    conda-forge
r-base                    4.1.1                h2b051ba_2    conda-forge
r-bh                      1.75.0_0          r41hc72bb7e_0    conda-forge
r-bit                     4.0.4             r41h28b5c78_0    conda-forge
r-bit64                   4.0.5             r41h28b5c78_0    conda-forge
r-bitops                  1.0_7             r41h28b5c78_0    conda-forge
r-blob                    1.2.2             r41hc72bb7e_0    conda-forge
r-brio                    1.1.3             r41h28b5c78_0    conda-forge
r-cachem                  1.0.6             r41h28b5c78_0    conda-forge
r-callr                   3.7.0             r41hc72bb7e_0    conda-forge
r-catools                 1.18.2            r41h9951f98_0    conda-forge
r-cli                     3.1.0             r41h9951f98_0    conda-forge
r-colorspace              2.0_2             r41h28b5c78_0    conda-forge
r-crayon                  1.4.2             r41hc72bb7e_0    conda-forge
r-curl                    4.3.2             r41h28b5c78_0    conda-forge
r-dbi                     1.1.1             r41hc72bb7e_0    conda-forge
r-desc                    1.4.0             r41hc72bb7e_0    conda-forge
r-diffobj                 0.3.5             r41h28b5c78_0    conda-forge
r-digest                  0.6.29            r41h9951f98_0    conda-forge
r-ellipsis                0.3.2             r41h28b5c78_0    conda-forge
r-evaluate                0.14              r41hc72bb7e_2    conda-forge
r-fansi                   0.5.0             r41h28b5c78_0    conda-forge
r-farver                  2.1.0             r41h9951f98_0    conda-forge
r-fastmap                 1.1.0             r41h9951f98_0    conda-forge
r-formatr                 1.11              r41hc72bb7e_0    conda-forge
r-futile.logger           1.4.3           r41hc72bb7e_1003    conda-forge
r-futile.options          1.0.1           r41hc72bb7e_1002    conda-forge
r-ggplot2                 3.3.5             r41hc72bb7e_0    conda-forge
r-glue                    1.5.1             r41h28b5c78_0    conda-forge
r-gplots                  3.1.1             r41hc72bb7e_0    conda-forge
r-gtable                  0.3.0             r41hc72bb7e_3    conda-forge
r-gtools                  3.9.2             r41h28b5c78_0    conda-forge
r-httr                    1.4.2             r41hc72bb7e_0    conda-forge
r-isoband                 0.2.5             r41h9951f98_0    conda-forge
r-jsonlite                1.7.2             r41h28b5c78_0    conda-forge
r-kernsmooth              2.23_20           r41he19034d_0    conda-forge
r-labeling                0.4.2             r41hc72bb7e_0    conda-forge
r-lambda.r                1.2.4             r41hc72bb7e_1    conda-forge
r-lattice                 0.20_45           r41h28b5c78_0    conda-forge
r-lifecycle               1.0.1             r41hc72bb7e_0    conda-forge
r-locfit                  1.5_9.4           r41h28b5c78_1    conda-forge
r-magrittr                2.0.1             r41h28b5c78_1    conda-forge
r-mass                    7.3_54            r41h28b5c78_0    conda-forge
r-matrix                  1.3_4             r41hc2c5f09_0    conda-forge
r-matrixstats             0.61.0            r41h28b5c78_0    conda-forge
r-memoise                 2.0.1             r41hc72bb7e_0    conda-forge
r-mgcv                    1.8_38            r41hf27e4f0_0    conda-forge
r-mime                    0.12              r41h28b5c78_0    conda-forge
r-munsell                 0.5.0           r41hc72bb7e_1003    conda-forge
r-nlme                    3.1_153           r41h749f5a1_0    conda-forge
r-openssl                 1.4.5             r41hfc1e889_1    conda-forge
r-pillar                  1.6.4             r41hc72bb7e_0    conda-forge
r-pkgconfig               2.0.3             r41hc72bb7e_1    conda-forge
r-pkgload                 1.2.4             r41h9951f98_0    conda-forge
r-plogr                   0.2.0           r41hc72bb7e_1003    conda-forge
r-png                     0.1_7           r41h28b5c78_1004    conda-forge
r-praise                  1.0.0           r41hc72bb7e_1004    conda-forge
r-processx                3.5.2             r41h28b5c78_0    conda-forge
r-ps                      1.6.0             r41h28b5c78_0    conda-forge
r-r6                      2.5.1             r41hc72bb7e_0    conda-forge
r-rcolorbrewer            1.1_2           r41h785f33e_1003    conda-forge
r-rcpp                    1.0.7             r41h9951f98_0    conda-forge
r-rcpparmadillo           0.10.7.3.0        r41h7190c71_0    conda-forge
r-rcurl                   1.98_1.5          r41h28b5c78_0    conda-forge
r-rematch2                2.1.2             r41hc72bb7e_1    conda-forge
r-rlang                   0.4.12            r41h28b5c78_0    conda-forge
r-rprojroot               2.0.2             r41hc72bb7e_0    conda-forge
r-rsqlite                 2.2.8             r41h9951f98_0    conda-forge
r-rstudioapi              0.13              r41hc72bb7e_0    conda-forge
r-scales                  1.1.1             r41hc72bb7e_0    conda-forge
r-snow                    0.4_4             r41hc72bb7e_0    conda-forge
r-survival                3.2_13            r41h28b5c78_0    conda-forge
r-sys                     3.4               r41h28b5c78_0    conda-forge
r-testthat                3.1.1             r41h9951f98_0    conda-forge
r-tibble                  3.1.6             r41h28b5c78_0    conda-forge
r-utf8                    1.2.2             r41h28b5c78_0    conda-forge
r-vctrs                   0.3.8             r41h28b5c78_1    conda-forge
r-viridislite             0.4.0             r41hc72bb7e_0    conda-forge
r-waldo                   0.3.1             r41hc72bb7e_0    conda-forge
r-withr                   2.4.3             r41hc72bb7e_0    conda-forge
r-xml                     3.99_0.8          r41h28b5c78_0    conda-forge
r-xtable                  1.8_4             r41hc72bb7e_3    conda-forge
readline                  8.1                  h05e3726_0    conda-forge
tapi                      1100.0.11            h9ce4665_0    conda-forge
tk                        8.6.11               h5dbffcc_1    conda-forge
tktable                   2.10                 h49f0cf7_3    conda-forge
xz                        5.2.5                haf1e3a3_1    conda-forge
zlib                      1.2.11            h9173be1_1013    conda-forge
zstd                      1.5.0                h582d3a0_0    conda-forge
(stats)
ialbert commented 2 years ago

What makes this difficult to troubleshoot is that my environment is just like yours. The error indicates a deeper problem, right there with the R installation itself. The basic dynamic libraries are not found, it is a deep disconnect inside there in the computer.

There is a history of the R tools breaking in conda, for example last year we had this:

https://github.com/bioconda/bioconda-recipes/issues/21084

I think a lot has to do with the new releases for MacOS, and I think in this case the way forward is to upgrade your Mac, Monterey is the new OS.

I would reinstall everything for Monterey.

rm -rf ~/miniconda3

then rerun the installation

EMDLvdHeijden commented 2 years ago

Alright thanks, will give that a whirl (probably tomorrow) and revert back to you.

Could a potential problem be that I had both DESeq2 and edgeR already installed through Rstudio? (I would think it shouldn't matter, since we're working specifically in the stats environment - but just exploring options).

ialbert commented 2 years ago

I think the two are separate, so it wouldn't affect it

Note that you can easily run my script in Rstudio, very little needs to be changed. Just the input and output file names.

I should actually make this point and perhaps I will provide an RStudio version of the scripts.

EMDLvdHeijden commented 2 years ago

Just wanted to give that a quick try. I changed the following:

# Command line argument.
args = commandArgs(trailingOnly=TRUE)  

# Input stream.
infile = "stdin"

# Output stream.
outfile = ""

to

# Command line argument.
args = "3x3"

# Input stream.
infile = "counts.txt"

# Output stream.
outfile = "deseq2-results.csv"

and seems to work just fine!

ialbert commented 2 years ago

great to know, I will put this into the book in the near future.

EMDLvdHeijden commented 2 years ago

Just to let you know I updated my mac to Monterey today and it all straight away worked (without having to do a fresh re-install of everything).

ialbert commented 2 years ago

excellent, thanks for letting me know.

ialbert commented 2 years ago

I have expanded the RNA Seq setup section to indicate that one could run the scripts in RStudio as well.