search

philarevalo / PopCOGenT

Microbial Populations as Clusters Of Gene Transfer
GNU General Public License v3.0
43 stars 12 forks source link

Python 3 pachage rpy2 needs to be installed to use the R function. #14

Open rsiani opened 4 years ago

rsiani commented 4 years ago

Hi, still having trouble running flexible_genome_sweeps (bash snakemake.sh).

After setting the configuration file and running the script I get this error:

Provided cores: 1 Rules claiming more threads will be scaled down. Job counts: count jobs 1 align_core_genes 1 all 1 cluster_tsv_to_tidy 1 get_flex_genes 1 make_master_table 5

rule cluster_tsv_to_tidy: input: proc/acidovorax/clusters/clusters.0.tsv output: output/acidovorax/acidovorax.0.master_presence_absence.csv jobid: 4 wildcards: organism=acidovorax

Error in job cluster_tsv_to_tidy while creating output file output/acidovorax/acidovorax.0.master_presence_absence.csv. RuleException: ValueError in line 129 of /home/rsiani/PopCOGenT-master/src/flexible_genome_sweeps/Snakefile: Python 3 package rpy2 needs to be installed to use the R function. File "/home/rsiani/PopCOGenT-master/src/flexible_genome_sweeps/Snakefile", line 129, in __rule_cluster_tsv_to_tidy File "/home/rsiani/.conda/envs/PopCOGenT/lib/python3.6/concurrent/futures/thread.py", line 56, in run Exiting because a job execution failed. Look above for error message Will exit after finishing currently running jobs. Exiting because a job execution failed. Look above for error message

However I now installed rpy2 in every possible way and still cannot get through this.

Any idea?

Thanks in advance, Rob

philarevalo commented 4 years ago

Ah! Sorry about letting the other bug fall through the cracks. I will look into this.

philarevalo commented 4 years ago

Can you open a terminal and paste the output of conda list -n PopCOGenT? I just want to make sure that there's nothing odd in the installation.

rsiani commented 4 years ago

Sure, here is the output

(base) rsiani@mooncake:~$ conda list -n PopCOGenT

packages in environment at /home/rsiani/.conda/envs/PopCOGenT:

#

Name Version Build Channel

_libgcc_mutex 0.1 main
_r-mutex 1.0.0 anacondar_1
asn1crypto 1.3.0 py36_0
bcrypt 3.1.7 py36h7b6447c_0
biopython 1.68 np111py36_0
blas 1.0 mkl
bzip2 1.0.8 h7b6447c_0
ca-certificates 2020.1.1 0
cairo 1.14.8 0
certifi 2019.11.28 py36_0
cffi 1.14.0 py36h2e261b9_0
chardet 3.0.4 py36_1003
cryptography 2.8 py36h1ba5d50_0
curl 7.68.0 hbc83047_0
decorator 4.4.1 py_0
docutils 0.16 py36_0
dropbox 9.4.0 py36_0
filechunkio 1.6 py36_0 bioconda fontconfig 2.12.1 3
freetype 2.5.5 2
ftputil 3.2 py36_0 bioconda glib 2.50.2 1
gsl 2.4 h14c3975_4
harfbuzz 0.9.39 2
icu 54.1 0
idna 2.8 py36_0
intel-openmp 2019.4 243
jbig 2.1 hdba287a_0
joblib 0.9.4 py36_0
jpeg 8d 2
krb5 1.17.1 h173b8e3_0
ld_impl_linux-64 2.33.1 h53a641e_7
libcurl 7.68.0 h20c2e04_0
libedit 3.1.20181209 hc058e9b_0
libffi 3.2.1 hd88cf55_4
libgcc 7.2.0 h69d50b8_2
libgcc-ng 9.1.0 hdf63c60_0
libgfortran 3.0.0 1
libgfortran-ng 7.3.0 hdf63c60_0
libiconv 1.14 0
libpng 1.6.37 hbc83047_0
libsodium 1.0.16 h1bed415_0
libssh2 1.8.2 h1ba5d50_0
libstdcxx-ng 9.1.0 hdf63c60_0
libtiff 4.0.6 2
libxml2 2.9.9 hea5a465_1
mkl 2017.0.4 h4c4d0af_0
ncurses 6.1 he6710b0_1
networkx 1.11 py36_1
numpy 1.11.3 py36h3dfced4_4
openssl 1.1.1d h7b6447c_4
pandas 0.19.2 np111py36_1
pango 1.40.3 1
paramiko 2.6.0 py36_0
patsy 0.5.1 py36_0
pcre 8.39 1
pip 20.0.2 py36_1
pixman 0.34.0 hceecf20_3
pycparser 2.19 py36_0
pynacl 1.3.0 py36h7b6447c_0
pyopenssl 19.1.0 py36_0
pysftp 0.2.9 py36_0 bioconda pysocks 1.7.1 py36_0
python 3.6.10 h0371630_0
python-dateutil 2.8.1 py_0
pytz 2019.3 py_0
pyyaml 5.3 py36h7b6447c_0
r-assertthat 0.1 r3.3.2_4
r-base 3.3.2 0
r-bh 1.62.0_1 r3.3.2_0
r-broom 0.4.1 r3.3.2_0
r-colorspace 1.3_1 r3.3.2_0
r-curl 2.3 r3.3.2_0
r-dbi 0.5_1 r3.3.2_0
r-dichromat 2.0_0 r3.3.2_2
r-digest 0.6.10 r3.3.2_0
r-dplyr 0.5.0 r3.3.2_0
r-forcats 0.1.1 r3.3.2_0
r-foreign 0.8_67 r3.3.2_0
r-ggplot2 2.2.0 r3.3.2_0
r-gtable 0.2.0 r3.3.2_0
r-haven 1.0.0 r3.3.2_0
r-hms 0.3 r3.3.2_0
r-httr 1.2.1 r3.3.2_0
r-jsonlite 1.1 r3.3.2_0
r-labeling 0.3 r3.3.2_2
r-lattice 0.20_34 r3.3.2_0
r-lazyeval 0.2.0 r3.3.2_0
r-lubridate 1.6.0 r3.3.2_0
r-magrittr 1.5 r3.3.2_2
r-mass 7.3_45 r3.3.2_0
r-mime 0.5 r3.3.2_0
r-mnormt 1.5_5 r3.3.2_0
r-modelr 0.1.0 r3.3.2_0
r-munsell 0.4.3 r3.3.2_0
r-nlme 3.1_128 r3.3.2_0
r-openssl 0.9.5 r3.3.2_0
r-plyr 1.8.4 r3.3.2_0
r-psych 1.6.9 r3.3.2_0
r-purrr 0.2.2 r3.3.2_0
r-r6 2.2.0 r3.3.2_0
r-rcolorbrewer 1.1_2 r3.3.2_3
r-rcpp 0.12.8 r3.3.2_0
r-readr 1.0.0 r3.3.2_0
r-readxl 0.1.1 r3.3.2_0
r-reshape2 1.4.2 r3.3.2_0
r-rvest 0.3.2 r3.3.2_0
r-scales 0.4.1 r3.3.2_0
r-selectr 0.3_0 r3.3.2_0
r-stringi 1.1.2 r3.3.2_0
r-stringr 1.1.0 r3.3.2_0
r-tibble 1.2 r3.3.2_0
r-tidyr 0.6.0 r3.3.2_0
r-tidyverse 1.0.0 r3.3.2_0
r-xml2 1.0.0 r3.3.2_0
readline 7.0 h7b6447c_5
requests 2.22.0 py36_1
rpy2 2.8.5 py36r3.3.2_2
scipy 0.18.1 np111py36_1
setuptools 45.2.0 py36_0
singledispatch 3.4.0.3 py36_0
six 1.14.0 py36_0
snakemake 3.11.2 py36_1 bioconda sqlite 3.31.1 h7b6447c_0
statsmodels 0.8.0 py36h8533d0b_0
tk 8.6.8 hbc83047_0
urllib3 1.25.8 py36_0
wheel 0.34.2 py36_0
wrapt 1.11.2 py36h7b6447c_0
xz 5.2.4 h14c3975_4
yaml 0.1.7 had09818_2
zlib 1.2.11 h7b6447c_3

philarevalo commented 4 years ago

Unfortunately, I've been unable to track this issue down. My best lead was that it had to do with the HPC having a competing environment which somehow takes precedence over conda, but I was unable to recreate this for myself. It may be a problem specific to your HPC, but I would specifically try to see if you load the conda environment and open an interactive python session whether you can import rpy2. If that works, then maybe your best bet is to contact your system administrator. I apologize for not being able to be more helpful!

npbhavya commented 3 years ago

I am running into the same error, any idea how this can be fixed?

ValueError in line 131 of /home/rsiani/PopCOGenT-master/src/flexible_genome_sweeps/Snakefile: Python 3 package rpy2 needs to be installed to use the R function.

I am able to run "import rpy2" in python shell and run the commands in rule "clusters_to_tsv" in R successfully. For now I am manually running the commands in this rule and then continue to run "bash snakemake.sh" command.

joneson commented 1 year ago

I also get the same error, using the provided Ruminococcus dataset on a linux pc. Is there a fix? I will try to figure out npbhavya's rule comment in the mean time. I have verified that rpy2 is in my PopCOGenT env.

I wrote about this in Issue https://github.com/philarevalo/PopCOGenT/issues/33#issue-1354947822 without realizing the current thread #14, and think Issue #33 can be merged here.

Elcquid commented 1 year ago

I was able to resolve this issue.

The problem seems to be in snakemake's R function: try: import rpy2.robjects as robjects except ImportError: raise ValueError( "Python 3 package rpy2 needs to be installed to use the R function.") robjects.r(format(textwrap.dedent(code), stepout=2))

If you are still having this problem, run "import rpy2.robjects as robjects" manually and see if there are missing dependencies. In my case, I got "ImportError: libncursesw.so.5". Running "conda install ncurses=5" solved it for me.