search

rsa-tools / rsat-code

This repo contains the code required to run a local version of the software suite Regulatory Sequence Analysis Tools (RSAT).
http://rsat.eu
GNU Affero General Public License v3.0
5 stars 6 forks source link

Problem installing RSAT on Ubuntu 22.04 #33

Closed pna059 closed 4 weeks ago

pna059 commented 1 year ago

Hello, I attempted installation following https://rsa-tools.github.io/installing-RSAT/unix-install-rsat/installing_RSAT_procedure.html ....but hit the wall regarding Python installation. 

root@pavla-H110M-S2H:/home/pavla/rsa-tools/rsat# source RSAT_config.bashrc && bash installer/01_${MY_OS}_packages.bash && bash installer/02_python_packages.bash  && bash installer/03_install_rsat.bash && bash installer/04_perl_packages.bash  && bash installer/06_install_organisms.bash && bash installer/07_R-and-packages.bash  && bash installer/08_apache_config.bash && bash installer/09_rsat_ws.bash && bash installer/10_clean_unnecessary_files.bash
    RSAT_SERVER_NAME=rsat_root_pavla-H110M-S2H
    OS_INSTALLER=sudo DEBIAN_FRONTEND=noninteractive apt-get
    INSTALLER_OPT=--quiet --assume-yes
    APACHE_CONFIG_FOLDER=/etc/apache2

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!     BEWARE: INSTALLATION REQUIRES SUDO RIGHTS       !!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

St 19. října 2022, 09:37:28 CEST
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
openssh-client is already the newest version (1:8.9p1-3).
0 upgraded, 0 newly installed, 0 to remove and 108 not upgraded.
Packages to be installed with sudo DEBIAN_FRONTEND=noninteractive apt-get --quiet --assume-yes

apt-utils
make
net-tools
ssh
git
cvs
wget
zip
unzip
screen
g++
apache2
libgdbm-dev
libgd-tools
libgd-gd2-perl
ghostscript
gnuplot
graphviz
mysql-client
default-jre
python
python-pip
python-setuptools
python-numpy
python-scipy
python-matplotlib
python-suds
python-rpy2
python3
python3-pip
python3-setuptools
python3-numpy
python3-scipy
python3-matplotlib
python3-rpy2
python-yaml
emacs
x11-apps
eog
ntp
curl
libcurl4-openssl-dev
libcurl4-gnutls-dev
libxml2-dev
libnet-ssleay-perl
libcrypt-ssleay-perl
libssl-dev
php
libapache2-mod-php
libapache2-mod-wsgi
rsync

bioperl-run
libbio-das-lite-perl
libbio-perl-perl
libclass-std-perl
libdbd-mysql-perl
libdbi-perl
libdigest-md5-file-perl
libemail-sender-perl
libemail-simple-creator-perl
libemail-simple-perl
libgd-perl
libio-all-perl
libjson-perl
liblockfile-simple-perl
liblog-log4perl-perl
libnet-address-ip-local-perl
libnet-smtp-tls-perl
libnet-smtps-perl
libnumber-format-perl
libobject-insideout-perl
libole-storage-lite-perl
libparallel-forkmanager-perl
libpostscript-simple-perl
librest-client-perl
libsoap-lite-perl
libsoap-wsdl-perl
libspreadsheet-xlsx-perl
libstatistics-distributions-perl
libxml-compile-cache-perl
libxml-compile-soap-perl
libxml-compile-wsdl11-perl
libxml-parser-perl
libxml-perl
libxml-simple-perl
libyaml-perl
perl-doc
pmtools

2022/10/19 09:37:29  installing apt-get library apt-utils
2022/10/19 09:37:30  installing apt-get library make
2022/10/19 09:37:30  installing apt-get library net-tools
2022/10/19 09:37:31  installing apt-get library ssh
2022/10/19 09:37:31  installing apt-get library git
2022/10/19 09:37:38  installing apt-get library cvs
2022/10/19 09:37:38  installing apt-get library wget
2022/10/19 09:37:39  installing apt-get library zip
2022/10/19 09:37:39  installing apt-get library unzip
2022/10/19 09:37:40  installing apt-get library screen
2022/10/19 09:37:40  installing apt-get library g++
2022/10/19 09:37:41  installing apt-get library apache2
2022/10/19 09:37:41  installing apt-get library libgdbm-dev
2022/10/19 09:37:42  installing apt-get library libgd-tools
2022/10/19 09:37:42  installing apt-get library libgd-gd2-perl
2022/10/19 09:37:43  installing apt-get library ghostscript
2022/10/19 09:37:43  installing apt-get library gnuplot
2022/10/19 09:37:44  installing apt-get library graphviz
2022/10/19 09:37:44  installing apt-get library mysql-client
2022/10/19 09:37:45  installing apt-get library default-jre
2022/10/19 09:37:46  installing apt-get library python
E: Package 'python' has no installation candidate

when I tried to continue compilation, there is an error at the end:

$ cat install_tests/crer_scan_python3_err.txt

Traceback (most recent call last):
  File "python-scripts/crer_scan.py", line 177, in <module>
    import scipy.stats
ModuleNotFoundError: No module named 'scipy'
make[1]: *** [makefiles/crer-scan_test.mk:42: demo_eve] Error 1

Is there any solution/workaround?

Thanks!

eead-csic-compbio commented 1 year ago

Hi @pna059 , it seems your Ubuntu's apt does not have a package called 'python', which apparently happens with 22.04, while it works on my 20.04 system.

A possible workaround is to edit lines 102-109 of file 01_ubuntu_packages.bash

https://github.com/rsa-tools/rsat-code/blob/51749dfbe0b7d116aec80657330a9de23f721304/installer/01_ubuntu_packages.bash#L102

replacing python- with python2- or python2.7-

Can you please try that? Bruno

pna059 commented 1 year ago

Hi, thank you for the advice. I will try an give feedback. However, in the meanwhile, I installed RSAT on a Linux server following

There, I am getting only partial output and the problem is probably matrix clustring step.

(/home/jovyan/my-conda-envs/rsat) rstudio@jupyter-pavlan--rsa-2dtools:~/pavlan/$ rsat peak-motifs -v 1 -title 24DAP_cluster1_position_analysis -i domCTSS_24DAP_cluster1_100bp.fa -motif_db footprintDB transfac footprintDB.plants.motif.tf -outdir domCTSS_24DAP_cluster1_100bp_RSATpos -prefix peak-motifs -r_plot -disco positions -origin center -nmotifs 5 -minol 6 -maxol 7 -merge_lengths -1str -markov -2 -noov 
# sequence length: 61408, number of masked symbols: 634 (1.03 percent of the sequences)
WARNING: ignoring environment value of R_HOME
WARNING: ignoring environment value of R_HOME
WARNING: ignoring environment value of R_HOME
WARNING: ignoring environment value of R_HOME
WARNING: ignoring environment value of R_HOME
WARNING: ignoring environment value of R_HOME
Illegal division by zero at /home/rstudio/my-conda-envs/rsat/bin/../share/rsat/perl-scripts/matrix-clustering line 2933, <__ANONIO__> line 128.
Error
        OpenInputFile: File domCTSS_24DAP_cluster1_100bp_RSATpos/results/positions_6-7nt/peak-motifs_positions-1str-noov_6-7nt_ci50_pssm_sig_matrices_nr_cluster_root_motifs.tf does not exist.
Error
eead-csic-compbio commented 1 year ago

Perhaps those warnings about R_HOME indicate that some previous config steps went wrong @morganeTC @jaimicore @jvanheld ?

pna059 commented 1 year ago

Hi, I am running RSAT from a docker container on a Jupyter notebook, so the environment might be unusual the WARNING was removed by unset R_HOME, The line 2906 of matrix-clustering script was fixed by setting the: my $max_sys_process = 8;

Another attempt gives:

(/home/jovyan/my-conda-envs/rsat) rstudio@jupyter-pavlan--rsa-2dtools:~/pavlan/Barley_CAGE_motifsrsat peak-motifs -v 1 -title 24DAP_cluster1_position_analysis -i domCTSS_24DAP_cluster1_100bp.fa -motif_db footprintDB transfac footprintDB.plants.motif.tf -outdir domCTSS_24DAP_cluster1_100bp_RSATpos -prefix peak-motifs -r_plot -disco positions -origin center -nmotifs 5 -minol 6 -maxol 7 -merge_lengths -1str -markov auto -img_format png
# sequence length: 61408, number of masked symbols: 634 (1.03 percent of the sequences)
Smartmatch is experimental at /home/rstudio/my-conda-envs/rsat/bin/../share/rsat/perl-scripts/lib/RSA2.cgi.lib line 2797.
/home/jovyan/my-conda-envs/rsat/share/rsat/perl-scripts/convert-matrix -i domCTSS_24DAP_cluster1_100bp_RSATpos/results/positions_6-7nt/peak-motifs_positions-1str-noov_6-7nt_ci50_pssm_sig_matrices_rescaled.tf -from tf -to tf -o domCTSS_24DAP_cluster1_100bp_RSATpos/results/positions_6-7nt/peak-motifs_positions-1str-noov_6-7nt_ci50_pssm_sig_matrices_nr_data/sig_input_motifs_processed_1.tf
Cannot start another process while you are in the child process at /home/rstudio/my-conda-envs/rsat/bin/../share/rsat/perl-scripts/matrix-clustering line 2939.
Cannot start another process while you are in the child process at /home/rstudio/my-conda-envs/rsat/bin/../share/rsat/perl-scripts/matrix-clustering line 2939.
Cannot start another process while you are in the child process at /home/rstudio/my-conda-envs/rsat/bin/../share/rsat/perl-scripts/matrix-clustering line 2939.
Error in round(digits = round.digits, x) : 
  non-numeric argument to mathematical function
Calls: heatmap.simple
Execution halted
Can't call method "get_attribute" on an undefined value at /home/rstudio/my-conda-envs/rsat/bin/../share/rsat/perl-scripts/lib/RSAT/MatrixAlignment.pm line 272.
Can't call method "get_attribute" on an undefined value at /home/rstudio/my-conda-envs/rsat/bin/../share/rsat/perl-scripts/lib/RSAT/MatrixAlignment.pm line 272.
Can't call method "get_attribute" on an undefined value at /home/rstudio/my-conda-envs/rsat/bin/../share/rsat/perl-scripts/lib/RSAT/MatrixAlignment.pm line 272.

and results in incomplete outputs.

eead-csic-compbio commented 1 year ago

Hi @najlaksouri , can you share with @pna059 the Dockerfile you created for https://hub.docker.com/r/ksouri1/rsat_nksouri ? That might help solve the problems encountered, Bruno

najlaksouri commented 1 year ago

Hi @pna059 ,

Unfortunately, as i am out of the office i cannot share the Dockerfile, i will do it as soon as possible.

However, i did some trials installing RSAT docker on Ubuntu VirtutalBox and running a test data. The installation was done perfectly using the following commands:

  1. $ docker pull ksouri1/rsat_nksouri
  2. $ docker run -t -i -p 8080:80 -v $HOME/rsat_results:/home/rsat_user/rsat_results -v $HOME/rsat_data:/packages/rsat/public_html/data/ -it ksouri1/rsat_nksouri:latest

Nevertheless, i got the same warnings "Can't call method "get_attribute" on an undefined value at /home/rstudio/my-conda-envs/rsat/bin/../share/rsat/perl-scripts/lib/RSAT/MatrixAlignment.pm line 272."

Those warnings do not really affect the results as I have obtained the same results as from RSAT webserver.
@jaimicore Do have any clue how to fix this?

If you wish i can try to use your sequences to check the final output.

xhejtman commented 1 year ago

Further problem here is, that perl code in matrix-clustering assumes, that ulimit -u returns always a number, which is not correct. It can return 'unlimited' which translates to 0 and few lines below (about 2933) is divison by zero error.

xhejtman commented 1 year ago

Using your container, the output is similar except: Cannot start another process while you are in the child process at /home/rstudio/my-conda-envs/rsat/bin/../share/rsat/perl-scripts/matrix-clustering line 2939. This error is not present. So this seems to be the real problem.

xhejtman commented 1 year ago

So, the difference is, that your docker version does not run matrix-clustering while ours does. I do not know why. And apparently, matrix-clustering is not working.

najlaksouri commented 1 year ago

Exactly, In fact, when i try to use the matrix-clustering tool separetely, i get erros

xhejtman commented 1 year ago

I found that peak-motifs is different to the one that is part of conda package. Perhaps, the version in conda package is older and it it missing -cluster none option at all.

line 2563 in your version:

$cmd .= " -cluster ".$main::param{asmb_clustering};

this line is missing completely in our conda version. If I fix it for me, computation is kinda working.

pna059 commented 1 year ago

To the installations on Ubuntu 20.04 and 22.04. I tried on both systems and the complaint about missing python appears on both. Both have python3. Are really both python and python3 required?

Required apt-get packages

PACKAGES_REQUIRED=" apt-utils make net-tools ssh git cvs wget zip unzip screen g++ apache2 libgdbm-dev libgd-tools libgd-gd2-perl ghostscript gnuplot graphviz mysql-client default-jre python python-pip python-setuptools python-numpy python-scipy python-matplotlib python-suds python-rpy2 python3 python3-pip python3-setuptools python3-numpy python3-scipy python3-matplotlib python3-rpy2 python-yaml emacs x11-apps eog ntp curl

Checking the installation result gives:

$ make -f makefiles/install_tests.mk all

Checking RSAT_path
    ./install_tests/RSAT_path.txt

Checking operating system
    ./install_tests/operating_system.txt

Checking Perl modules
    ./install_tests/perl_modules_check_log.txt
    ./install_tests/perl_modules_check_err.txt
    Missing modules 43 ./install_tests/perl_modules_check_missing.txt

    ./install_tests/supported_organisms.tsv
    ./install_tests/supported_organisms_err.txt
Supported organisms 0

R version
    ./install_tests/R_version.txt
    ./install_tests/R_version_err.txt

Testing random-seq | purge-sequence
    ./install_tests/rand_purged.fa
    ./install_tests/rand_purged_err.txt

crer-scan with python 3
    ./install_tests/crer_scan_python3_log.txt
    ./install_tests/crer_scan_python3_err.txt
make: *** [makefiles/install_tests.mk:126: crer_scan_python3] Error 2
eead-csic-compbio commented 1 year ago

This one's for you probably @jvanheld

eead-csic-compbio commented 4 weeks ago

For @pna059 and other facing similar problems, the Docker container is being regularly updated and can be run as explained at https://rsa-tools.github.io/installing-RSAT/RSAT-Docker/RSAT-Docker-tuto.html