Test FROGS-3.0.0 problem

[najouamghazli]

Hello!

I'm encountering the same problem as #28

najoua@Bioinfo:~/FROGS-3.0.0/test$ bash test.sh ~/FROGS-3.0.0/ 2 4 ~/FROGSTest/

Step preprocess : Flash vendredi 23 novembre 2018, 10:48:18 (UTC+0100)
Step preprocess : Vsearch vendredi 23 novembre 2018, 10:52:48 (UTC+0100)
Step clustering vendredi 23 novembre 2018, 10:56:50 (UTC+0100)
Step remove_chimera vendredi 23 novembre 2018, 11:00:02 (UTC+0100)
Step filters vendredi 23 novembre 2018, 11:04:06 (UTC+0100)
Step ITSx vendredi 23 novembre 2018, 11:04:18 (UTC+0100)
Step affiliation_OTU vendredi 23 novembre 2018, 11:04:18 (UTC+0100)
Step affiliation_postprocess vendredi 23 novembre 2018, 11:04:55 (UTC+0100)
Step clusters_stat vendredi 23 novembre 2018, 11:04:55 (UTC+0100)
Step affiliations_stat vendredi 23 novembre 2018, 11:04:57 (UTC+0100)
Step biom_to_tsv vendredi 23 novembre 2018, 11:05:06 (UTC+0100)
Step biom_to_stdBiom vendredi 23 novembre 2018, 11:05:06 (UTC+0100)
Step tsv_to_biom vendredi 23 novembre 2018, 11:05:07 (UTC+0100)
Step tree : pynast vendredi 23 novembre 2018, 11:05:07 (UTC+0100)
Traceback (most recent call last):
  File "/home/najoua/FROGS-3.0.0//app/tree.py", line 330, in <module>
    FastTree(align, fasttree, fasttree_stderr).submit( args.log_file )
  File "/home/najoua/FROGS-3.0.0/lib/frogsUtils.py", line 141, in submit
    subprocess.check_output( self.get_cmd(), shell=True )
  File "/usr/lib/python2.7/subprocess.py", line 223, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command 'FastTree -nt -gtr /home/najoua/FROGSTest/1542967507.28_3050_pynast_aligned.fasta > /home/najoua/FROGSTest/1542967507.28_3050_fasttree.nwk 2> /home/najoua/FROGSTest/1542967507.28_3050_fasttree.stderr' returned non-zero exit status 127
Error in tree : pynast

najoua@Bioinfo:~$ ls -l FROGSTest//04-filters.fasta /home/najoua/FROGS-3.0.0//test/data/otus_pynast.fasta 
-rw-r--r-- 1 najoua najoua   65554 nov.  23 11:04 FROGSTest//04-filters.fasta
-rwxrwxr-x 1 najoua najoua 1922750 nov.   9 10:35 /home/najoua/FROGS-3.0.0//test/data/otus_pynast.fasta

najoua@Bioinfo:~$ pynast --input_fp FROGSTest//04-filters.fasta --template_fp /home/najoua/FROGS-3.0.0//test/data/otus_pynast.fasta --min_len 349 --fasta_out FROGSTest/04-filters_pynast_aligned.fasta --failure_fp FROGSTest/04-filters_pynast_fail.fasta --log_fp FROGSTest/04-filters_pynast_log.txt

najoua@Bioinfo:~$ pynast --version
Version: pynast 0.1

najoua@Bioinfo:~$ uclust --version
uclust v1.2.22q

All of this seems right, but then I can't find the "root_tree.R"

najoua@Bioinfo:~$ ./libexec/root_tree.R -v
bash: ./libexec/root_tree.R: No such file or directory

najoua@Bioinfo:~$ ls FROGS-3.0.0/libexec/
addAffiliation2biom.py        blastn                 filterSeq.py               needleallSam_to_tsv.py  root_tree.R                   splitbc.pl
aggregate_affiliated_otus.py  classifier.jar         flash                      parallelChimera.py      Rscript                       swarm
biom2tsv.py                   combine_and_split.py   ITSx                       parallelITSx.py         select_inclusive_amplicon.py  swarm2biom.py
biomFastaUpdate.py            derepSamples.py        multiAffiFromBiom.py       remove454Adapt.py       silva2retrainRDP.py           tsv2biom.py
biomTools.py                  extractSwarmsFasta.py  needleall_on_bestBlast.py  removeConta.py          sortAbundancies.py

najoua@Bioinfo:~$ Rscript -e "rmarkdown::render('/home/najoua/FROGS-3.0.0/app/r_composition.Rmd',output_file='/home/najoua/FROGS-3.0.0/test/results/12-phylo_composition.html', params=list(data='/home/najoua/FROGS-3.0.0/test/results/11-phylo_import.Rdata', varExp='Color',taxaRank1='Kingdom',taxaSet1='Bacteria',taxaRank2='Phylum',numberOfTaxa=9), intermediates_dir='/home/najoua/FROGS-3.0.0/test/results')"

processing file: r_composition.Rmd
  |....                                                             |   6%
   inline R code fragments

  |........                                                         |  12%
label: setup (with options) 
List of 1
 $ include: logi FALSE

  |............                                                     |  19%
   inline R code fragments

  |................                                                 |  25%
label: unnamed-chunk-1 (with options) 
List of 3
 $ include: logi FALSE
 $ cache  : logi FALSE
 $ warning: logi FALSE

Quitting from lines 64-68 (r_composition.Rmd) 
Error: This is R 3.2.2, package 'phyloseq' needs >= 3.4.0
In addition: Warning messages:
1: In output_metadata$get() : bytecode version mismatch; using eval
2: package 'plotly' was built under R version 3.4.4 
3: package 'ggplot2' was built under R version 3.4.4 
4: In .doLoadActions(where, attach) :
  trying to execute load actions without 'methods' package

Execution halted

I've tried to reinstall phyloseq package

source("https://bioconductor.org/biocLite.R")

Bioconductor version 3.6 (BiocInstaller 1.28.0), ?biocLite for help
A new version of Bioconductor is available after installing the most recent
version of R; see http://bioconductor.org/install

biocLite("phyloseq", type = "source")

BioC_mirror: https://bioconductor.org
Using Bioconductor 3.6 (BiocInstaller 1.28.0), R 3.4.4 (2018-03-15).
Installing package(s) ‘phyloseq’
trying URL 'https://bioconductor.org/packages/3.6/bioc/src/contrib/phyloseq_1.22.3.tar.gz'
Content type 'application/x-gzip' length 5366200 bytes (5.1 MB)

library(phyloseq)

But the same problem arises again

Can anyone help me solve this ?

Best,

Najoua

[mariabernard]

Hello Najoua

I think that both problem are linked is the version of R you have (not shure for root_tree.R).

1. Problem 1 Step tree : pynast, failed

You should still have a file ~/FROGSTest/13-tree-pynast.log. I think you already looked at it. In this file you will find the command launched by tree.py:

• one for pynast, that you tried I think
• one for fasttree
• one for root_tree.R

The final command root_tree.R should be test by using this syntaxe (as indicated in the log file): Rscript root_tree.R ....

Could you follow the log file and tell me which command fails ?

2. Problem 2: phyloseq

When you executed the RScript command of the r_composition.py, you obtain an error : Error: This is R 3.2.2, package 'phyloseq' needs >= 3.4.0

In FROGS we start working with this version of R, and I never test an older version. As it said and as we recommend, you need to upgrade your R version. This will may be solve your problem in tree if it the root_tree.R which fails.

Let me know!

[najouamghazli]

Hello Maria,

thank you for your fast response.

For the first problem, the last command that were launched is:

# reconstruction a phylogenetic tree (FastTree version : found)
Command:
    FastTree -nt -gtr /home/najoua/FROGSTest/1542967507.28_3050_pynast_aligned.fasta > /home/najoua/FROGSTest/1542967507.28_3050_fasttree.nwk 2> /home/najoua/FROGSTest/1542967507.28_3050_fasttree.stderr

Execution:
    start: 23 Nov 2018 11:05:17 

And there is no command that used the syntax: Rscript root_tree.R

As for the second problem, I don't seem to understand ! Because I'm already using R 3.4.4 and installed the phyloseq package under this version

najoua@Bioinfo:~$ R --version
R version 3.4.4 (2018-03-15) -- "Someone to Lean On"
Copyright (C) 2018 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under the terms of the
GNU General Public License versions 2 or 3.
For more information about these matters see
http://www.gnu.org/licenses/.

Thank you,

Najoua

[mariabernard]

Ok, so it's two different problem.

Could you test manually the pynast command line followed by the fasttree command line ? It seem's that th problem is during the fasttree step but what is the Error ?

Concerning R, what do you have in the ~/FROGSTest/15-phylo_composition.log ? You may have some lines like this :

# Run 1 code Rmarkdown (Rscript version : [1] "R version 3.4.0 (2017-04-21)"
[1] "Rmarkdown version:  1.5"
[1] "Phyloseq version:  1.20.0")

Do not take car about the version, my point is to check which version of R is used for you.

[najouamghazli]

Okay so here's the result of the command:

najoua@Bioinfo:~$ /home/najoua/FROGS-3.0.0//app/tree.py --nb-cpus 2 --input-otu /home/najoua/FROGS-3.0.0//test/data/to_test_pynast.fasta --biomfile /home/najoua/FROGS-3.0.0//test/data/to_test_pynast.biom --template-pynast /home/najoua/FROGS-3.0.0//test/data/otus_pynast.fasta --out-tree /home/najoua/FROGSErrors//13-tree-pynast.nwk --html /home/najoua/FROGSErrors//13-tree-pynast.html --log-file /home/najoua/FROGSErrors//13-tree-pynast.log
Traceback (most recent call last):
  File "/home/najoua/FROGS-3.0.0//app/tree.py", line 330, in <module>
    FastTree(align, fasttree, fasttree_stderr).submit( args.log_file )
  File "/home/najoua/FROGS-3.0.0/lib/frogsUtils.py", line 141, in submit
    subprocess.check_output( self.get_cmd(), shell=True )
  File "/usr/lib/python2.7/subprocess.py", line 223, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command 'FastTree -nt -gtr /home/najoua/FROGSErrors/1542985853.55_5149_pynast_aligned.fasta > /home/najoua/FROGSErrors/1542985853.55_5149_fasttree.nwk 2> /home/najoua/FROGSErrors/1542985853.55_5149_fasttree.stderr' returned non-zero exit status 127

And for the R problem, I don't even have the file ~/FROGSTest/15-phylo_composition.log

The last file that were generated is 13-tree-pynast.log

[mariabernard]

What I was asking was the results of :

FastTree -nt -gtr /home/najoua/FROGSErrors/1542985853.55_5149_pynast_aligned.fasta > /home/najoua/FROGSErrors/1542985853.55_5149_fasttree.nwk 2> /home/najoua/FROGSErrors/1542985853.55_5149_fasttree.stderr

If you do not have the input file anymore you may launch the tree.py command line with the --debug option

/home/najoua/FROGS-3.0.0//app/tree.py --nb-cpus 2 --input-otu /home/najoua/FROGS-3.0.0//test/data/to_test_pynast.fasta --biomfile /home/najoua/FROGS-3.0.0//test/data/to_test_pynast.biom --template-pynast /home/najoua/FROGS-3.0.0//test/data/otus_pynast.fasta --out-tree /home/najoua/FROGSErrors//13-tree-pynast.nwk --html /home/najoua/FROGSErrors//13-tree-pynast.html --log-file /home/najoua/FROGSErrors//13-tree-pynast.log --debug

and then the fasttree command line on those outputfiles

For the R problem where did you get the Rscript command line ?

[mariabernard]

is Fasttree in you PATH ? is it not in the libexec directory ?

for R, you have a Rscript file (probably link) in the FROGS/libexec directory. make shure that this Rscript is linked to th R 3.4.4 version

[najouamghazli]

I just soft linked R 3.4.4 to the Rscript in the libexec directory, and added Fasttree to the PATH (it already exist in the libexec directory)

but nothing changed.

najoua@Bioinfo:~$ echo $PATH
/home/najoua/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/lib/jvm/java-8-oracle/bin:/usr/lib/jvm/java-8-oracle/db/bin:/usr/lib/jvm/java-8-oracle/jre/bin:/usr/bin/fasttree:/usr/bin/libexec

najoua@Bioinfo:~/FROGS-3.0.0/test$ ls -l /usr/bin/libexec/
total 1480
-rwxr-xr-x 1 root root  12133 nov.  23 10:04 addAffiliation2biom.py
-rwxr-xr-x 1 root root  11229 nov.  23 10:04 aggregate_affiliated_otus.py
-rwxr-xr-x 1 root root  10702 nov.  23 10:04 biom2tsv.py
-rwxr-xr-x 1 root root   3594 nov.  23 10:04 biomFastaUpdate.py
-rwxr-xr-x 1 root root  26717 nov.  23 10:04 biomTools.py
-rwxr-xr-x 1 root root 195064 nov.  23 10:04 blastn
lrwxrwxrwx 1 root root     36 nov.  23 10:04 classifier.jar -> /home/najoua/RDPTools/classifier.jar
-rwxr-xr-x 1 root root  14321 nov.  23 10:04 combine_and_split.py
-rwxr-xr-x 1 root root  19874 nov.  23 10:04 derepSamples.py
-rwxr-xr-x 1 root root   4177 nov.  23 10:04 extractSwarmsFasta.py
-rwxr-xr-x 1 root root 198584 nov.  26 10:02 fasttree
-rwxr-xr-x 1 root root  10617 nov.  23 10:04 filterSeq.py
-rwxr-xr-x 1 root root  73472 nov.  23 10:04 flash
-rwxr-xr-x 1 root root 124229 nov.  23 10:04 ITSx
-rwxr-xr-x 1 root root   4393 nov.  23 10:04 multiAffiFromBiom.py
-rwxr-xr-x 1 root root   8728 nov.  23 10:04 needleall_on_bestBlast.py
-rwxr-xr-x 1 root root   6078 nov.  23 10:04 needleallSam_to_tsv.py
-rwxr-xr-x 1 root root  28745 nov.  23 10:04 parallelChimera.py
-rwxr-xr-x 1 root root  21873 nov.  23 10:04 parallelITSx.py
-rwxr-xr-x 1 root root  11012 nov.  23 10:04 remove454Adapt.py
-rwxr-xr-x 1 root root  12833 nov.  23 10:04 removeConta.py
-rwxr-xr-x 1 root root   1401 nov.  23 10:04 root_tree.R
lrwxrwxrwx 1 root root     10 nov.  26 10:16 Rscript -> /usr/bin/R
-rwxr-xr-x 1 root root   8604 nov.  23 10:04 select_inclusive_amplicon.py
-rwxr-xr-x 1 root root  18685 nov.  23 10:04 silva2retrainRDP.py
-rwxr-xr-x 1 root root   6525 nov.  23 10:04 sortAbundancies.py
-rwxr-xr-x 1 root root  26709 nov.  23 10:04 splitbc.pl
-rwxr-xr-x 1 root root 575352 nov.  23 10:04 swarm
-rwxr-xr-x 1 root root   5613 nov.  23 10:04 swarm2biom.py
-rwxr-xr-x 1 root root  14825 nov.  23 10:04 tsv2biom.py

najoua@Bioinfo:~/FROGS-3.0.0/test$ /usr/bin/libexec/Rscript --version 
R version 3.4.4 (2018-03-15) -- "Someone to Lean On"
Copyright (C) 2018 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under the terms of the
GNU General Public License versions 2 or 3.
For more information about these matters see
http://www.gnu.org/licenses/.

When I run the first command, here what it gives:

najoua@Bioinfo:~$ /home/najoua/FROGS-3.0.0//app/tree.py --nb-cpus 2 --input-otu /home/najoua/FROGS-3.0.0//test/data/to_test_pynast.fasta --biomfile /home/najoua/FROGS-3.0.0//test/data/to_test_pynast.biom --template-pynast /home/najoua/FROGS-3.0.0//test/data/otus_pynast.fasta --out-tree /home/najoua/FROGSErrors//13-tree-pynast.nwk --html /home/najoua/FROGSErrors//13-tree-pynast.html --log-file /home/najoua/FROGSErrors//13-tree-pynast.log --debug
Traceback (most recent call last):
  File "/home/najoua/FROGS-3.0.0//app/tree.py", line 330, in <module>
    FastTree(align, fasttree, fasttree_stderr).submit( args.log_file )
  File "/home/najoua/FROGS-3.0.0/lib/frogsUtils.py", line 141, in submit
    subprocess.check_output( self.get_cmd(), shell=True )
  File "/usr/lib/python2.7/subprocess.py", line 223, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command 'FastTree -nt -gtr /home/najoua/FROGSErrors/1543225355.19_4932_pynast_aligned.fasta > /home/najoua/FROGSErrors/1543225355.19_4932_fasttree.nwk 2> /home/najoua/FROGSErrors/1543225355.19_4932_fasttree.stderr' returned non-zero exit status 127

Merci beaucoup

[mariabernard]

Hi najouamghazli

What is in your /home/najoua/FROGSErrors/1543225355.19_4932_fasttree.stderr ?

a+

[najouamghazli]

Here is what in the file /home/najoua/FROGSErrors/1543225355.19_4932_fasttree.stderr

/bin/sh: 1: FastTree: not found

[mariabernard]

So it said that FastTree is not in your path.

fasttree is not named correctly in your libexec directory!

[najouamghazli]

Exactely. I renamed it and I'm running the test again right now, I'll tell you if everthing went okay ... or else if there's an error (I don't hope so)

Thank you so much for your help.