Closed Toliman06 closed 6 years ago
Hi,
This means that you did not install RDP Classifier or that the java jar file classifier.jar is not in your PATH (which is very classical as jar file are not binary file).
Does it solve the problem ?
oh no! sorry, I do not read well enough your message.
You installed RDP and you created the link, but it's not a explicitly a classifier.jar . Which version of RDP did you install ? We tested the version 2.0.2 which should contain after installation a classifier.jar file ....
Oups... Sorry for the bad symbolic link...
I'm unsing the latest github version. I will try the 2.0.2...
Good news! I can go through the affiliation_OTU step! ... but I have a new problem:
Step preprocess lundi 19 mars 2018, 16:06:38 (UTC+0100)
Step clustering lundi 19 mars 2018, 16:07:26 (UTC+0100)
Step remove_chimera lundi 19 mars 2018, 16:08:06 (UTC+0100)
Step filters lundi 19 mars 2018, 16:09:20 (UTC+0100)
Step affiliation_OTU lundi 19 mars 2018, 16:09:20 (UTC+0100)
Step clusters_stat lundi 19 mars 2018, 16:09:24 (UTC+0100)
Step affiliations_stat lundi 19 mars 2018, 16:09:26 (UTC+0100)
Step biom_to_tsv lundi 19 mars 2018, 16:09:36 (UTC+0100)
Step biom_to_stdBiom lundi 19 mars 2018, 16:09:36 (UTC+0100)
Step tsv_to_biom lundi 19 mars 2018, 16:09:36 (UTC+0100)
Step tree : pynast lundi 19 mars 2018, 16:09:36 (UTC+0100)
Traceback (most recent call last):
File "/usr/bin/FROGS-2.0.0//app/tree.py", line 328, in <module>
Pynast(args.input_otu, args.template_pynast, min_len, align, pynast_fail, pynast_log).submit( args.log_file )
File "/usr/bin/FROGS-2.0.0/lib/frogsUtils.py", line 140, in submit
subprocess.check_output( self.get_cmd(), shell=True )
File "/usr/lib/python2.7/subprocess.py", line 574, in check_output
raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command 'pynast --input_fp results//04-filters.fasta --template_fp /usr/bin/FROGS-2.0.0//test/data/otus_pynast.fasta --min_len 349 --fasta_out results/04-filters_pynast_aligned.fasta --failure_fp results/04-filters_pynast_fail.fasta --log_fp results/04-filters_pynast_log.txt 2> /dev/null' returned non-zero exit status 1
Error in tree : pynast
[EDIT] and I'm using the 1.2.2 version of pynast
/usr/bin/FROGS-2.0.0/test$ pip freeze | grep pynast
pynast==1.2.2
Could you test manually
export PATH=[FROGS_DIR]/libexec:$PATH # change by your FROGS_DIR path! pynast --version
ok ?
ls -l results//04-filters.fasta /usr/bin/FROGS-2.0.0//test/data/otus_pynast.fasta
Do they exist ?
pynast --input_fp results//04-filters.fasta --template_fp /usr/bin/FROGS-2.0.0//test/data/otus_pynast.fasta --min_len 349 --fasta_out results/04-filters_pynast_aligned.fasta --failure_fp results/04-filters_pynast_fail.fasta --log_fp results/04-filters_pynast_log.txt
Does it work ?
changing the path variable:
dunoyerd@XXX:/usr/bin/FROGS-2.0.0/test$`
export PATH=/usr/bin/FROGS-2.0.0/libexec:$PATH
dunoyerd@XXX:/usr/bin/FROGS-2.0.0/test$ pynast --version
/usr/local/lib/python2.7/dist-packages/cogent/evolve/likelihood_tree.py:7: UserWarning: Not using MPI as mpi4py not found
from cogent.util.parallel import MPI
Version: pynast 0.1
Verify that the files exist:
dunoyerd@XXX:/usr/bin/FROGS-2.0.0/test$ ls -l results/04-filters.fasta /usr/bin/FROGS-2.0.0//test/data/otus_pynast.fasta
-rw-rw-r-- 1 dunoyerd dunoyerd 668450 mars 20 09:11 results/04-filters.fasta
-rwxr-xr-x 1 dunoyerd dunoyerd 1922750 août 8 2017 /usr/bin/FROGS-2.0.0//test/data/otus_pynast.fasta
finally:
dunoyerd@XXX:/usr/bin/FROGS-2.0.0/test$ pynast --input_fp results//04-filters.fasta --template_fp /usr/bin/FROGS-2.0.0//test/data/otus_pynast.fasta --min_len 349 --fasta_out results/04-filters_pynast_aligned.fasta --failure_fp results/04-filters_pynast_fail.fasta --log_fp results/04-filters_pynast_log.txt
/usr/local/lib/python2.7/dist-packages/cogent/evolve/likelihood_tree.py:7: UserWarning: Not using MPI as mpi4py not found
from cogent.util.parallel import MPI
Traceback (most recent call last):
File "/usr/local/bin/pynast", line 194, in <module>
main()
File "/usr/local/bin/pynast", line 178, in main
for seq, status in pynast_iterator:
File "/usr/local/lib/python2.7/dist-packages/pynast/util.py", line 651, in ipynast_seqs
current_result = pw_alignment_iterator.next()
File "/usr/local/lib/python2.7/dist-packages/pynast/pycogent_backports/uclust.py", line 399, in uclust_search_and_align_from_fasta_filepath
TmpDir=tmp_dir,HALT_EXEC=HALT_EXEC)
File "/usr/local/lib/python2.7/dist-packages/cogent/app/util.py", line 166, in __init__
self._error_on_missing_application(params)
File "/usr/local/lib/python2.7/dist-packages/cogent/app/util.py", line 435, in _error_on_missing_application
% command
cogent.app.util.ApplicationNotFoundError: Cannot find uclust. Is it installed? Is it in your path?
I seems there is a problem with uclust...
[EDIT] Befor you ask me...
dunoyerd@XXX:/usr/bin/FROGS-2.0.0/test$ locate uclust
/usr/local/lib/python2.7/dist-packages/cogent/app/uclust.py
/usr/local/lib/python2.7/dist-packages/cogent/app/uclust.pyc
/usr/local/lib/python2.7/dist-packages/pynast/pycogent_backports/uclust.py
/usr/local/lib/python2.7/dist-packages/pynast/pycogent_backports/uclust.pyc
Hi again,
Yes uclust is in my PATH! ( http://www.drive5.com/uclust/downloads1_2_22q.html ) it doest not seem to be your case!
uclust --version uclust v1.2.22q
Normally the cogent warning doesn't cause any trouble because we redirect stderr to /dev/null in FROGS..... but pynast need to work!
Thanks!
I installed it and it seems to work now (at least pynast --input_fp results//04-filters.fasta --template_fp /usr/bin/FROGS-2.0.0//test/data/otus_pynast.fasta --min_len 349 --fasta_out results/04-filters_pynast_aligned.fasta --failure_fp results/04-filters_pynast_fail.fasta --log_fp results/04-filters_pynast_log.txt does not crash)
I also installed mpi4py:
sudo apt install libopenmpi-dev
sudo pip install mpi4py
These programs are nor mentioned in the required dependencies list...
indeed, but mpi is not used, so the dependency is not necessary (and the warning message is redirected to /dev/null in FROGS so not a problem).
Could you confirm (when you finish) that you successfuly install all dependencies, so I could close this issue ?
Thank you any way to take time to test FROGS. (we will propose a conda/toolshed for the next release, at most for summer .....)
Unfortunately, I have a new problem with pynast:
dunoyerd@ADNid:/usr/bin/FROGS-2.0.0/test$ bash test.sh /usr/bin/FROGS-2.0.0 4 4 results2
Step preprocess mardi 20 mars 2018, 13:09:01 (UTC+0100)
Step clustering mardi 20 mars 2018, 13:09:47 (UTC+0100)
Step remove_chimera mardi 20 mars 2018, 13:10:26 (UTC+0100)
Step filters mardi 20 mars 2018, 13:11:34 (UTC+0100)
Step affiliation_OTU mardi 20 mars 2018, 13:11:35 (UTC+0100)
Step clusters_stat mardi 20 mars 2018, 13:11:40 (UTC+0100)
Step affiliations_stat mardi 20 mars 2018, 13:11:41 (UTC+0100)
Step biom_to_tsv mardi 20 mars 2018, 13:11:51 (UTC+0100)
Step biom_to_stdBiom mardi 20 mars 2018, 13:11:51 (UTC+0100)
Step tsv_to_biom mardi 20 mars 2018, 13:11:51 (UTC+0100)
Step tree : pynast mardi 20 mars 2018, 13:11:51 (UTC+0100)
Traceback (most recent call last):
File "/usr/bin/FROGS-2.0.0/app/tree.py", line 330, in <module>
RootTree(fasttree, args.out_tree).submit(args.log_file)
File "/usr/bin/FROGS-2.0.0/lib/frogsUtils.py", line 135, in submit
FH_log.write( '# ' + self.description + ' (' + os.path.basename(self.program) + ' version : ' + self.get_version() + ')\n' )
File "/usr/bin/FROGS-2.0.0/app/tree.py", line 147, in get_version
return Cmd.get_version(self, 'stdout').split()[-1].strip()
IndexError: list index out of range
Error in tree : pynast
I seems to be an Index out of range problem...
pynast --version
Version: pynast 0.1
which pynast
/usr/local/bin/pynast
I checked with pip, this is the latest version installed. I downloaded package from https://pypi.python.org/pypi/pynast (as precognized) then build it and copy it on the previous pip installed version... The error still persist...
[EDIT] The "Step tree" takes 3-4 minutes before crashing...
Nop it's not a pynast error.
RootTree(fasttree, args.out_tree).submit(args.log_file)
RootTree is a python class which call the FROGS_DIR/libexec/root_tree.R
Do you install Rscript and link it in the libexec dir ?
Could you test
./libexec/root_tree.R -v root_tree.R : 1.0.0
dunoyerd@XXX:/usr/bin/FROGS-2.0.0$` ./libexec/root_tree.R -v
Error in library(phangorn) : aucun package nommé ‘phangorn’ n'est trouvé
Exécution arrêtée
then
sudo Rscript -e 'install.packages("phangorn", repos="https://cran.rstudio.com")'
(strange I already installed phangorn...)
dunoyerd@XXX:/usr/bin/FROGS-2.0.0$ ./libexec/root_tree.R -v
root_tree.R : 1.0.0
retry the test launch... (I also re-installed rmarkdown...)
dunoyerd@XXX:/usr/bin/FROGS-2.0.0/test$ bash test.sh /usr/bin/FROGS-2.0.0 4 4 /usr/bin/FROGS-2.0.0/test/results
Step preprocess mardi 20 mars 2018, 16:45:15 (UTC+0100)
Step clustering mardi 20 mars 2018, 16:46:02 (UTC+0100)
Step remove_chimera mardi 20 mars 2018, 16:46:41 (UTC+0100)
Step filters mardi 20 mars 2018, 16:47:49 (UTC+0100)
Step affiliation_OTU mardi 20 mars 2018, 16:47:50 (UTC+0100)
Step clusters_stat mardi 20 mars 2018, 16:47:53 (UTC+0100)
Step affiliations_stat mardi 20 mars 2018, 16:47:53 (UTC+0100)
Step biom_to_tsv mardi 20 mars 2018, 16:48:03 (UTC+0100)
Step biom_to_stdBiom mardi 20 mars 2018, 16:48:03 (UTC+0100)
Step tsv_to_biom mardi 20 mars 2018, 16:48:03 (UTC+0100)
Step tree : pynast mardi 20 mars 2018, 16:48:03 (UTC+0100)
Step tree : mafft mardi 20 mars 2018, 16:51:30 (UTC+0100)
Step r_import_data mardi 20 mars 2018, 16:52:16 (UTC+0100)
Step r_composition mardi 20 mars 2018, 16:52:32 (UTC+0100)
Traceback (most recent call last):
File "/usr/bin/FROGS-2.0.0/app/r_composition.py", line 107, in <module>
Rscript(html, data, args.varExp, args.taxaRank1.strip(), str(taxaSet1.strip()), args.taxaRank2.strip(), args.numberOfTaxa).submit( args.log_file )
File "/usr/bin/FROGS-2.0.0/lib/frogsUtils.py", line 140, in submit
subprocess.check_output( self.get_cmd(), shell=True )
File "/usr/lib/python2.7/subprocess.py", line 574, in check_output
raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command 'Rscript -e "rmarkdown::render('/usr/bin/FROGS-2.0.0/app/r_composition.Rmd',output_file='/usr/bin/FROGS-2.0.0/test/results/12-phylo_composition.html', params=list(data='/usr/bin/FROGS-2.0.0/test/results/11-phylo_import.Rdata', varExp='Color',taxaRank1='Kingdom',taxaSet1='Bacteria',taxaRank2='Phylum',numberOfTaxa=9), intermediates_dir='/usr/bin/FROGS-2.0.0/test/results')" 2> /dev/null' returned non-zero exit status 1
Error in r_composition
I will try to be sure to install all the R libraries...
One day, I will do it! I launch the faulty command. Here I have more information:
dunoyerd@XXX:/usr/bin/FROGS-2.0.0/test$ Rscript -e "rmarkdown::render('/usr/bin/FROGS-2.0.0/app/r_composition.Rmd',output_file='/usr/bin/FROGS-2.0.0/test/results/12-phylo_composition.html', params=list(data='/usr/bin/FROGS-2.0.0/test/results/11-phylo_import.Rdata', varExp='Color',taxaRank1='Kingdom',taxaSet1='Bacteria',taxaRank2='Phylum',numberOfTaxa=9), intermediates_dir='/usr/bin/FROGS-2.0.0/test/results')"
[WARNING] This document format requires a nonempty <title> element.
Please specify either 'title' or 'pagetitle' in the metadata.
Falling back to 'file2d21b0e37af.utf8'
processing file: r_composition.Rmd
|.... | 6%
ordinary text without R code
|........ | 12%
label: setup (with options)
List of 1
$ include: logi FALSE
|............ | 19%
inline R code fragments
|................ | 25%
label: unnamed-chunk-1 (with options)
List of 3
$ include: logi FALSE
$ cache : logi FALSE
$ warning: logi FALSE
|.................... | 31%
ordinary text without R code
|........................ | 38%
label: unnamed-chunk-2 (with options)
List of 5
$ fig.width : num 10
$ fig.height: num 8
$ echo : logi FALSE
$ comment : logi NA
$ cache : logi FALSE
Quitting from lines 70-73 (r_composition.Rmd)
Error in combine_vars(data, params$plot_env, vars, drop = params$drop) :
At least one layer must contain all variables used for facetting
Calls: <Anonymous> ... f -> <Anonymous> -> f -> <Anonymous> -> combine_vars
Hi!
We will do it!!
I can't reproduce the problem, so my guess is it's a version problem. Could you open the results/12-phylo_composition.log, and copy the R, rmarkdown and phyloseq versions ?
minimal requirement
# Run 1 code Rmarkdown (Rscript version : [1] "R version 3.4.0 (2017-04-21)" [1] "Rmarkdown version: 1.5" [1] "Phyloseq version: 1.20.0")
it also work for me with those versions
# Run 1 code Rmarkdown (Rscript version : [1] "R version 3.4.3 (2017-11-30)" [1] "Rmarkdown version: 1.6" [1] "Phyloseq version: 1.19.1")
MMMhhhh... It seems I have a newer version of Rmarkdown and Phyloseq...
# Run 1 code Rmarkdown (Rscript version : [1] "R version 3.4.3 (2017-11-30)"
[1] "Rmarkdown version: 1.9"
[1] "Phyloseq version: 1.22.3")
I resintalled Rmarkdown and relaunch analysis...
dunoyerd@XXX:/usr/bin/FROGS-2.0.0/test$ bash test.sh /usr/bin/FROGS-2.0.0 4 4 /usr/bin/FROGS-2.0.0/test/results
Step preprocess mercredi 21 mars 2018, 11:59:38 (UTC+0100)
Step clustering mercredi 21 mars 2018, 12:00:24 (UTC+0100)
Step remove_chimera mercredi 21 mars 2018, 12:01:02 (UTC+0100)
Step filters mercredi 21 mars 2018, 12:02:06 (UTC+0100)
Step affiliation_OTU mercredi 21 mars 2018, 12:02:06 (UTC+0100)
Step clusters_stat mercredi 21 mars 2018, 12:02:11 (UTC+0100)
Step affiliations_stat mercredi 21 mars 2018, 12:02:13 (UTC+0100)
Step biom_to_tsv mercredi 21 mars 2018, 12:02:23 (UTC+0100)
Step biom_to_stdBiom mercredi 21 mars 2018, 12:02:23 (UTC+0100)
Step tsv_to_biom mercredi 21 mars 2018, 12:02:23 (UTC+0100)
Step tree : pynast mercredi 21 mars 2018, 12:02:23 (UTC+0100)
Step tree : mafft mercredi 21 mars 2018, 12:05:51 (UTC+0100)
Step r_import_data mercredi 21 mars 2018, 12:06:41 (UTC+0100)
Traceback (most recent call last):
File "/usr/bin/FROGS-2.0.0/app/r_import_data.py", line 117, in <module>
Rscript(biomfile, samplefile, treefile, html, str(args.normalization).upper(), data, ranks).submit(args.log_file)
File "/usr/bin/FROGS-2.0.0/lib/frogsUtils.py", line 140, in submit
subprocess.check_output( self.get_cmd(), shell=True )
File "/usr/lib/python2.7/subprocess.py", line 574, in check_output
raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command 'Rscript -e "rmarkdown::render('/usr/bin/FROGS-2.0.0/app/r_import_data.Rmd',output_file='/usr/bin/FROGS-2.0.0/test/results/11-phylo_import.html', params=list(biomfile='/usr/bin/FROGS-2.0.0/test/results/08-affiliation_std.biom', samplefile='/usr/bin/FROGS-2.0.0/test/data/sample_metadata.tsv', treefile='/usr/bin/FROGS-2.0.0/test/results/10b-tree.nwk', normalization=FALSE, outputRdata='/usr/bin/FROGS-2.0.0/test/results/11-phylo_import.Rdata', ranks='Kingdom Phylum Class Order Family Genus Species'), intermediates_dir='/usr/bin/FROGS-2.0.0/test/results')" 2> /dev/null' returned non-zero exit status 1
Error in r_import_data
and
dunoyerd@ADNid:/usr/bin/FROGS-2.0.0/test/results$ cat 11-phylo_import.log
## Application
Software :/usr/bin/FROGS-2.0.0/app/r_import_data.py (version : 1.0.0)
Command : /usr/bin/FROGS-2.0.0/app/r_import_data.py --biomfile /usr/bin/FROGS-2.0.0/test/results/08-affiliation_std.biom --samplefile /usr/bin/FROGS-2.0.0/test/data/sample_metadata.tsv --treefile /usr/bin/FROGS-2.0.0/test/results/10b-tree.nwk --rdata /usr/bin/FROGS-2.0.0/test/results/11-phylo_import.Rdata --html /usr/bin/FROGS-2.0.0/test/results/11-phylo_import.html --log-file /usr/bin/FROGS-2.0.0/test/results/11-phylo_import.log
# Run r_import_data.Rmd (Rscript version : [1] "R version 3.4.3 (2017-11-30)"
[1] "Rmarkdown version: 1.6"
[1] "Phyloseq version: 1.22.3")
Command:
Rscript -e "rmarkdown::render('/usr/bin/FROGS-2.0.0/app/r_import_data.Rmd',output_file='/usr/bin/FROGS-2.0.0/test/results/11-phylo_import.html', params=list(biomfile='/usr/bin/FROGS-2.0.0/test/results/08-affiliation_std.biom', samplefile='/usr/bin/FROGS-2.0.0/test/data/sample_metadata.tsv', treefile='/usr/bin/FROGS-2.0.0/test/results/10b-tree.nwk', normalization=FALSE, outputRdata='/usr/bin/FROGS-2.0.0/test/results/11-phylo_import.Rdata', ranks='Kingdom Phylum Class Order Family Genus Species'), intermediates_dir='/usr/bin/FROGS-2.0.0/test/results')" 2> /dev/null
Execution:
start: 21 Mar 2018 12:06:46
[EDIT] the problem is no more in r_composition but in r_import_data
Sorry for the delay.
I install a new VM, with the exact same version of R, and associated Rpackage you indicated. The problem is not there, it's in the pandoc version you install (I guess).
I successfully reproduce your problem (r_composition problem), when using the last version of pandoc 2.1.3. In fact it's only a warning, the output file was correctly generating, but it return an exit code of 1 and FROGS crashes.....
If you install an older version of pandoc (I have 1.19.2), it works even with your newer version of R.
Could you test this ?
NB: if you have keep test results files, you can modify the test.sh script to avoid relaunch all tests ....
dpkg -R pandoc
wget https://github.com/jgm/pandoc/releases/download/1.19.2/pandoc-1.19.2-1-amd64.deb
sudo dpkg -i pandoc-1.19.2-1-amd64.deb
rm pandoc-1.19.2-1-amd64.deb
then:
dunoyerd@XXX:/usr/bin/FROGS-2.0.0/test$ bash test.sh /usr/bin/FROGS-2.0.0 4 4 /usr/bin/FROGS-2.0.0/test/results/
Step preprocess mercredi 21 mars 2018, 18:02:49 (UTC+0100)
Step clustering mercredi 21 mars 2018, 18:03:36 (UTC+0100)
Step remove_chimera mercredi 21 mars 2018, 18:04:15 (UTC+0100)
Step filters mercredi 21 mars 2018, 18:05:18 (UTC+0100)
Step affiliation_OTU mercredi 21 mars 2018, 18:05:18 (UTC+0100)
Step clusters_stat mercredi 21 mars 2018, 18:05:25 (UTC+0100)
Step affiliations_stat mercredi 21 mars 2018, 18:05:27 (UTC+0100)
Step biom_to_tsv mercredi 21 mars 2018, 18:05:37 (UTC+0100)
Step biom_to_stdBiom mercredi 21 mars 2018, 18:05:37 (UTC+0100)
Step tsv_to_biom mercredi 21 mars 2018, 18:05:37 (UTC+0100)
Step tree : pynast mercredi 21 mars 2018, 18:05:38 (UTC+0100)
Step tree : mafft mercredi 21 mars 2018, 18:09:03 (UTC+0100)
Step r_import_data mercredi 21 mars 2018, 18:09:49 (UTC+0100)
Step r_composition mercredi 21 mars 2018, 18:10:03 (UTC+0100)
Traceback (most recent call last):
File "/usr/bin/FROGS-2.0.0/app/r_composition.py", line 107, in <module>
Rscript(html, data, args.varExp, args.taxaRank1.strip(), str(taxaSet1.strip()), args.taxaRank2.strip(), args.numberOfTaxa).submit( args.log_file )
File "/usr/bin/FROGS-2.0.0/lib/frogsUtils.py", line 140, in submit
subprocess.check_output( self.get_cmd(), shell=True )
File "/usr/lib/python2.7/subprocess.py", line 574, in check_output
raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command 'Rscript -e "rmarkdown::render('/usr/bin/FROGS-2.0.0/app/r_composition.Rmd',output_file='/usr/bin/FROGS-2.0.0/test/results/12-phylo_composition.html', params=list(data='/usr/bin/FROGS-2.0.0/test/results/11-phylo_import.Rdata', varExp='Color',taxaRank1='Kingdom',taxaSet1='Bacteria',taxaRank2='Phylum',numberOfTaxa=9), intermediates_dir='/usr/bin/FROGS-2.0.0/test/results')" 2> /dev/null' returned non-zero exit status 1
Error in r_composition
sorry...
ok .....
I htink you are you french ? could we call each other ? you can send me privatly your phone at frogs@inra.fr
I'll trace back for other user, the problem ...
First of all, I really want to thank you for all you help...
Unfortunatly my OS went bad and... I reformated everything :s
But, now I am planning to re-install everything... in a Docker. I like Docker, and if I have a problem with a wrong version of a software, it will be easier to re-install everything. Did you do som tests on docker?
Regards,
Denis
PS: C'est vrai que nous pourrions parler en français, ça serait plus simple.
We tested docker some times ago, but, as it's not our priority it's not updated.
We have currently a collaboration to create a bioconda recipe, it will be finallized soon (command line user may already use it). FROGS will also be available soon in the Toolshed.
Keep us in touch, after installing the versions we recommend.
Best regards
hello everybody,
I'm trying to install FROGS for CLI usage but I have problems.
Here are the java exception I have:
I can't find any solution
here is my libexec folder:
Is it a Java problem?