test execution problems

geraldinepascal / FROGS

FROGS is a galaxy/CLI workflow designed to produce an OTUs count matrix from high depth sequencing amplicon data.

GNU General Public License v3.0

24 stars 22 forks source link

test execution problems #28

Closed Toliman06 closed 6 years ago

Toliman06 commented 6 years ago

hello everybody,

I'm trying to install FROGS for CLI usage but I have problems.

Here are the java exception I have:

Step preprocess lundi 19 mars 2018, 15:42:06 (UTC+0100)
Step clustering lundi 19 mars 2018, 15:42:58 (UTC+0100)
Step remove_chimera lundi 19 mars 2018, 15:43:44 (UTC+0100)
Step filters lundi 19 mars 2018, 15:44:58 (UTC+0100)
Step affiliation_OTU lundi 19 mars 2018, 15:44:59 (UTC+0100)
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/bin/FROGS-2.0.0//app/affiliation_OTU.py", line 191, in process_rdp
    rdp_cmd = RDPAffiliation(reference, input, output, memory)
  File "/usr/bin/FROGS-2.0.0//app/affiliation_OTU.py", line 87, in __init__
    which("classifier.jar"),
  File "/usr/bin/FROGS-2.0.0/lib/frogsUtils.py", line 44, in which
    raise Exception( "The software '" + exec_name + "' cannot be retrieved in path." )
Exception: The software 'classifier.jar' cannot be retrieved in path.

Traceback (most recent call last):
  File "/usr/bin/FROGS-2.0.0//app/affiliation_OTU.py", line 372, in <module>
    log_append_files(args.log_file, log_rdp_list + log_blast_list)
  File "/usr/bin/FROGS-2.0.0//app/affiliation_OTU.py", line 278, in log_append_files
    FH_input = open(current_file)
IOError: [Errno 2] No such file or directory: 'results/1521470699.34_2613_1521470699.34_2613_04-filters.fasta_0_rdp.log'
Error in affiliation_OTU

I can't find any solution

here is my libexec folder:

total 44780
drwxr-xr-x 2 dunoyerd dunoyerd     4096 mars  19 15:36 .
drwxr-xr-x 9 dunoyerd dunoyerd     4096 août   8  2017 ..
-rwxr-xr-x 1 dunoyerd dunoyerd    12133 août   8  2017 addAffiliation2biom.py
-rwxr-xr-x 1 dunoyerd dunoyerd    10702 août   8  2017 biom2tsv.py
-rwxr-xr-x 1 dunoyerd dunoyerd     3594 août   8  2017 biomFastaUpdate.py
-rwxr-xr-x 1 dunoyerd dunoyerd    26717 août   8  2017 biomTools.py
-rwxr-xr-x 1 root     root     42141104 mars   2 16:02 blastn
lrwxrwxrwx 1 dunoyerd dunoyerd       43 mars  19 15:36 classifier -> /home/dunoyerd/GIT_SOFT/RDPTools/classifier
-rwxr-xr-x 1 dunoyerd dunoyerd    16792 août   8  2017 derepSamples.py
-rwxr-xr-x 1 dunoyerd dunoyerd     3942 août   8  2017 extractSwarmsFasta.py
-rwxr-xr-x 1 root     root       409188 mars   2 17:14 FastTree
-rwxr-xr-x 1 dunoyerd dunoyerd     9910 août   8  2017 filterSeq.py
-rwxr-xr-x 1 root     root        74144 mars   2 14:48 flash
-rwxr-xr-x 1 dunoyerd dunoyerd     4393 août   8  2017 multiAffiFromBiom.py
-rwxr-xr-x 1 dunoyerd dunoyerd    28734 août   8  2017 parallelChimera.py
-rwxr-xr-x 1 dunoyerd dunoyerd    11012 août   8  2017 remove454Adapt.py
-rwxr-xr-x 1 dunoyerd dunoyerd    12833 août   8  2017 removeConta.py
-rwxr-xr-x 1 dunoyerd dunoyerd     1401 août   8  2017 root_tree.R
-rwxr-xr-x 1 dunoyerd dunoyerd    18685 août   8  2017 silva2retrainRDP.py
-rwxr-xr-x 1 dunoyerd dunoyerd     6525 août   8  2017 sortAbundancies.py
-rwxr-xr-x 1 dunoyerd dunoyerd    26705 août   8  2017 splitbc.pl
-rwxr-xr-x 1 root     root       461624 mars   2 14:59 swarm
-rwxr-xr-x 1 dunoyerd dunoyerd     5329 août   8  2017 swarm2biom.py
-rwxr-xr-x 1 root     root       409188 mars   2 17:13 tree
-rwxr-xr-x 1 dunoyerd dunoyerd    14040 août   8  2017 tsv2biom.py
-rwxr-xr-x 1 root     root      2089664 mars   2 15:11 vsearch

Is it a Java problem?

mariabernard commented 6 years ago

Hi,

This means that you did not install RDP Classifier or that the java jar file classifier.jar is not in your PATH (which is very classical as jar file are not binary file).

check that you installed RDP
if yes, the simplest things to do is simply to create a symbolic link of the classifier.jar file in the FROGS libexec directory

Does it solve the problem ?

mariabernard commented 6 years ago

oh no! sorry, I do not read well enough your message.

You installed RDP and you created the link, but it's not a explicitly a classifier.jar . Which version of RDP did you install ? We tested the version 2.0.2 which should contain after installation a classifier.jar file ....

Toliman06 commented 6 years ago

Oups... Sorry for the bad symbolic link...

I'm unsing the latest github version. I will try the 2.0.2...

Toliman06 commented 6 years ago

Good news! I can go through the affiliation_OTU step! ... but I have a new problem:

Step preprocess lundi 19 mars 2018, 16:06:38 (UTC+0100)
Step clustering lundi 19 mars 2018, 16:07:26 (UTC+0100)
Step remove_chimera lundi 19 mars 2018, 16:08:06 (UTC+0100)
Step filters lundi 19 mars 2018, 16:09:20 (UTC+0100)
Step affiliation_OTU lundi 19 mars 2018, 16:09:20 (UTC+0100)
Step clusters_stat lundi 19 mars 2018, 16:09:24 (UTC+0100)
Step affiliations_stat lundi 19 mars 2018, 16:09:26 (UTC+0100)
Step biom_to_tsv lundi 19 mars 2018, 16:09:36 (UTC+0100)
Step biom_to_stdBiom lundi 19 mars 2018, 16:09:36 (UTC+0100)
Step tsv_to_biom lundi 19 mars 2018, 16:09:36 (UTC+0100)
Step tree : pynast lundi 19 mars 2018, 16:09:36 (UTC+0100)
Traceback (most recent call last):
  File "/usr/bin/FROGS-2.0.0//app/tree.py", line 328, in <module>
    Pynast(args.input_otu, args.template_pynast, min_len, align, pynast_fail, pynast_log).submit( args.log_file )
  File "/usr/bin/FROGS-2.0.0/lib/frogsUtils.py", line 140, in submit
    subprocess.check_output( self.get_cmd(), shell=True )
  File "/usr/lib/python2.7/subprocess.py", line 574, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command 'pynast --input_fp results//04-filters.fasta --template_fp /usr/bin/FROGS-2.0.0//test/data/otus_pynast.fasta --min_len 349 --fasta_out results/04-filters_pynast_aligned.fasta --failure_fp results/04-filters_pynast_fail.fasta --log_fp results/04-filters_pynast_log.txt 2> /dev/null' returned non-zero exit status 1
Error in tree : pynast

[EDIT] and I'm using the 1.2.2 version of pynast

/usr/bin/FROGS-2.0.0/test$ pip freeze | grep pynast
pynast==1.2.2

mariabernard commented 6 years ago

Could you test manually

export PATH=[FROGS_DIR]/libexec:$PATH # change by your FROGS_DIR path!
pynast --version

ok ?

ls -l results//04-filters.fasta /usr/bin/FROGS-2.0.0//test/data/otus_pynast.fasta

Do they exist ?

pynast --input_fp results//04-filters.fasta --template_fp /usr/bin/FROGS-2.0.0//test/data/otus_pynast.fasta --min_len 349 --fasta_out results/04-filters_pynast_aligned.fasta --failure_fp results/04-filters_pynast_fail.fasta --log_fp results/04-filters_pynast_log.txt

Does it work ?

Toliman06 commented 6 years ago

changing the path variable:

dunoyerd@XXX:/usr/bin/FROGS-2.0.0/test$`
 export PATH=/usr/bin/FROGS-2.0.0/libexec:$PATH
dunoyerd@XXX:/usr/bin/FROGS-2.0.0/test$ pynast --version 
/usr/local/lib/python2.7/dist-packages/cogent/evolve/likelihood_tree.py:7: UserWarning: Not using MPI as mpi4py not found
  from cogent.util.parallel import MPI
Version: pynast 0.1

Verify that the files exist:

dunoyerd@XXX:/usr/bin/FROGS-2.0.0/test$ ls -l results/04-filters.fasta /usr/bin/FROGS-2.0.0//test/data/otus_pynast.fasta
-rw-rw-r-- 1 dunoyerd dunoyerd  668450 mars  20 09:11 results/04-filters.fasta
-rwxr-xr-x 1 dunoyerd dunoyerd 1922750 août   8  2017 /usr/bin/FROGS-2.0.0//test/data/otus_pynast.fasta

finally:

dunoyerd@XXX:/usr/bin/FROGS-2.0.0/test$ pynast --input_fp results//04-filters.fasta --template_fp /usr/bin/FROGS-2.0.0//test/data/otus_pynast.fasta --min_len 349 --fasta_out results/04-filters_pynast_aligned.fasta --failure_fp results/04-filters_pynast_fail.fasta --log_fp results/04-filters_pynast_log.txt
/usr/local/lib/python2.7/dist-packages/cogent/evolve/likelihood_tree.py:7: UserWarning: Not using MPI as mpi4py not found
  from cogent.util.parallel import MPI
Traceback (most recent call last):
  File "/usr/local/bin/pynast", line 194, in <module>
    main()
  File "/usr/local/bin/pynast", line 178, in main
    for seq, status in pynast_iterator:
  File "/usr/local/lib/python2.7/dist-packages/pynast/util.py", line 651, in ipynast_seqs
    current_result = pw_alignment_iterator.next()
  File "/usr/local/lib/python2.7/dist-packages/pynast/pycogent_backports/uclust.py", line 399, in uclust_search_and_align_from_fasta_filepath
    TmpDir=tmp_dir,HALT_EXEC=HALT_EXEC)
  File "/usr/local/lib/python2.7/dist-packages/cogent/app/util.py", line 166, in __init__
    self._error_on_missing_application(params)
  File "/usr/local/lib/python2.7/dist-packages/cogent/app/util.py", line 435, in _error_on_missing_application
    % command
cogent.app.util.ApplicationNotFoundError: Cannot find uclust. Is it installed? Is it in your path?

I seems there is a problem with uclust...

[EDIT] Befor you ask me...

dunoyerd@XXX:/usr/bin/FROGS-2.0.0/test$ locate uclust
/usr/local/lib/python2.7/dist-packages/cogent/app/uclust.py
/usr/local/lib/python2.7/dist-packages/cogent/app/uclust.pyc
/usr/local/lib/python2.7/dist-packages/pynast/pycogent_backports/uclust.py
/usr/local/lib/python2.7/dist-packages/pynast/pycogent_backports/uclust.pyc

mariabernard commented 6 years ago

Hi again,

Yes uclust is in my PATH! ( http://www.drive5.com/uclust/downloads1_2_22q.html ) it doest not seem to be your case!

uclust --version
uclust v1.2.22q

Normally the cogent warning doesn't cause any trouble because we redirect stderr to /dev/null in FROGS..... but pynast need to work!

Toliman06 commented 6 years ago

Thanks!

I installed it and it seems to work now (at least pynast --input_fp results//04-filters.fasta --template_fp /usr/bin/FROGS-2.0.0//test/data/otus_pynast.fasta --min_len 349 --fasta_out results/04-filters_pynast_aligned.fasta --failure_fp results/04-filters_pynast_fail.fasta --log_fp results/04-filters_pynast_log.txt does not crash)

I also installed mpi4py:

sudo apt install libopenmpi-dev
sudo pip install mpi4py

These programs are nor mentioned in the required dependencies list...

mariabernard commented 6 years ago

indeed, but mpi is not used, so the dependency is not necessary (and the warning message is redirected to /dev/null in FROGS so not a problem).

Could you confirm (when you finish) that you successfuly install all dependencies, so I could close this issue ?

Thank you any way to take time to test FROGS. (we will propose a conda/toolshed for the next release, at most for summer .....)

Toliman06 commented 6 years ago

Unfortunately, I have a new problem with pynast:

dunoyerd@ADNid:/usr/bin/FROGS-2.0.0/test$ bash test.sh /usr/bin/FROGS-2.0.0 4 4  results2
Step preprocess mardi 20 mars 2018, 13:09:01 (UTC+0100)
Step clustering mardi 20 mars 2018, 13:09:47 (UTC+0100)
Step remove_chimera mardi 20 mars 2018, 13:10:26 (UTC+0100)
Step filters mardi 20 mars 2018, 13:11:34 (UTC+0100)
Step affiliation_OTU mardi 20 mars 2018, 13:11:35 (UTC+0100)
Step clusters_stat mardi 20 mars 2018, 13:11:40 (UTC+0100)
Step affiliations_stat mardi 20 mars 2018, 13:11:41 (UTC+0100)
Step biom_to_tsv mardi 20 mars 2018, 13:11:51 (UTC+0100)
Step biom_to_stdBiom mardi 20 mars 2018, 13:11:51 (UTC+0100)
Step tsv_to_biom mardi 20 mars 2018, 13:11:51 (UTC+0100)
Step tree : pynast mardi 20 mars 2018, 13:11:51 (UTC+0100)
Traceback (most recent call last):
  File "/usr/bin/FROGS-2.0.0/app/tree.py", line 330, in <module>
    RootTree(fasttree, args.out_tree).submit(args.log_file)
  File "/usr/bin/FROGS-2.0.0/lib/frogsUtils.py", line 135, in submit
    FH_log.write( '# ' + self.description + ' (' + os.path.basename(self.program) + ' version : ' + self.get_version() + ')\n' )
  File "/usr/bin/FROGS-2.0.0/app/tree.py", line 147, in get_version
    return Cmd.get_version(self, 'stdout').split()[-1].strip()
IndexError: list index out of range
Error in tree : pynast

I seems to be an Index out of range problem...

pynast --version
Version: pynast 0.1
which pynast
/usr/local/bin/pynast

I checked with pip, this is the latest version installed. I downloaded package from https://pypi.python.org/pypi/pynast (as precognized) then build it and copy it on the previous pip installed version... The error still persist...

[EDIT] The "Step tree" takes 3-4 minutes before crashing...

mariabernard commented 6 years ago

Nop it's not a pynast error.

RootTree(fasttree, args.out_tree).submit(args.log_file)

RootTree is a python class which call the FROGS_DIR/libexec/root_tree.R

Do you install Rscript and link it in the libexec dir ?

Could you test

./libexec/root_tree.R -v
root_tree.R :  1.0.0

Toliman06 commented 6 years ago

dunoyerd@XXX:/usr/bin/FROGS-2.0.0$` ./libexec/root_tree.R -v
Error in library(phangorn) : aucun package nommé ‘phangorn’ n'est trouvé
Exécution arrêtée

then

sudo Rscript -e 'install.packages("phangorn", repos="https://cran.rstudio.com")'

(strange I already installed phangorn...)

dunoyerd@XXX:/usr/bin/FROGS-2.0.0$ ./libexec/root_tree.R -v
root_tree.R :  1.0.0

retry the test launch... (I also re-installed rmarkdown...)

dunoyerd@XXX:/usr/bin/FROGS-2.0.0/test$ bash test.sh /usr/bin/FROGS-2.0.0 4 4 /usr/bin/FROGS-2.0.0/test/results
Step preprocess mardi 20 mars 2018, 16:45:15 (UTC+0100)
Step clustering mardi 20 mars 2018, 16:46:02 (UTC+0100)
Step remove_chimera mardi 20 mars 2018, 16:46:41 (UTC+0100)
Step filters mardi 20 mars 2018, 16:47:49 (UTC+0100)
Step affiliation_OTU mardi 20 mars 2018, 16:47:50 (UTC+0100)
Step clusters_stat mardi 20 mars 2018, 16:47:53 (UTC+0100)
Step affiliations_stat mardi 20 mars 2018, 16:47:53 (UTC+0100)
Step biom_to_tsv mardi 20 mars 2018, 16:48:03 (UTC+0100)
Step biom_to_stdBiom mardi 20 mars 2018, 16:48:03 (UTC+0100)
Step tsv_to_biom mardi 20 mars 2018, 16:48:03 (UTC+0100)
Step tree : pynast mardi 20 mars 2018, 16:48:03 (UTC+0100)
Step tree : mafft mardi 20 mars 2018, 16:51:30 (UTC+0100)
Step r_import_data mardi 20 mars 2018, 16:52:16 (UTC+0100)
Step r_composition mardi 20 mars 2018, 16:52:32 (UTC+0100)
Traceback (most recent call last):
  File "/usr/bin/FROGS-2.0.0/app/r_composition.py", line 107, in <module>
    Rscript(html, data, args.varExp, args.taxaRank1.strip(), str(taxaSet1.strip()), args.taxaRank2.strip(), args.numberOfTaxa).submit( args.log_file )
  File "/usr/bin/FROGS-2.0.0/lib/frogsUtils.py", line 140, in submit
    subprocess.check_output( self.get_cmd(), shell=True )
  File "/usr/lib/python2.7/subprocess.py", line 574, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command 'Rscript -e "rmarkdown::render('/usr/bin/FROGS-2.0.0/app/r_composition.Rmd',output_file='/usr/bin/FROGS-2.0.0/test/results/12-phylo_composition.html', params=list(data='/usr/bin/FROGS-2.0.0/test/results/11-phylo_import.Rdata', varExp='Color',taxaRank1='Kingdom',taxaSet1='Bacteria',taxaRank2='Phylum',numberOfTaxa=9), intermediates_dir='/usr/bin/FROGS-2.0.0/test/results')" 2> /dev/null' returned non-zero exit status 1
Error in r_composition

I will try to be sure to install all the R libraries...

Toliman06 commented 6 years ago

One day, I will do it! I launch the faulty command. Here I have more information:

dunoyerd@XXX:/usr/bin/FROGS-2.0.0/test$ Rscript -e "rmarkdown::render('/usr/bin/FROGS-2.0.0/app/r_composition.Rmd',output_file='/usr/bin/FROGS-2.0.0/test/results/12-phylo_composition.html', params=list(data='/usr/bin/FROGS-2.0.0/test/results/11-phylo_import.Rdata', varExp='Color',taxaRank1='Kingdom',taxaSet1='Bacteria',taxaRank2='Phylum',numberOfTaxa=9), intermediates_dir='/usr/bin/FROGS-2.0.0/test/results')"
[WARNING] This document format requires a nonempty <title> element.
  Please specify either 'title' or 'pagetitle' in the metadata.
  Falling back to 'file2d21b0e37af.utf8'

processing file: r_composition.Rmd
  |....                                                             |   6%
  ordinary text without R code

  |........                                                         |  12%
label: setup (with options) 
List of 1
 $ include: logi FALSE

  |............                                                     |  19%
   inline R code fragments

  |................                                                 |  25%
label: unnamed-chunk-1 (with options) 
List of 3
 $ include: logi FALSE
 $ cache  : logi FALSE
 $ warning: logi FALSE

  |....................                                             |  31%
  ordinary text without R code

  |........................                                         |  38%
label: unnamed-chunk-2 (with options) 
List of 5
 $ fig.width : num 10
 $ fig.height: num 8
 $ echo      : logi FALSE
 $ comment   : logi NA
 $ cache     : logi FALSE

Quitting from lines 70-73 (r_composition.Rmd) 
Error in combine_vars(data, params$plot_env, vars, drop = params$drop) : 
  At least one layer must contain all variables used for facetting
Calls: <Anonymous> ... f -> <Anonymous> -> f -> <Anonymous> -> combine_vars

mariabernard commented 6 years ago

Hi!

We will do it!!

I can't reproduce the problem, so my guess is it's a version problem. Could you open the results/12-phylo_composition.log, and copy the R, rmarkdown and phyloseq versions ?

minimal requirement

# Run 1 code Rmarkdown (Rscript version : [1] "R version 3.4.0 (2017-04-21)"
[1] "Rmarkdown version:  1.5"
[1] "Phyloseq version:  1.20.0")

it also work for me with those versions

# Run 1 code Rmarkdown (Rscript version : [1] "R version 3.4.3 (2017-11-30)"
[1] "Rmarkdown version:  1.6"
[1] "Phyloseq version:  1.19.1")

Toliman06 commented 6 years ago

MMMhhhh... It seems I have a newer version of Rmarkdown and Phyloseq...

# Run 1 code Rmarkdown (Rscript version : [1] "R version 3.4.3 (2017-11-30)"
[1] "Rmarkdown version:  1.9"
[1] "Phyloseq version:  1.22.3")

Toliman06 commented 6 years ago

I resintalled Rmarkdown and relaunch analysis...

dunoyerd@XXX:/usr/bin/FROGS-2.0.0/test$ bash test.sh /usr/bin/FROGS-2.0.0 4 4 /usr/bin/FROGS-2.0.0/test/results
Step preprocess mercredi 21 mars 2018, 11:59:38 (UTC+0100)
Step clustering mercredi 21 mars 2018, 12:00:24 (UTC+0100)
Step remove_chimera mercredi 21 mars 2018, 12:01:02 (UTC+0100)
Step filters mercredi 21 mars 2018, 12:02:06 (UTC+0100)
Step affiliation_OTU mercredi 21 mars 2018, 12:02:06 (UTC+0100)
Step clusters_stat mercredi 21 mars 2018, 12:02:11 (UTC+0100)
Step affiliations_stat mercredi 21 mars 2018, 12:02:13 (UTC+0100)
Step biom_to_tsv mercredi 21 mars 2018, 12:02:23 (UTC+0100)
Step biom_to_stdBiom mercredi 21 mars 2018, 12:02:23 (UTC+0100)
Step tsv_to_biom mercredi 21 mars 2018, 12:02:23 (UTC+0100)
Step tree : pynast mercredi 21 mars 2018, 12:02:23 (UTC+0100)
Step tree : mafft mercredi 21 mars 2018, 12:05:51 (UTC+0100)
Step r_import_data mercredi 21 mars 2018, 12:06:41 (UTC+0100)
Traceback (most recent call last):
  File "/usr/bin/FROGS-2.0.0/app/r_import_data.py", line 117, in <module>
    Rscript(biomfile, samplefile, treefile, html, str(args.normalization).upper(), data, ranks).submit(args.log_file)
  File "/usr/bin/FROGS-2.0.0/lib/frogsUtils.py", line 140, in submit
    subprocess.check_output( self.get_cmd(), shell=True )
  File "/usr/lib/python2.7/subprocess.py", line 574, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command 'Rscript -e "rmarkdown::render('/usr/bin/FROGS-2.0.0/app/r_import_data.Rmd',output_file='/usr/bin/FROGS-2.0.0/test/results/11-phylo_import.html', params=list(biomfile='/usr/bin/FROGS-2.0.0/test/results/08-affiliation_std.biom', samplefile='/usr/bin/FROGS-2.0.0/test/data/sample_metadata.tsv', treefile='/usr/bin/FROGS-2.0.0/test/results/10b-tree.nwk', normalization=FALSE, outputRdata='/usr/bin/FROGS-2.0.0/test/results/11-phylo_import.Rdata', ranks='Kingdom Phylum Class Order Family Genus Species'), intermediates_dir='/usr/bin/FROGS-2.0.0/test/results')" 2> /dev/null' returned non-zero exit status 1
Error in r_import_data

and

dunoyerd@ADNid:/usr/bin/FROGS-2.0.0/test/results$ cat 11-phylo_import.log
## Application
Software :/usr/bin/FROGS-2.0.0/app/r_import_data.py (version : 1.0.0)
Command : /usr/bin/FROGS-2.0.0/app/r_import_data.py --biomfile /usr/bin/FROGS-2.0.0/test/results/08-affiliation_std.biom --samplefile /usr/bin/FROGS-2.0.0/test/data/sample_metadata.tsv --treefile /usr/bin/FROGS-2.0.0/test/results/10b-tree.nwk --rdata /usr/bin/FROGS-2.0.0/test/results/11-phylo_import.Rdata --html /usr/bin/FROGS-2.0.0/test/results/11-phylo_import.html --log-file /usr/bin/FROGS-2.0.0/test/results/11-phylo_import.log

# Run r_import_data.Rmd (Rscript version : [1] "R version 3.4.3 (2017-11-30)"
[1] "Rmarkdown version:  1.6"
[1] "Phyloseq version:  1.22.3")
Command:
    Rscript -e "rmarkdown::render('/usr/bin/FROGS-2.0.0/app/r_import_data.Rmd',output_file='/usr/bin/FROGS-2.0.0/test/results/11-phylo_import.html', params=list(biomfile='/usr/bin/FROGS-2.0.0/test/results/08-affiliation_std.biom', samplefile='/usr/bin/FROGS-2.0.0/test/data/sample_metadata.tsv', treefile='/usr/bin/FROGS-2.0.0/test/results/10b-tree.nwk', normalization=FALSE, outputRdata='/usr/bin/FROGS-2.0.0/test/results/11-phylo_import.Rdata', ranks='Kingdom Phylum Class Order Family Genus Species'), intermediates_dir='/usr/bin/FROGS-2.0.0/test/results')" 2> /dev/null

Execution:
    start: 21 Mar 2018 12:06:46

[EDIT] the problem is no more in r_composition but in r_import_data

mariabernard commented 6 years ago

Sorry for the delay.

I install a new VM, with the exact same version of R, and associated Rpackage you indicated. The problem is not there, it's in the pandoc version you install (I guess).

I successfully reproduce your problem (r_composition problem), when using the last version of pandoc 2.1.3. In fact it's only a warning, the output file was correctly generating, but it return an exit code of 1 and FROGS crashes.....

If you install an older version of pandoc (I have 1.19.2), it works even with your newer version of R.

Could you test this ?

NB: if you have keep test results files, you can modify the test.sh script to avoid relaunch all tests ....

Toliman06 commented 6 years ago

dpkg -R pandoc
wget https://github.com/jgm/pandoc/releases/download/1.19.2/pandoc-1.19.2-1-amd64.deb
sudo dpkg -i pandoc-1.19.2-1-amd64.deb
rm pandoc-1.19.2-1-amd64.deb

then:

dunoyerd@XXX:/usr/bin/FROGS-2.0.0/test$ bash test.sh /usr/bin/FROGS-2.0.0 4 4 /usr/bin/FROGS-2.0.0/test/results/
Step preprocess mercredi 21 mars 2018, 18:02:49 (UTC+0100)
Step clustering mercredi 21 mars 2018, 18:03:36 (UTC+0100)
Step remove_chimera mercredi 21 mars 2018, 18:04:15 (UTC+0100)
Step filters mercredi 21 mars 2018, 18:05:18 (UTC+0100)
Step affiliation_OTU mercredi 21 mars 2018, 18:05:18 (UTC+0100)
Step clusters_stat mercredi 21 mars 2018, 18:05:25 (UTC+0100)
Step affiliations_stat mercredi 21 mars 2018, 18:05:27 (UTC+0100)
Step biom_to_tsv mercredi 21 mars 2018, 18:05:37 (UTC+0100)
Step biom_to_stdBiom mercredi 21 mars 2018, 18:05:37 (UTC+0100)
Step tsv_to_biom mercredi 21 mars 2018, 18:05:37 (UTC+0100)
Step tree : pynast mercredi 21 mars 2018, 18:05:38 (UTC+0100)
Step tree : mafft mercredi 21 mars 2018, 18:09:03 (UTC+0100)
Step r_import_data mercredi 21 mars 2018, 18:09:49 (UTC+0100)
Step r_composition mercredi 21 mars 2018, 18:10:03 (UTC+0100)
Traceback (most recent call last):
  File "/usr/bin/FROGS-2.0.0/app/r_composition.py", line 107, in <module>
    Rscript(html, data, args.varExp, args.taxaRank1.strip(), str(taxaSet1.strip()), args.taxaRank2.strip(), args.numberOfTaxa).submit( args.log_file )
  File "/usr/bin/FROGS-2.0.0/lib/frogsUtils.py", line 140, in submit
    subprocess.check_output( self.get_cmd(), shell=True )
  File "/usr/lib/python2.7/subprocess.py", line 574, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command 'Rscript -e "rmarkdown::render('/usr/bin/FROGS-2.0.0/app/r_composition.Rmd',output_file='/usr/bin/FROGS-2.0.0/test/results/12-phylo_composition.html', params=list(data='/usr/bin/FROGS-2.0.0/test/results/11-phylo_import.Rdata', varExp='Color',taxaRank1='Kingdom',taxaSet1='Bacteria',taxaRank2='Phylum',numberOfTaxa=9), intermediates_dir='/usr/bin/FROGS-2.0.0/test/results')" 2> /dev/null' returned non-zero exit status 1
Error in r_composition

sorry...

mariabernard commented 6 years ago

ok .....

I htink you are you french ? could we call each other ? you can send me privatly your phone at frogs@inra.fr

I'll trace back for other user, the problem ...

Toliman06 commented 6 years ago

First of all, I really want to thank you for all you help...

Unfortunatly my OS went bad and... I reformated everything :s

But, now I am planning to re-install everything... in a Docker. I like Docker, and if I have a problem with a wrong version of a software, it will be easier to re-install everything. Did you do som tests on docker?

Regards,

Denis

PS: C'est vrai que nous pourrions parler en français, ça serait plus simple.

mariabernard commented 6 years ago

We tested docker some times ago, but, as it's not our priority it's not updated.

We have currently a collaboration to create a bioconda recipe, it will be finallized soon (command line user may already use it). FROGS will also be available soon in the Toolshed.

Keep us in touch, after installing the versions we recommend.

Best regards