search

marbl / metAMOS

A metagenomic and isolate assembly and analysis pipeline built with AMOS
http://marbl.github.io/metAMOS
Other
93 stars 45 forks source link

iMetAMOS failed to assemble with some read files without red error message but not all #227

Closed TakashiKoyama closed 9 years ago

TakashiKoyama commented 9 years ago

Hi,

Recently I tried to assemble several BAC clones one by one. All clones were sequenced by Hiseq2000. At first, I mapped all reads to BAC and E. coli genome sequences and resultant bam files consisting of unmapped reads were converted to fastq as follows;

# map fastq to BAC vector and E. coli genome
$ bwa mem -t $threads -M -R "@RG\tID:$sample\tSM:$sample\tPL:$platform" $reference $read1 $read2 > $outpath/$sample.sam

# extract mapped/unmapped reads
$ samtools view -bhf4 -@ $threadsminusone $outpath/$sample.sam \
| samtools sort -@ $threadsminusone -T $outpath/$sample.unmapped.sorted.tmp -o $outpath/$sample.unmapped.sorted.bam -
$ samtools view -bhF4 -@ $threadsminusone $outpath/$sample.sam \
| samtools sort -@ $threadsminusone -T $outpath/$sample.mapped.sorted.tmp -o $outpath/$sample.mapped.sorted.bam -

# estimate insert size
$ java -d64 -Xmx30g -jar $PICARDDIR/CollectInsertSizeMetrics.jar I=$outpath/$sample.mapped.sorted.bam H=$outpath/$sample.mapped.sorted.hist.pdf O=$outpath/$sample.mapped.sorted.hist.out

# convert unmapped bam to fastq
$ java -d64 -Xmx30g -jar $PICARDDIR/SamToFastq.jar I=$outpath/$sample.unmapped.sorted.bam F=$outpath/$sample.unmapped.sorted.R1.fastq F2=$outpath/$sample.unmapped.sorted.R2.fastq FU=$outpath/$sample.unmapped.sorted.UP.fastq VALIDATION_STRINGENCY=SILENT

# edit fastq header
$ sed -E 's/(^@.+)\/1$/\1/g' $outpath/$sample.unmapped.sorted.R1.fastq > $outpath/$sample.unmapped.sorted.R1.sed.fastq
$ sed -E 's/(^@.+)\/2$/\1/g' $outpath/$sample.unmapped.sorted.R2.fastq > $outpath/$sample.unmapped.sorted.R2.sed.fastq

# reduce number of reads
$ head -960000 $outpath/$sample.unmapped.sorted.R1.sed.fastq > $outpath/$sample.unmapped.sorted.R1.sed.head.fastq
$ head -960000 $outpath/$sample.unmapped.sorted.R2.sed.fastq > $outpath/$sample.unmapped.sorted.R2.sed.head.fastq

Then I tried assembly by iMetAMOS as follows;

$ readdir=/home/koyama/Documents/yellowtailBenQTL/seq/trimming
$ asmdir=/home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS
$ sample=013a02
$ cd $asmdir/$sample
$ initPipeline -q -1 $readdir/$sample/Hiseq2000/$sample.unmapped.sorted.R1.sed.head.fastq,$readdir/BACends/BACends.sed.fq -2 $readdir/$sample/Hiseq2000/$sample.unmapped.sorted.R2.sed.head.fastq -i 416:242 -d ./assembly1 -W iMetAMOS
$ runPipeline -d assembly1 -p 40 -m bowtie2 -a abyss,idba,sga,spades,masurca,velvet,newbler

I could assemble some clones but, strange to say, failed some clones without red error message. I am really curious why some clones could be assembled and some could not. As a matter of common sense, read processing should be the cause. I thought 'edit fastq header' edited unexpected line such as quality line started with '@' and ended with '\1' or '\2'. But I did not see such line.

$ grep -E "^@.+\/1$" 013a02.unmapped.sorted.R1.fastq | wc -l
30787553
$ wc -l 013a02.unmapped.sorted.R1.fastq
123150212
#30787553*4 = 123150212
$ grep -E "^@.+\/2$" 013a02.unmapped.sorted.R2.fastq | wc -l
30787553
$ wc -l 013a02.unmapped.sorted.R2.fastq 
123150212

So, the cause should be others but I have not still find. Could you give me any idea? Any suggestions would be really appreciated. I attached stdout from iMetAMOS below.

Project dir /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly1 successfully created!
Use runPipeline.py to start Pipeline
[Steps to be skipped]:  set(['FunctionalAnnotation', 'FindScaffoldORFS', 'FindRepeats'])
Starting metAMOS pipeline
Warning: MetaGeneMark is not found, some functionality will not be available
Warning: SignalP+ is not found, some functionality will not be available
Warning: PHmmer is not found, some functionality will not be available
[Available RAM: 56 GB]
        *ok
[Available CPUs: 40]
        *ok

________________________________________
Tasks which will be run:

Task = preprocess.Preprocess
Task = assemble.SplitAssemblers
Task = assemble.Assemble
Task = assemble.CheckAsmResults
Task = assemble.SplitMappers
Task = mapreads.MapReads
Task = mapreads.CheckMapResults
Task = mapreads.SplitForORFs
Task = findorfs.FindORFS
Task = validate.Validate
Task = findreps.FindRepeats
Task = annotate.Annotate
Task = fannotate.FunctionalAnnotation
Task = scaffold.Scaffold
Task = findscforfs.FindScaffoldORFS
Task = abundance.Abundance
Task = propagate.Propagate
Task = classify.Classify
Task = postprocess.Postprocess
________________________________________
metAMOS configuration summary:
metAMOS Version:        v1.5rc2 "Praline Brownie"  workflows: core,optional,imetamos
Time and Date:          2015-08-08
Working directory:      /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly1
Prefix:                 proba
K-Mer:                  31
Threads:                40
Taxonomic level:        genus
Verbose:                False
Steps to skip:          FunctionalAnnotation, FindScaffoldORFS, Scaffold, Propagate, FindRepeats
Steps to force: 

[citation]
MetAMOS
        Treangen, TJ ⇔  Koren, S, Sommer, DD, Liu, B, Astrovskaya, I, Ondov, B, Darling AE, Phillippy AM, Pop, M. MetAMOS: a modular and open source metagenomic assembly and analysis pipeline. Genome biology, 14(1), R2, 2013.

iMetAMOS
        Koren, S, Treangen, TJ, Hill, CM, Pop, M, Phillippy, AM. Automated ensemble assembly and validation of microbial genomes. BMC Bioinformatics 15:126, 2014.

Step-specific configuration:
[abundance]
MetaPhyler
        /home/samba/public/software/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64
        Liu B, Gibbons T, Ghodsi M, Treangen T, Pop M. Accurate and fast estimation of taxonomic profiles from metagenomic shotgun sequences. BMC Genomics. 2011;12 Suppl 2:S4. Epub 2011 Jul 27.

[multialign]
M-GCAT
        /home/samba/public/software/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64
        Treangen TJ, Messeguer X. M-GCAT: interactively and efficiently constructing large-scale multiple genome comparison frameworks in closely related species. BMC Bioinformatics, 2006.

[fannotate]
BLAST
        /home/samba/public/software/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64
        Altschul SF, Gish W, Miller W, Myers EW, Lipman DJ. Basic local alignment search tool. J Mol Biol. 1990 Oct 5;215(3):403-10.

[scaffold]
Bambus 2
        /home/samba/public/software/metAMOS-1.5rc3/AMOS/Linux-x86_64/bin
        Koren S, Treangen TJ, Pop M. Bambus 2: scaffolding metagenomes. Bioinformatics 27(21): 2964-2971 2011.

[findorfs]
Prokka
        /home/samba/public/software/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/prokka/bin
        Seemann, T. Prokka: rapid prokaryotic genome annotation. Bioinformatics, btu153. 2014.

[annotate]
Kraken
        /home/samba/public/software/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/kraken/bin
        Wood DE, Salzberg SL: Kraken: ultrafast metagenomic sequence classification using exact alignments. Genome Biology 2014, 15:R46.

[assemble]
ABySS
        /home/samba/public/software/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/abyss/bin
        Simpson, JT, Wong, K, Jackman, SD, Schein, JE, Jones, SJ, Birol, İ. ABySS: a parallel assembler for short read sequence data. Genome research, 19(6), 1117-1123, 2009.

Newbler
        /usr/local/bin
        NA

SPAdes
        /home/samba/public/software/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/spades/bin
        Anton Bankevich, Sergey Nurk, Dmitry Antipov, Alexey A. Gurevich, Mikhail Dvorkin, Alexander S. Kulikov, Valery M. Lesin, Sergey I. Nikolenko, Son Pham, Andrey D. Prjibelski, Alexey V. Pyshkin, Alexander V. Sirotkin, Nikolay Vyahhi, Glenn Tesler, Max A. Alekseyev, and Pavel A. Pevzner. Journal of Computational Biology. May 2012, 19(5): 455-477. doi:10.1089/cmb.2012.0021.

IDBA-UD
        /home/samba/public/software/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/idba/bin
        Peng, Y., Leung, H. C., Yiu, S. M., & Chin, F. Y. IDBA-UD: a de novo assembler for single-cell and metagenomic sequencing data with highly uneven depth. Bioinformatics, 28(11), 1420-1428, 2012.

SGA
        /home/samba/public/software/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/sga/bin
        Simpson, JT, Durbin, R. Efficient de novo assembly of large genomes using compressed data structures. Genome Research, 22(3), 549-556, 2012.

Velvet
        /home/samba/public/software/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/velvet
        Zerbino DR, Birney E. Velvet: algorithms for de novo short read assembly using de Bruijn graphs. Genome Res. 2008 May;18(5):821-9.

MaSuRCA
        /home/samba/public/software/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/MaSuRCA/bin
        Zimin, A, Marçais, G, Puiu, D, Roberts, M, Salzberg, SL, Yorke, JA. The MaSuRCA genome assembler. Bioinformatics, btt476, 2013.

[mapreads]
Bowtie2
        /home/samba/public/software/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64
        Langmead B, Salzberg SL. Fast gapped-read alignment with Bowtie 2. Nat Methods. 2012 Mar 4;9(4):357-9. doi: 10.1038/nmeth.1923.

[preprocess]
metAMOS built-in filtering
        N/A

[validate]
FRCbam
        /home/samba/public/software/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/FRCbam/bin
        Vezzi, F, Narzisi, G, Mishra, B. Reevaluating assembly evaluations with feature response curves: GAGE and assemblathons. PloS ONE, 7(12), e52210, 2013.

CGAL
        /home/samba/public/software/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/cgal
        Rahman, A, Pachter, L CGAL: computing genome assembly likelihoods. Genome biology, 14(1), R8, 2013.

ALE
        /home/samba/public/software/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/ALE/src
        Clark, SC, Egan, R, Frazier, PI, Wang, Z. ALE: a generic assembly likelihood evaluation framework for assessing the accuracy of genome and metagenome assemblies. Bioinformatics, 29(4) 435-443, 2013.

QUAST
        /home/samba/public/software/metAMOS-1.5rc3/quast
        Gurevich, A, Saveliev, V, Vyahhi, N, Tesler, G. QUAST: quality assessment tool for genome assemblies. Bioinformatics, 29(8), 1072-1075, 2013.

Prokka
        /home/samba/public/software/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/prokka/bin
        Seemann, T. Prokka: rapid prokaryotic genome annotation. Bioinformatics, btu153. 2014.

FreeBayes
        /home/samba/public/software/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/freebayes/bin
        Garrison, E, Marth, G. Haplotype-based variant detection from short-read sequencing. arXiv preprint arXiv:1207.3907, 2012.

LAP
        /home/samba/public/software/metAMOS-1.5rc3/LAP
        Ghodsi M, Hill CM, Astrovskaya I, Lin H, Sommer DD, Koren S, Pop M. De novo likelihood-based measures for comparing genome assemblies. BMC research notes 6:334, 2013.

REAPR
        /home/samba/public/software/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/REAPR
        Hunt, M, Kikuchi, T, Sanders, M, Newbold, C, Berriman, M, & Otto, TD. REAPR: a universal tool for genome assembly evaluation. Genome biology, 14(5), R47, 2013.

[other]
Krona
        /home/samba/public/software/metAMOS-1.5rc3/KronaTools/bin
        Ondov BD, Bergman NH, Phillippy AM. Interactive metagenomic visualization in a Web browser. BMC Bioinformatics. 2011 Sep 30;12:385.

KmerGenie
        /home/samba/public/software/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/kmergenie
        Chikhi, R, Medvedev, P. Informed and Automated k-Mer Size Selection for Genome Assembly. Bioinformatics btt310, 2013.

Starting Task = preprocess.PREPROCESS
    Job  = [[013a02.unmapped.sorted.R1.sed.head.fastq, 013a02.unmapped.sorted.R2.sed.head.fastq, BACends.sed.fq] -> preprocess.success] completed
Completed Task = preprocess.Preprocess
Starting Task = assemble.ASSEMBLE
*** metAMOS: Selected kmer size 95
*** metAMOS: Estimated genome size 92395 bp
    Job  = [preprocess.success -> *.run] completed
Completed Task = assemble.SplitAssemblers
    Job  = [abyss.95.run -> abyss.95.asm.contig] completed
    Job  = [idba-ud.95.run -> idba-ud.95.asm.contig] completed
    Job  = [masurca.95.run -> masurca.95.asm.contig] completed
    Job  = [newbler.95.run -> newbler.95.asm.contig] completed
    Job  = [sga.95.run -> sga.95.asm.contig] completed
    Job  = [spades.95.run -> spades.95.asm.contig] completed
    Job  = [velvet.95.run -> velvet.95.asm.contig] completed
Completed Task = assemble.Assemble
    Job  = [[abyss.95.asm.contig, idba-ud.95.asm.contig, masurca.95.asm.contig, newbler.95.asm.contig, sga.95.asm.contig, spades.95.asm.contig, velvet.95.asm.contig] -> [assemble.ok]] completed
Completed Task = assemble.CheckAsmResults
Uptodate Task = assemble.SplitMappers
Starting Task = mapreads.MAPREADS
    Job  = [abyss.95.asm.contig -> abyss.95.contig.cvg] completed
    Job  = [idba-ud.95.asm.contig -> idba-ud.95.contig.cvg] completed
    Job  = [masurca.95.asm.contig -> masurca.95.contig.cvg] completed
    Job  = [newbler.95.asm.contig -> newbler.95.contig.cvg] completed
    Job  = [sga.95.asm.contig -> sga.95.contig.cvg] completed
    Job  = [spades.95.asm.contig -> spades.95.contig.cvg] completed
    Job  = [velvet.95.asm.contig -> velvet.95.contig.cvg] completed
Completed Task = mapreads.MapReads
    Job  = [[abyss.95.contig.cvg, idba-ud.95.contig.cvg, masurca.95.contig.cvg, newbler.95.contig.cvg, sga.95.contig.cvg, spades.95.contig.cvg, velvet.95.contig.cvg] -> [mapreads.ok]] completed
Completed Task = mapreads.CheckMapResults
Uptodate Task = mapreads.SplitForORFs
Starting Task = findorfs.FINDORFS

    Job  = [abyss.95.contig.cvg -> abyss.95.faa] completed

    Job  = [idba-ud.95.contig.cvg -> idba-ud.95.faa] completed

    Job  = [masurca.95.contig.cvg -> masurca.95.faa] completed

    Job  = [newbler.95.contig.cvg -> newbler.95.faa] completed

    Job  = [sga.95.contig.cvg -> sga.95.faa] completed

    Job  = [spades.95.contig.cvg -> spades.95.faa] completed

    Job  = [velvet.95.contig.cvg -> velvet.95.faa] completed
Completed Task = findorfs.FindORFS
Starting Task = validate.VALIDATE
recruiting genomes..
done! recruited 4703 genomes!

    Job  = [[abyss.95.faa, idba-ud.95.faa, masurca.95.faa, newbler.95.faa, sga.95.faa, spades.95.faa, velvet.95.faa] -> [validate.ok]] completed
Completed Task = validate.Validate
Starting Task = findrepeats.FINDREPEATS
    Job  = [proba.fna -> proba.repeats] completed
Completed Task = findreps.FindRepeats
Starting Task = annotate.ANNOTATE
    Job  = [proba.faa -> proba.hits] completed
Completed Task = annotate.Annotate
Starting Task = functionalannotation.FUNCTIONALANNOTATION
    Job  = [proba.faa -> [blast.out, krona.ec.input]] completed
Completed Task = fannotate.FunctionalAnnotation
Starting Task = scaffold.SCAFFOLD
    Job  = [[proba.asm.contig] -> scaffold.ok] completed
Completed Task = scaffold.Scaffold
Oops, MetAMOS finished with errors! see text in red above for details.

Thanks, TKoyama

skoren commented 9 years ago

Most likely there was an error internal to metAMOS. You can try running runPipeline with the -v flag to enable verbose mode to get more details.

TakashiKoyama commented 9 years ago

Thanks for your comment. I did it and got following message. What does it mean and how can I fix this?

Oops, MetAMOS finished with errors! see text in red above for details.
Traceback (most recent call last):
  File "/home/samba/public/software/metAMOS-1.5rc3/runPipeline", line 984, in <module>
    verbose = 1)
  File "/home/samba/public/software/metAMOS-1.5rc3/Utilities/ruffus/task.py", line 2959, in pipeline_run
    fill_queue_with_job_parameters(job_parameters, parameter_q, multiprocess, logger, verbose)
  File "/home/samba/public/software/metAMOS-1.5rc3/Utilities/ruffus/task.py", line 2680, in fill_queue_with_job_parameters
    for param in job_parameters:
  File "/home/samba/public/software/metAMOS-1.5rc3/Utilities/ruffus/task.py", line 2619, in parameter_generator
    raise errt
RethrownJobError: 

    Exceptions generating parameters for

    def findscforfs.FindScaffoldORFS(...):

    Exception #1
      'ruffus.ruffus_exceptions.MissingInputFileError(    

        No way to run job: Input file ['/home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly1/Scaffold/out/proba.linearize.scaffolds.final'] does not exist)' raised in ...
       Task = def findscforfs.FindScaffoldORFS(...):

    Traceback (most recent call last):
      File "/home/samba/public/software/metAMOS-1.5rc3/Utilities/ruffus/task.py", line 2553, in parameter_generator
        check_input_files_exist (*param)
      File "/home/samba/public/software/metAMOS-1.5rc3/Utilities/ruffus/file_name_parameters.py", line 232, in check_input_files_exist
        "Input file ['%s'] does not exist" % f)
    MissingInputFileError:     

        No way to run job: Input file ['/home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly1/Scaffold/out/proba.linearize.scaffolds.final'] does not exist
TakashiKoyama commented 9 years ago

Hi, I noticed that iMetAMOS always fails when abyss is selected to be the best assembler.

*** metAMOS: Score LAP voted for assembler spades.71, weight 0.333333
*** metAMOS: Score ALE voted for assembler abyss.71, weight 0.333333
*** metAMOS: Score CGAL voted for assembler spades.71, weight 0.333333
*** metAMOS: Score SNP voted for assembler abyss.71, weight 1.000000
*** metAMOS: Score FRCBAM voted for assembler abyss.71, weight 1.000000
*** metAMOS: Score ORF voted for assembler spades.71, weight 0.000000
*** metAMOS: Score REAPR voted for assembler abyss.71, weight 1.000000
*** metAMOS: Score N50 voted for assembler spades.71, weight 1.000000
*** metAMOS assembler abyss.71 selected.
*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.asm.contig

*** metAMOS running command: mkdir -p /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Validate/out/recruit

recruiting genomes..
*** metAMOS running command: /home/samba/public/software/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/mgcat -M -r /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/abyss.71.asm.contig -d /home/samba/public/software/metAMOS-1.5rc3/Utilities/DB//refseq -o /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Validate/out/recruit -p 40

done! recruited 4729 genomes!
*** metAMOS running command: cat /home/samba/public/software/metAMOS-1.5rc3/Utilities/DB/refseq/Abaca_bunchy_top_virus_uid28697.fna |awk -F "|" '{if (match($1, ">")) { if (NF >= 2) { line=$1"_"$2; for (i=3; i<= NF; i++) { line=line"   "$i; } } else { line = $1; } print line; } else {print $0; }}' > /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Validate/out/proba.ref.fasta

*** metAMOS running command: /home/samba/public/software/metAMOS-1.5rc3/quast/metaquast.py --est-ref-size 96488 -o /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Validate/out/quast -R /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Validate/out/proba.ref.fasta -T 40 -l "abyss_71,idba-ud_71,masurca_71,newbler_71,sga_71,spades_71,velvet_71" /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/abyss.71.asm.contig /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/idba-ud.71.asm.contig /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/masurca.71.asm.contig /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/newbler.71.asm.contig /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/sga.71.asm.contig /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/spades.71.asm.contig /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/velvet.71.asm.contig

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.asm.contig

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/abyss.71.asm.contig /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.asm.contig

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.linearize.scaffolds.final

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/abyss.71.linearize.scaffolds.final /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.linearize.scaffolds.final

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.asm.tigr

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/abyss.71.asm.tigr /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.asm.tigr

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.contig.cnt

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/abyss.71.contig.cnt /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.contig.cnt

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.contig.cvg

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/abyss.71.contig.cvg /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.contig.cvg

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.afg

*** metAMOS running command: rm /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Logs/findorfs.ok

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/in/proba.asm.contig

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.asm.contig /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/in/proba.asm.contig

*** metAMOS running command: rm -rf /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba.prokka

*** metAMOS running command: /home/samba/public/software/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/prokka/bin/prokka  --outdir /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba.prokka --prefix proba --force /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/in/proba.asm.contig

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba.prokka/proba.ffn /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba.ctg.fna

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba.prokka/proba.faa /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba.ctg.faa

*** metAMOS running command: mv /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba.gene.cvg /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba.ctg.gene.cvg

*** metAMOS running command: mv /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba.gene.map /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba.ctg.gene.map

*** metAMOS running command: rm -r /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba.fna

*** metAMOS running command: cat /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba*.fna > /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba.fna

*** metAMOS running command: rm -r /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba.fna.bnk

*** metAMOS running command: /home/samba/public/software/metAMOS-1.5rc3/AMOS/Linux-x86_64/bin/toAmos_new -s /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba.fna -b /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba.fna.bnk

*** metAMOS running command: rm -r /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba.faa

*** metAMOS running command: cat /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba*.faa > /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba.faa

*** metAMOS running command: rm -r /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba.faa.bnk

*** metAMOS running command: /home/samba/public/software/metAMOS-1.5rc3/AMOS/Linux-x86_64/bin/toAmos_new -s /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba.faa -b /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba.faa.bnk

*** metAMOS running command: rm -r /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba.gene.cvg

*** metAMOS running command: cat /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba*.gene.cvg > /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba.gene.cvg

*** metAMOS running command: rm -r /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba.gene.map

*** metAMOS running command: cat /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba*.gene.map > /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba.gene.map

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba.faa /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.faa

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba.fna /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.fna

*** metAMOS running command: touch /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Logs/findorfs.ok

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.faa

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba.faa /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.faa

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindRepeats/in/proba.fna

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba.fna /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindRepeats/in/proba.fna

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindRepeats/in/proba.faa

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba.faa /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindRepeats/in/proba.faa

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/in/proba.fna

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba.fna /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/in/proba.fna

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/in/proba.faa

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FindORFS/out/proba.faa /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/in/proba.faa

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.lib1.badmates

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/abyss.71.lib1.badmates /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.lib1.badmates

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.lib1.hdr

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/abyss.71.lib1.hdr /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.lib1.hdr

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.lib1.mappedmates

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/abyss.71.lib1.mappedmates /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.lib1.mappedmates

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.lib1.mates_in_diff_contigs

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/abyss.71.lib1.mates_in_diff_contigs /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.lib1.mates_in_diff_contigs

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.lib1contig.reads

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/abyss.71.lib1contig.reads /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.lib1contig.reads

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/lib1.unaligned.fasta

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/abyss.71.lib1.unaligned.fasta /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/lib1.unaligned.fasta

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/lib1.unaligned.fastq

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.lib2.badmates

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/abyss.71.lib2.badmates /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.lib2.badmates

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.lib2.hdr

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/abyss.71.lib2.hdr /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.lib2.hdr

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.lib2.mappedmates

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/abyss.71.lib2.mappedmates /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.lib2.mappedmates

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.lib2.mates_in_diff_contigs

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/abyss.71.lib2.mates_in_diff_contigs /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.lib2.mates_in_diff_contigs

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.lib2contig.reads

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/abyss.71.lib2contig.reads /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.lib2contig.reads

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/lib2.unaligned.fasta

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/abyss.71.lib2.unaligned.fasta /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/lib2.unaligned.fasta

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/lib2.unaligned.fastq

    Job  = [[abyss.71.faa, idba-ud.71.faa, masurca.71.faa, newbler.71.faa, sga.71.faa, spades.71.faa, velvet.71.faa] -> [validate.ok]] completed
Completed Task = validate.Validate
Starting Task = findrepeats.FINDREPEATS
*** metAMOS running command: touch /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Logs/findrepeats.skip

    Job  = [proba.fna -> proba.repeats] completed
Completed Task = findreps.FindRepeats
Starting Task = annotate.ANNOTATE
*** metAMOS running command: touch /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/out/proba.annots

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/in/proba.asm.contig

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.asm.contig /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/in/proba.asm.contig

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/out/proba.hits

*** metAMOS running command: rm -f /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/out/*.hits

*** metAMOS running command: rm -f /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/out/*.epsilon-nb_results.txt

*** metAMOS running command: rm -f /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/out/*.phymm.out

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/lib1.unaligned.fasta /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/in/lib1.unaligned.fasta

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/lib2.unaligned.fasta /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/in/lib2.unaligned.fasta

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/lib1.unaligned.fasta /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/in/lib1.unaligned.fasta

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/lib2.unaligned.fasta /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/in/lib2.unaligned.fasta

*** metAMOS running command: /home/samba/public/software/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/kraken/bin/kraken --threads 40 -db /home/samba/public/software/metAMOS-1.5rc3/Utilities/DB//kraken /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/in/proba.asm.contig --preload   | kraken-filter --db /home/samba/public/software/metAMOS-1.5rc3/Utilities/DB//kraken --threshold 0.05 > /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/out/proba.intermediate.ctg.hits

*** metAMOS running command: /home/samba/public/software/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/kraken/bin/kraken --threads 40 -db /home/samba/public/software/metAMOS-1.5rc3/Utilities/DB//kraken /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/in/lib1.unaligned.fasta --preload   | kraken-filter --db /home/samba/public/software/metAMOS-1.5rc3/Utilities/DB//kraken --threshold 0.05 > /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/out/proba.intermediate.lib1.hits

*** metAMOS running command: /home/samba/public/software/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/kraken/bin/kraken --threads 40 -db /home/samba/public/software/metAMOS-1.5rc3/Utilities/DB//kraken /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/in/lib2.unaligned.fasta --preload   | kraken-filter --db /home/samba/public/software/metAMOS-1.5rc3/Utilities/DB//kraken --threshold 0.05 > /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/out/proba.intermediate.lib2.hits

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/out/proba.hits

*** metAMOS running command: cat /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/out/proba.intermediate.ctg.hits >> /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/out/proba.hits

*** metAMOS running command: cat /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/out/proba.intermediate.lib1.hits >> /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/out/proba.hits

*** metAMOS running command: cat /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/out/proba.intermediate.lib2.hits >> /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/out/proba.hits

*** metAMOS running command: perl /home/samba/public/software/metAMOS-1.5rc3/Utilities/perl/ImportKraken.pl  -c -i -f /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/in/proba.asm.contig:/home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/lib1.unaligned.fasta:/home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/lib2.unaligned.fasta /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/out/proba.hits:/home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.contig.cnt:genus

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Postprocess/in/proba.hits

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Postprocess/out/proba.hits

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/out/proba.hits /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Postprocess/in/proba.hits

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Annotate/out/proba.hits /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Postprocess/out/proba.hits

    Job  = [proba.faa -> proba.hits] completed
Completed Task = annotate.Annotate
Starting Task = functionalannotation.FUNCTIONALANNOTATION
*** metAMOS running command: touch /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Logs/functionalannotation.skip

*** metAMOS running command: touch /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/FunctionalAnnotation/out/blast.out

    Job  = [proba.faa -> [blast.out, krona.ec.input]] completed
Completed Task = fannotate.FunctionalAnnotation
Starting Task = scaffold.SCAFFOLD
*** metAMOS running command: rm -rf /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Scaffold/in/proba.bnk

*** metAMOS running command: /home/samba/public/software/metAMOS-1.5rc3/AMOS/Linux-x86_64/bin/toAmos_new -Q /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Preprocess/out/lib1.seq -i --min 6 --max 8080 --libname lib1 -b /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Scaffold/in/proba.bnk 

*** metAMOS running command: /home/samba/public/software/metAMOS-1.5rc3/AMOS/Linux-x86_64/bin/toAmos_new -Q /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Preprocess/out/lib2.seq  -b /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Scaffold/in/proba.bnk 

*** metAMOS running command: /home/samba/public/software/metAMOS-1.5rc3/AMOS/Linux-x86_64/bin/toAmos_new -c /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.asm.tigr -b /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Scaffold/in/proba.bnk 

*** metAMOS running command: /home/samba/public/software/metAMOS-1.5rc3/AMOS/Linux-x86_64/bin/bank2fasta -eid -b /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Scaffold/in/proba.bnk > /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Scaffold/out/proba.contigs

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.linearize.scaffolds.final /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Scaffold/out/proba.linearize.scaffolds.final

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Scaffold/out/proba.linearize.scaffolds.final /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Scaffold/out/proba.scaffolds.final

*** metAMOS running command: touch /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Scaffold/out/proba.motifs

*** metAMOS running command: touch /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Logs/scaffold.skip

    Job  = [[proba.asm.contig] -> scaffold.ok] completed
Completed Task = scaffold.Scaffold
Oops, MetAMOS finished with errors! see text in red above for details.
Traceback (most recent call last):
  File "/home/samba/public/software/metAMOS-1.5rc3/runPipeline", line 984, in <module>
    verbose = 1)
  File "/home/samba/public/software/metAMOS-1.5rc3/Utilities/ruffus/task.py", line 2959, in pipeline_run
    fill_queue_with_job_parameters(job_parameters, parameter_q, multiprocess, logger, verbose)
  File "/home/samba/public/software/metAMOS-1.5rc3/Utilities/ruffus/task.py", line 2680, in fill_queue_with_job_parameters
    for param in job_parameters:
  File "/home/samba/public/software/metAMOS-1.5rc3/Utilities/ruffus/task.py", line 2619, in parameter_generator
    raise errt
RethrownJobError: 

    Exceptions generating parameters for

    def findscforfs.FindScaffoldORFS(...):

    Exception #1
      'ruffus.ruffus_exceptions.MissingInputFileError(    

        No way to run job: Input file ['/home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Scaffold/out/proba.linearize.scaffolds.final'] does not exist)' raised in ...
       Task = def findscforfs.FindScaffoldORFS(...):

    Traceback (most recent call last):
      File "/home/samba/public/software/metAMOS-1.5rc3/Utilities/ruffus/task.py", line 2553, in parameter_generator
        check_input_files_exist (*param)
      File "/home/samba/public/software/metAMOS-1.5rc3/Utilities/ruffus/file_name_parameters.py", line 232, in check_input_files_exist
        "Input file ['%s'] does not exist" % f)
    MissingInputFileError:     

        No way to run job: Input file ['/home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Scaffold/out/proba.linearize.scaffolds.final'] does not exist

I traced 'proba.linearize.scaffolds.final' in the log to identify in which step the file was lost. The first proba.linearize.scaffolds.final could be opened.

$ less /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Assemble/out/proba.linearize.scaffolds.final
# the file could be opened

The second proba.linearize.scaffolds.final could not be opened.

$ less /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Scaffold/out/proba.linearize.scaffolds.final
/home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/070j15/assembly1/Scaffold/out/proba.linearize.scaffolds.final: No such file or directory

Why the link is lost from /Scaffold/out/proba.linearize.scaffolds.final?

skoren commented 9 years ago

Hi,

I think the issue is that the Scaffold/out/proba.linearize.scaffolds.final file is incorrectly linked by an earlier step. Then, the Scaffold step does not try to remove the bad link, causing the error. Can you try modifying your src/scaffold.py lines 88-91 from:

      if os.path.exists("%s/Assemble/out/%s.linearize.scaffolds.final"%(_settings.rundir, _settings.PREFIX)):
         run_process(_settings, "ln %s/Assemble/out/%s.linearize.scaffolds.final %s/Scaffold/out/%s.linearize.scaffolds.final"%(_settings.rundir, _settings.PREFIX, _settings.rundir, _settings.PREFIX), "Scaffold")
      else:
         run_process(_settings, "ln %s/Assemble/out/%s.asm.contig %s/Scaffold/out/%s.linearize.scaffolds.final"%(_settings.rundir, _settings.PREFIX, _settings.rundir, _settings.PREFIX), "Scaffold")

to 

      if os.path.exists("%s/Assemble/out/%s.linearize.scaffolds.final"%(_settings.rundir, _settings.PREFIX)):
         run_process(_settings, "unlink  %s/Scaffold/out/%s.linearize.scaffolds.final"%(_settings.rundir, _settings.PREFIX), "Scaffold")
         run_process(_settings, "ln %s/Assemble/out/%s.linearize.scaffolds.final %s/Scaffold/out/%s.linearize.scaffolds.final"%(_settings.rundir, _settings.PREFIX, _settings.rundir, _settings.PREFIX), "Scaffold")
      else:
         run_process(_settings, "unlink  %s/Scaffold/out/%s.linearize.scaffolds.final"%(_settings.rundir, _settings.PREFIX), "Scaffold")
         run_process(_settings, "ln %s/Assemble/out/%s.asm.contig %s/Scaffold/out/%s.linearize.scaffolds.final"%(_settings.rundir, _settings.PREFIX, _settings.rundir, _settings.PREFIX), "Scaffold")

and running the test again from scratch to see if the issue is resolved?

TakashiKoyama commented 9 years ago

Hi,

Thank you for your kind helps. I modified src/scaffold.py according to your comment but got same message. Here is the message I got.

*** metAMOS assembler velvet.95 scores:  LAP:-18.1966535239 ALE:-22134462.484049 CGAL:-2.23732e+07 SNP:-12 FRCBAM:-435 ORF:57 REAPR:7.16 N50:8162
*** metAMOS: Score LAP voted for assembler abyss.95, weight 0.333333
*** metAMOS: Score ALE voted for assembler abyss.95, weight 0.333333
*** metAMOS: Score CGAL voted for assembler abyss.95, weight 0.333333
*** metAMOS: Score SNP voted for assembler abyss.95, weight 1.000000
*** metAMOS: Score FRCBAM voted for assembler abyss.95, weight 1.000000
*** metAMOS: Score ORF voted for assembler abyss.95, weight 0.000000
*** metAMOS: Score REAPR voted for assembler abyss.95, weight 1.000000
*** metAMOS: Score N50 voted for assembler abyss.95, weight 1.000000
*** metAMOS assembler abyss.95 selected.
*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.asm.contig

*** metAMOS running command: mkdir -p /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Validate/out/recruit

recruiting genomes..
*** metAMOS running command: /home/samba/public/software/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/mgcat -M -r /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95.asm.contig -d /home/samba/public/software/metAMOS-1.5rc3/Utilities/DB//refseq -o /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Validate/out/recruit -p 40

done! recruited 4709 genomes!
*** metAMOS running command: cat /home/samba/public/software/metAMOS-1.5rc3/Utilities/DB/refseq/Abaca_bunchy_top_virus_uid28697.fna |awk -F "|" '{if (match($1, ">")) { if (NF >= 2) { line=$1"_"$2; for (i=3; i<= NF; i++) { line=line"    "$i; } } else { line = $1; } print line; } else {print $0; }}' > /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Validate/out/proba.ref.fasta

*** metAMOS running command: /home/samba/public/software/metAMOS-1.5rc3/quast/metaquast.py --est-ref-size 92826 -o /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Validate/out/quast -R /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Validate/out/proba.ref.fasta -T 40 -l "abyss_95,idba-ud_95,masurca_95,newbler_95,sga_95,spades_95,velvet_95" /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95.asm.contig /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/idba-ud.95.asm.contig /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/masurca.95.asm.contig /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/newbler.95.asm.contig /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/sga.95.asm.contig /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/spades.95.asm.contig /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/velvet.95.asm.contig

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.asm.contig

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95.asm.contig /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.asm.contig

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.linearize.scaffolds.final

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95.linearize.scaffolds.final /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.linearize.scaffolds.final

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.asm.tigr

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95.asm.tigr /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.asm.tigr

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.contig.cnt

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95.contig.cnt /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.contig.cnt

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.contig.cvg

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95.contig.cvg /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.contig.cvg

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.afg

*** metAMOS running command: rm /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Logs/findorfs.ok

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/in/proba.asm.contig

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.asm.contig /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/in/proba.asm.contig

*** metAMOS running command: rm -rf /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba.prokka

*** metAMOS running command: /home/samba/public/software/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/prokka/bin/prokka  --outdir /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba.prokka --prefix proba --force /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/in/proba.asm.contig

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba.prokka/proba.ffn /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba.ctg.fna

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba.prokka/proba.faa /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba.ctg.faa

*** metAMOS running command: mv /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba.gene.cvg /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba.ctg.gene.cvg

*** metAMOS running command: mv /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba.gene.map /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba.ctg.gene.map

*** metAMOS running command: rm -r /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba.fna

*** metAMOS running command: cat /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba*.fna > /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba.fna

*** metAMOS running command: rm -r /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba.fna.bnk

*** metAMOS running command: /home/samba/public/software/metAMOS-1.5rc3/AMOS/Linux-x86_64/bin/toAmos_new -s /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba.fna -b /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba.fna.bnk

*** metAMOS running command: rm -r /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba.faa

*** metAMOS running command: cat /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba*.faa > /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba.faa

*** metAMOS running command: rm -r /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba.faa.bnk

*** metAMOS running command: /home/samba/public/software/metAMOS-1.5rc3/AMOS/Linux-x86_64/bin/toAmos_new -s /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba.faa -b /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba.faa.bnk

*** metAMOS running command: rm -r /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba.gene.cvg

*** metAMOS running command: cat /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba*.gene.cvg > /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba.gene.cvg

*** metAMOS running command: rm -r /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba.gene.map

*** metAMOS running command: cat /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba*.gene.map > /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba.gene.map

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba.faa /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.faa

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba.fna /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.fna

*** metAMOS running command: touch /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Logs/findorfs.ok

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.faa

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba.faa /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.faa

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindRepeats/in/proba.fna

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba.fna /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindRepeats/in/proba.fna

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindRepeats/in/proba.faa

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba.faa /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindRepeats/in/proba.faa

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/in/proba.fna

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba.fna /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/in/proba.fna

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/in/proba.faa

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FindORFS/out/proba.faa /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/in/proba.faa

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.lib1.badmates

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95.lib1.badmates /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.lib1.badmates

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.lib1.hdr

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95.lib1.hdr /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.lib1.hdr

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.lib1.mappedmates

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95.lib1.mappedmates /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.lib1.mappedmates

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.lib1.mates_in_diff_contigs

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95.lib1.mates_in_diff_contigs /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.lib1.mates_in_diff_contigs

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.lib1contig.reads

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95.lib1contig.reads /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.lib1contig.reads

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/lib1.unaligned.fasta

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95.lib1.unaligned.fasta /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/lib1.unaligned.fasta

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/lib1.unaligned.fastq

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.lib2.badmates

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95.lib2.badmates /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.lib2.badmates

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.lib2.hdr

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95.lib2.hdr /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.lib2.hdr

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.lib2.mappedmates

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95.lib2.mappedmates /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.lib2.mappedmates

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.lib2.mates_in_diff_contigs

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95.lib2.mates_in_diff_contigs /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.lib2.mates_in_diff_contigs

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.lib2contig.reads

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95.lib2contig.reads /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.lib2contig.reads

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/lib2.unaligned.fasta

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95.lib2.unaligned.fasta /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/lib2.unaligned.fasta

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/lib2.unaligned.fastq

    Job  = [[abyss.95.faa, idba-ud.95.faa, masurca.95.faa, newbler.95.faa, sga.95.faa, spades.95.faa, velvet.95.faa] -> [validate.ok]] completed
Completed Task = validate.Validate
Starting Task = findrepeats.FINDREPEATS
*** metAMOS running command: touch /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Logs/findrepeats.skip

    Job  = [proba.fna -> proba.repeats] completed
Completed Task = findreps.FindRepeats
Starting Task = annotate.ANNOTATE
*** metAMOS running command: touch /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/out/proba.annots

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/in/proba.asm.contig

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.asm.contig /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/in/proba.asm.contig

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/out/proba.hits

*** metAMOS running command: rm -f /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/out/*.hits

*** metAMOS running command: rm -f /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/out/*.epsilon-nb_results.txt

*** metAMOS running command: rm -f /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/out/*.phymm.out

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/lib1.unaligned.fasta /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/in/lib1.unaligned.fasta

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/lib2.unaligned.fasta /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/in/lib2.unaligned.fasta

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/lib1.unaligned.fasta /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/in/lib1.unaligned.fasta

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/lib2.unaligned.fasta /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/in/lib2.unaligned.fasta

*** metAMOS running command: /home/samba/public/software/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/kraken/bin/kraken --threads 40 -db /home/samba/public/software/metAMOS-1.5rc3/Utilities/DB//kraken /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/in/proba.asm.contig --preload   | kraken-filter --db /home/samba/public/software/metAMOS-1.5rc3/Utilities/DB//kraken --threshold 0.05 > /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/out/proba.intermediate.ctg.hits

*** metAMOS running command: /home/samba/public/software/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/kraken/bin/kraken --threads 40 -db /home/samba/public/software/metAMOS-1.5rc3/Utilities/DB//kraken /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/in/lib1.unaligned.fasta --preload   | kraken-filter --db /home/samba/public/software/metAMOS-1.5rc3/Utilities/DB//kraken --threshold 0.05 > /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/out/proba.intermediate.lib1.hits

*** metAMOS running command: /home/samba/public/software/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/kraken/bin/kraken --threads 40 -db /home/samba/public/software/metAMOS-1.5rc3/Utilities/DB//kraken /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/in/lib2.unaligned.fasta --preload   | kraken-filter --db /home/samba/public/software/metAMOS-1.5rc3/Utilities/DB//kraken --threshold 0.05 > /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/out/proba.intermediate.lib2.hits

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/out/proba.hits

*** metAMOS running command: cat /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/out/proba.intermediate.ctg.hits >> /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/out/proba.hits

*** metAMOS running command: cat /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/out/proba.intermediate.lib1.hits >> /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/out/proba.hits

*** metAMOS running command: cat /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/out/proba.intermediate.lib2.hits >> /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/out/proba.hits

*** metAMOS running command: perl /home/samba/public/software/metAMOS-1.5rc3/Utilities/perl/ImportKraken.pl  -c -i -f /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/in/proba.asm.contig:/home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/lib1.unaligned.fasta:/home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/lib2.unaligned.fasta /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/out/proba.hits:/home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.contig.cnt:genus

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Postprocess/in/proba.hits

*** metAMOS running command: unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Postprocess/out/proba.hits

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/out/proba.hits /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Postprocess/in/proba.hits

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Annotate/out/proba.hits /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Postprocess/out/proba.hits

    Job  = [proba.faa -> proba.hits] completed
Completed Task = annotate.Annotate
Starting Task = functionalannotation.FUNCTIONALANNOTATION
*** metAMOS running command: touch /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Logs/functionalannotation.skip

*** metAMOS running command: touch /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/FunctionalAnnotation/out/blast.out

    Job  = [proba.faa -> [blast.out, krona.ec.input]] completed
Completed Task = fannotate.FunctionalAnnotation
Starting Task = scaffold.SCAFFOLD
*** metAMOS running command: rm -rf /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Scaffold/in/proba.bnk

*** metAMOS running command: /home/samba/public/software/metAMOS-1.5rc3/AMOS/Linux-x86_64/bin/toAmos_new -Q /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/lib1.seq -i --min 1 --max 14873 --libname lib1 -b /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Scaffold/in/proba.bnk 

*** metAMOS running command: /home/samba/public/software/metAMOS-1.5rc3/AMOS/Linux-x86_64/bin/toAmos_new -Q /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/lib2.seq  -b /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Scaffold/in/proba.bnk 

*** metAMOS running command: /home/samba/public/software/metAMOS-1.5rc3/AMOS/Linux-x86_64/bin/toAmos_new -c /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.asm.tigr -b /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Scaffold/in/proba.bnk 

*** metAMOS running command: /home/samba/public/software/metAMOS-1.5rc3/AMOS/Linux-x86_64/bin/bank2fasta -eid -b /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Scaffold/in/proba.bnk > /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Scaffold/out/proba.contigs

*** metAMOS running command: unlink  /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Scaffold/out/proba.linearize.scaffolds.final

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.linearize.scaffolds.final /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Scaffold/out/proba.linearize.scaffolds.final

*** metAMOS running command: ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Scaffold/out/proba.linearize.scaffolds.final /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Scaffold/out/proba.scaffolds.final

*** metAMOS running command: touch /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Scaffold/out/proba.motifs

*** metAMOS running command: touch /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Logs/scaffold.skip

    Job  = [[proba.asm.contig] -> scaffold.ok] completed
Completed Task = scaffold.Scaffold
Oops, MetAMOS finished with errors! see text in red above for details.
Traceback (most recent call last):
  File "/home/samba/public/software/metAMOS-1.5rc3/runPipeline", line 984, in <module>
    verbose = 1)
  File "/home/samba/public/software/metAMOS-1.5rc3/Utilities/ruffus/task.py", line 2959, in pipeline_run
    fill_queue_with_job_parameters(job_parameters, parameter_q, multiprocess, logger, verbose)
  File "/home/samba/public/software/metAMOS-1.5rc3/Utilities/ruffus/task.py", line 2680, in fill_queue_with_job_parameters
    for param in job_parameters:
  File "/home/samba/public/software/metAMOS-1.5rc3/Utilities/ruffus/task.py", line 2619, in parameter_generator
    raise errt
RethrownJobError: 

    Exceptions generating parameters for

    def findscforfs.FindScaffoldORFS(...):

    Exception #1
      'ruffus.ruffus_exceptions.MissingInputFileError(    

        No way to run job: Input file ['/home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Scaffold/out/proba.linearize.scaffolds.final'] does not exist)' raised in ...
       Task = def findscforfs.FindScaffoldORFS(...):

    Traceback (most recent call last):
      File "/home/samba/public/software/metAMOS-1.5rc3/Utilities/ruffus/task.py", line 2553, in parameter_generator
        check_input_files_exist (*param)
      File "/home/samba/public/software/metAMOS-1.5rc3/Utilities/ruffus/file_name_parameters.py", line 232, in check_input_files_exist
        "Input file ['%s'] does not exist" % f)
    MissingInputFileError:     

        No way to run job: Input file ['/home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Scaffold/out/proba.linearize.scaffolds.final'] does not exist
skoren commented 9 years ago

Is this a new clean run? Can you run

ls -la /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95.linearize.scaffolds.final
ls -la /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.linearize.scaffolds.final
ls -la /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Scaffold/out/proba.linearize.scaffolds.final 

head /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95.linearize.scaffolds.final
head /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.linearize.scaffolds.final
head /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Scaffold/out/proba.linearize.scaffolds.final
TakashiKoyama commented 9 years ago

Hi,

Yes, this is new run. Scaffold/out/proba.linearize.scaffolds.final could not be identified.

$ head /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95.linearize.scaffolds.final
>12 156 1722
GTATGGCATAAAATTTCAAAAAAACATCATAGTATACTAAGGCATAAAAATGGCAAAAAACATCATAGTATAGTATGGCATAAAATTTCAAAAAGCGTCATAGTATAATAAGGCATAAAATTTCAAAAAAACATCTTAGTATAATAAGGCATAAAA
>25 146 1397
TGGCAAAAAAACGTCATAGTATAGTATGGCATAAAATTTAAAAAAACGTCATAGTATAATAAGGCATAAAAATAGCAAAAAACGTCATAGTATAATAAGGCATAAAAATAGCAAAAAACATCATAGTATAGTCAGGCACAAAACGT
>27 178 2086
ATGCCAAAAAAATACATATTATAGTAGCGCATAAAAATGGCAAAAAACGTCATAGTATAGTATGGCATAAAATTTGAAAAAATGTCATAGTATAGTAGAGCATAAAAATGGCAAAAAACGTCATAGTATAGTCAGGCACAAAACGTCACAGTATAGTAAGCCATAAAAATGCCAAAAA
>35 140 3999
GAAAAAATGTCATAGTATACTAAGGCATAAAAATGCCAATAAAACGTCATAGTATAGTAGAGCATAAAAATGGCAAAAAACGTCACAGTATAGTCAGGCACAAAACGTCACAGTATAGTAAGGCATAAAAATGCCAAAAA
>49 189 2977
CATAGTATAATAAGGCATAAAAATGGCAAAAAACGTCATAGTATAGTCAGGCACAAAACGTCACAGTATAGTAAGGCATAAAAATGCCAAAAAAGTACATATTATAGTAGAGCATAAAAATGGCGAAAAACGTCATAGTATAGTATGGCATAAAATTTCAAAAAATGTCATAGTATAGTAGAGCATAAA
$ head /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.linearize.scaffolds.final
>12 156 1722
GTATGGCATAAAATTTCAAAAAAACATCATAGTATACTAAGGCATAAAAATGGCAAAAAACATCATAGTATAGTATGGCATAAAATTTCAAAAAGCGTCATAGTATAATAAGGCATAAAATTTCAAAAAAACATCTTAGTATAATAAGGCATAAAA
>25 146 1397
TGGCAAAAAAACGTCATAGTATAGTATGGCATAAAATTTAAAAAAACGTCATAGTATAATAAGGCATAAAAATAGCAAAAAACGTCATAGTATAATAAGGCATAAAAATAGCAAAAAACATCATAGTATAGTCAGGCACAAAACGT
>27 178 2086
ATGCCAAAAAAATACATATTATAGTAGCGCATAAAAATGGCAAAAAACGTCATAGTATAGTATGGCATAAAATTTGAAAAAATGTCATAGTATAGTAGAGCATAAAAATGGCAAAAAACGTCATAGTATAGTCAGGCACAAAACGTCACAGTATAGTAAGCCATAAAAATGCCAAAAA
>35 140 3999
GAAAAAATGTCATAGTATACTAAGGCATAAAAATGCCAATAAAACGTCATAGTATAGTAGAGCATAAAAATGGCAAAAAACGTCACAGTATAGTCAGGCACAAAACGTCACAGTATAGTAAGGCATAAAAATGCCAAAAA
>49 189 2977
CATAGTATAATAAGGCATAAAAATGGCAAAAAACGTCATAGTATAGTCAGGCACAAAACGTCACAGTATAGTAAGGCATAAAAATGCCAAAAAAGTACATATTATAGTAGAGCATAAAAATGGCGAAAAACGTCATAGTATAGTATGGCATAAAATTTCAAAAAATGTCATAGTATAGTAGAGCATAAA
$ head /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Scaffold/out/proba.linearize.scaffolds.final  
head: cannot open `/home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Scaffold/out/proba.linearize.scaffolds.final' for reading: No such file or directory
skoren commented 9 years ago

That is strange since the commands run show:

unlink  /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Scaffold/out/proba.linearize.scaffolds.final
ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.linearize.scaffolds.final /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Scaffold/out/proba.linearize.scaffolds.final

So if Assemble/out/proba.linearize.scaffolds.final exists, the file in Scaffold/out should just be a link to that result and it should have been removed if it existed. What does ls -la show and what's the output in SCAFFOLD.log?

TakashiKoyama commented 9 years ago

Hi,

ls -la seems to show Scaffold/out/proba.linearize.scaffolds.final is not linked. I did not show this because correct/incorrect link is indicated by color and git does not support color font. Sorry about that. Let me show the result of ls -la.

$ ls -la /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95.linearize.scaffolds.final
lrwxrwxrwx. 6 koyama users 13  8月 28 17:54 2015 /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95.linearize.scaffolds.final -> abyss.95-6.fa

'Assemble/out/abyss.95.linearize.scaffolds.final' is indicated by light blue font and 'abyss.95-6.fa' is white.

$ ls -la /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.linearize.scaffolds.final
lrwxrwxrwx. 6 koyama users 13  8月 28 17:54 2015 /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.linearize.scaffolds.final -> abyss.95-6.fa

same as above.

$ ls -la /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Scaffold/out/proba.linearize.scaffolds.final
lrwxrwxrwx. 6 koyama users 13  8月 28 17:54 2015 /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Scaffold/out/proba.linearize.scaffolds.final -> abyss.95-6.fa

This last ls -la result is different from the first two results. 'Scaffold/out/proba.linearize.scaffolds.final' is indicated by red font and 'abyss.95-6.fa' is by red font with white back ground.
And below is the output in SCAFFOLD.log.

Read Bank doesn't exist; creating
frag Bank doesn't exist; creating
lib Bank doesn't exist; creating
parsing fastq file
Finished reading 478856 reads 
Closing frag stream/bank
Closing read stream/bank
Read bank exists; opening
frag bank exists; opening
lib bank exists; opening
parsing fastq file
Finished reading 26 reads 
Closing frag stream/bank
Closing read stream/bank
Contig Bank doesn't exist; creating
Read bank exists; opening
frag bank exists; opening
lib bank exists; opening
Closing frag stream/bank
Closing read stream/bank
Handling contigs
Contig stream closed.
unlink: cannot unlink `/home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Scaffold/out/proba.linearize.scaffolds.final': No such file or directory


I noticed that I can make a correct link in Assemble/out/ but cannot outside that directory.

$ pwd
/home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out
$ ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.linearize.scaffolds.final ./testlink.fa
$ ls -la testlink.fa 
lrwxrwxrwx. 8 koyama users 13  8月 28 17:54 2015 testlink.fa -> abyss.95-6.fa # this shows correct link
$ cd ..
$ pwd
/home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble
$ ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.linearize.scaffolds.final ./testlink.fa
$ ls -la testlink.fa 
lrwxrwxrwx. 8 koyama users 13  8月 28 17:54 2015 testlink.fa -> abyss.95-6.f # this shows incorrect link

I cannot make correct link from Assemble/out/abyss.95.linearize.scaffolds.final too.

$ pwd
/home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble
$ unlink testlink.fa
$ ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95.linearize.scaffolds.final ./testlink.fa
$ ls -la testlink.fa 
lrwxrwxrwx. 8 koyama users 13  8月 28 17:54 2015 testlink.fa -> abyss.95-6.fa # shows incorrect link

Same from Assemble/out/abyss.95-contigs.fa.

$ pwd
/home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble
$ unlink testlink.fa
$ ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95-contigs.fa ./testlink.fa
$ ls -la testlink.fa 
lrwxrwxrwx. 8 koyama users 13  8月 28 17:54 2015 testlink.fa -> abyss.95-6.fa # shows incorrect link


I could not find ln command that link Assemble/out/abyss.95-contigs.fa to abyss.95-6.fa in COMMANDS.log. I have rarely used hard link but all ls -la results seems to be similar to soft link. Don't you think the first ln command could be 'ln -s'? If so, I think all incorrect link should be caused by the first 'ln -s'.

skoren commented 9 years ago

The difference between ln -s and ln is that ln is supposed to make a hard-link so that if you move the directory to another volume or folder structure, the file wouldn't become a broken link The hardlinks should not be distinguishable from normal files via ls (they both point to the same inode, you'd have to track down that two files point to the same inode and that the inode has a reference counter > 1). It seems that on your system the ln command is still making symbolic links even without the -s option, do you have any aliases for ln set up?

I'm not sure why the linking between folders is failing since it is using a full path. You can try changing the code to use ln -s instead of ln as a workaround on your system.

TakashiKoyama commented 9 years ago

Hi,
Thank you for your comments. I am really sorry to make you complicated.

It seems that on your system the ln command is still making symbolic links even without the -s option, do you have any aliases for ln set up?

No. The 'ln' and 'ln -s' commands work properly on my system.

$ pwd
/home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble
$ ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95-6.fa ./testlink_hard.fa
$ ln -s /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95-6.fa ./testlink_soft.fa
$ ls -la
-rw-r--r--.  2 koyama users 102077  8月 28 17:54 2015 testlink_hard.fa
lrwxrwxrwx.  1 koyama users    108  9月  2 23:46 2015 testlink_soft.fa -> /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95-6.fa

And if 'ln' works as 'ln -s', I should also have same problem when other assembler is selected. However, the problem occurs only when abyss is selected.
I think the problem come from Assemble/out/abyss.95-contigs.fa, which is the symbolic link of Assemble/out/abyss.95-6.fa

$ ls -al out/abyss.95-contigs.fa 
lrwxrwxrwx. 7 koyama users 13  8月 28 17:54 2015 out/abyss.95-contigs.fa -> abyss.95-6.fa

In COMMANDS.log, I do not see the command which makes this symbolic link. Hence, I think the cause of this problem must be commands other than those in this log. Here is the first 40 lines of the COMMAND.log

|2015-08-28 17:51:16| perl /home/samba/public/software/metAMOS-1.5rc3/Utilities/perl/shuffleSequences_fastq.pl  /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/013a02.unmapped.sorted.R1.sed.tail.fastq /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/013a02.unmapped.sorted.R2.sed.tail.fastq /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/lib1.seq
|2015-08-28 17:51:16| ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/013a02.unmapped.sorted.R1.sed.tail.fastq /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/lib1.1.fastq
|2015-08-28 17:51:16| ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/013a02.unmapped.sorted.R2.sed.tail.fastq /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/lib1.2.fastq
|2015-08-28 17:51:16| which python
|2015-08-28 17:51:16| echo $PYTHONPATH
|2015-08-28 17:51:17| ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/lib1.seq /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/lib1.fastq
|2015-08-28 17:51:26| java -cp /home/samba/public/software/metAMOS-1.5rc3/Utilities/java:. convertFastqToFasta /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/lib1.seq /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/lib1.fasta /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/lib1.fasta.qual
|2015-08-28 17:51:31| java -cp /home/samba/public/software/metAMOS-1.5rc3/Utilities/java:. convertFastqToFasta /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/lib1.1.fastq /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/lib1.1.fasta /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/lib1.1.fasta.qual
|2015-08-28 17:51:36| java -cp /home/samba/public/software/metAMOS-1.5rc3/Utilities/java:. convertFastqToFasta /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/lib1.2.fastq /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/lib1.2.fasta /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/lib1.2.fasta.qual
|2015-08-28 17:51:36| ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/BACends.sed.qual_SE.fastq /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/lib2.seq
|2015-08-28 17:51:36| touch /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/lib2.seq.mates
|2015-08-28 17:51:36| ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/lib2.seq /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/lib2.fastq
|2015-08-28 17:51:36| java -cp /home/samba/public/software/metAMOS-1.5rc3/Utilities/java:. convertFastqToFasta /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/lib2.seq /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/lib2.fasta /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/lib2.fasta.qual
|2015-08-28 17:51:41| /home/samba/public/software/metAMOS-1.5rc3/FastQC/fastqc -t 40 /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/013a02.unmapped.sorted.R1.sed.tail.fastq /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/013a02.unmapped.sorted.R2.sed.tail.fastq /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/BACends.sed.qual_SE.fastq

|2015-08-28 17:51:41| mv /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/013a02.unmapped.sorted.R1.sed.tail_fastqc /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/lib1.1.fastqc
|2015-08-28 17:51:41| mv /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/013a02.unmapped.sorted.R2.sed.tail_fastqc /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/lib1.2.fastqc
|2015-08-28 17:51:41| mv /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/BACends.sed.qual_SE_fastqc /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/lib2.1.fastqc
|2015-08-28 17:51:41| touch /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/preprocess.success
|2015-08-28 17:51:42|# [ASSEMBLE]
|2015-08-28 17:51:42| head -n 4 /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/lib1.fastq > /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/tmp.fastq
|2015-08-28 17:51:43| rm /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/tmp.fastq
|2015-08-28 17:51:43| head -n 4 /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/lib2.fastq > /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/tmp.fastq
|2015-08-28 17:51:43| rm /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/tmp.fastq
|2015-08-28 17:53:15| /home/samba/public/software/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/kmergenie/kmergenie /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.kmergenie.in -t 40 -k 151 -o proba
|2015-08-28 17:53:15| touch /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95.run
|2015-08-28 17:53:15| touch /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/newbler.95.run
|2015-08-28 17:53:15| touch /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/spades.95.run
|2015-08-28 17:53:15| touch /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/idba-ud.95.run
|2015-08-28 17:53:15| touch /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/sga.95.run
|2015-08-28 17:53:15| touch /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/velvet.95.run
|2015-08-28 17:53:15| touch /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/masurca.95.run
|2015-08-28 17:53:16| head -n 4 /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/lib1.fastq > /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/tmp.fastq
|2015-08-28 17:53:17| rm -f /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/tmp.fastq
|2015-08-28 17:54:14| /home/samba/public/software/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/abyss/bin/abyss-pe k=95 name=abyss.95 lib='lib1' mp='' np=40  lib1='/home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/lib1.1.fastq /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Preprocess/out/lib1.2.fastq' 
|2015-08-28 17:54:14| unlink /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95.asm.contig
|2015-08-28 17:54:14| ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95-contigs.fa /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95.asm.contig
|2015-08-28 17:54:14| ln /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95-contigs.fa /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95.linearize.scaffolds.final

Line 39 is the first line where 'abyss.95-contigs.fa' to come. But 'abyss.95-contigs.fa' is symbolic link of the abyss.95-6 and I do not see 'abyss.95-6' in this log. Is there any other command which is not shown in COMMAND.log?

skoren commented 9 years ago

Ah, sorry for the confusion, the symlink is being created within the ABySS assembler pipeline (which also creates abyss.95-contigs.fa), which is why it doesn't show up in the COMMAND.log. This likely also explains why the error only occurs when ABySS is selected as the best. I think if you change how metAMOS links to the ABySS results, the issue should be fixed. If you edit the code in src/generic.py on lines 426, 430, 437, and 439 from

ln

to 

ln -L

It should link to the source of the symlink rather than propagating the symlink. You can test this out if you remove Assemble/out/proba.linearize.scaffolds.final and run 

ln -L /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/abyss.95-contigs.fa /home/koyama/Documents/yellowtailBenQTL/seq/BACassembly/iMetAMOS/013a02/assembly2/Assemble/out/proba.linearize.scaffolds.final

Then try linking to proba.linearize.scaffolds.final as you did before from inside Assemble/out and outside that directory.

TakashiKoyama commented 9 years ago

Finally, it works!! Thank you for taking time for solving my problem!

skoren commented 9 years ago

I've committed the fix. Annoyingly, the ln -L option is the default behavior on OSX (and in fact -L is an invalid option) which is why I didn't observe the issue on my system.