IndexError: list index out of range when runPipeline executed

[peterdfields]

When trying to run either run_pipeline_test.sh or runPipeline more generally I get the following sort of error:

./run_pipeline_test.sh
Warning: Newbler is not found, some functionality will not be available
Warning: MetaGeneMark is not found, some functionality will not be available
Warning: SignalP+ is not found, some functionality will not be available
Warning: PHmmer is not found, some functionality will not be available
Project directory already exists, please specify another
Alternatively, use runPipeline to run an existing project
[Steps to be skipped]:  set(['FunctionalAnnotation', 'FindRepeats'])
Starting Task = runpipeline.RUNPIPELINE
Starting metAMOS pipeline
Warning: Newbler is not found, some functionality will not be available
Warning: MetaGeneMark is not found, some functionality will not be available
Warning: SignalP+ is not found, some functionality will not be available
Warning: PHmmer is not found, some functionality will not be available
[Available RAM: 359 GB]
        *ok
[Available CPUs: 56]
        *ok
IndexError: list index out of range

However, the INSTALL.py with the 'core' and 'iMetAMOS' option completes successfully. Any ideas about what might be going wrong?

[skoren]

This looks like it might be a python compatibility or package version. What was the output from INSTALL.py?

Can you modify run_pipeline_test.sh to add -v to the runPipeline command: runPipeline -v -a soap -c kraken -g fraggenescan -p 15 -d test1 -k 55 -f Assemble,MapReads,FindORFS,Annotate,FunctionalAnnotation,Propagate,Classify,Abundance,FindScaffoldORFS -n FunctionalAnnotation

That will give more details on the error.

[peterdfields]

Hm. The output I'm not getting from the INSTALL.py is as follows:

curl: (7) Failed to connect to cbcb.umd.
edu port 21: Connection timed out
unzip:  cannot find or open frcbam.zip, frcbam.zip.zip or frcbam.zip.ZIP.
mv: cannot stat ‘FRC_align-3398ca469b2077d6672b85317eee6fea171b6a27’: No such file or directory
<<Welcome to metAMOS install>>
Selected to install workflowName: CORE.
Selected to install workflowName: OPTIONAL.
Selected to install workflowName: IMETAMOS.
Will automatically install:
        Pysam
        Fastqc
        Ca
        Ale
        Sga
        Velvet-Sc
        Metaphyler
        Phylosift
        Metavelvet
        Velvet
        Soap2
        Frcbam
        Quast
        Eautils
        Ray
        Numpy
        Freebayes
        Fcp
        Edena
        Uniprot
        Cython
        Cgal
        Mira
        Matplotlib
        Prokka
        Idba
        Cmake
        Lap
        Masurca
        Kronatools
        Abyss
        Sra
        Spades
        Kraken
        Psutil
        Setuptools
        Amos
        Phmmer
        Kmergenie
        Reapr
Checking whether refseq_protein is complete. Expecting 25 partitions.

Traceback (most recent call last):
  File "INSTALL.py", line 1678, in <module>
    os.chdir("./Utilities/cpp/%s%s-%s%sFRCbam/src/samtools"%(os.sep, OSTYPE, MACHINETYPE, os.sep))
OSError: [Errno 2] No such file or directory: './Utilities/cpp//Linux-x86_64/FRCbam/src/samtools'

And then from the modified run_pipeline_test.sh script:

[Steps to be skipped]:  set(['FunctionalAnnotation', 'FindRepeats'])
Starting Task = runpipeline.RUNPIPELINE
*** metAMOS running command: rm /home/peter/bioinformatics/metAMOS-1.5rc3/Test/test1/Assemble/out/*.contig.cvg

*** metAMOS running command: rm /home/peter/bioinformatics/metAMOS-1.5rc3/Test/test1/Logs/mapreads.ok

*** metAMOS running command: rm /home/peter/bioinformatics/metAMOS-1.5rc3/Test/test1/Logs/propagate.ok

*** metAMOS running command: touch /home/peter/bioinformatics/metAMOS-1.5rc3/Test/test1/Annotate/out/proba.annots

*** metAMOS running command: touch /home/peter/bioinformatics/metAMOS-1.5rc3/Test/test1/Scaffold/out/proba.linearize.scaffolds.final

*** metAMOS running command: touch /home/peter/bioinformatics/metAMOS-1.5rc3/Test/test1/FindORFS/out/proba.faa

*** metAMOS running command: rm /home/peter/bioinformatics/metAMOS-1.5rc3/Test/test1/Abundance/out/proba.taxprof.pct.txt

*** metAMOS running command: rm /home/peter/bioinformatics/metAMOS-1.5rc3/Test/test1/Annotate/out/proba.hits

*** metAMOS running command: rm /home/peter/bioinformatics/metAMOS-1.5rc3/Test/test1/Assemble/out/*.faa

*** metAMOS running command: rm /home/peter/bioinformatics/metAMOS-1.5rc3/Test/test1/Assemble/out/*.fna

*** metAMOS running command: rm /home/peter/bioinformatics/metAMOS-1.5rc3/Test/test1/FindORFS/out/*.faa

*** metAMOS running command: rm /home/peter/bioinformatics/metAMOS-1.5rc3/Test/test1/FindORFS/out/*.fna

*** metAMOS running command: rm /home/peter/bioinformatics/metAMOS-1.5rc3/Test/test1/Logs/assemble.ok

*** metAMOS running command: rm /home/peter/bioinformatics/metAMOS-1.5rc3/Test/test1/Assemble/out/*.asm.contig

*** metAMOS running command: rm /home/peter/bioinformatics/metAMOS-1.5rc3/Test/test1/Logs/classify.ok

Starting metAMOS pipeline
Found pysam in /home/peter/bioinformatics/metAMOS-1.5rc3/Utilities/python/lib/python/pysam-0.6-py2.7-linux-x86_64.egg/pysam/__init__.pyc
Found psutil in /home/peter/bioinformatics/metAMOS-1.5rc3/Utilities/python/lib/python/psutil-0.6.1-py2.7-linux-x86_64.egg/psutil/__init__.pyc
Warning: Newbler is not found, some functionality will not be available
Warning: MetaGeneMark is not found, some functionality will not be available
Warning: SignalP+ is not found, some functionality will not be available
Warning: PHmmer is not found, some functionality will not be available
[Available RAM: 325 GB]
        *ok
[Available CPUs: 56]
        *ok
IndexError: list index out of range

[peterdfields]

I think at least some of the issue had to do with the paths you modified in the master repo, and the initial error is something related to python though I'm not at all sure what it is (though maybe psutil is involved). I use pyenv to keep a couple different python versions so that's a possible issue, and it seems to bridge a couple different linux distros I've tried (opensuse, centos, and ubuntu). The best luck I've had so far is cloning from github rather than from compiling from the release. I'll close this for now.