OrientContigs passed unrecognized option `-minRedundancy'

[JustGitting]

Hi,

I started off by trying out the metAMOS test script:

$ cd Test $ ./run_test.sh

However, the following error appeared in the log

. /metAMOS/Test/test1/Logs/SCAFFOLD.log

...

Clique range = 1199, 1754 Clique range = 1199, 1658 new NORMALS: 76 ANTINORMALS: 558 INNIES: 0 OUTIES: 0 Writing edge link with it 3 Writing edge link with it 4

/local/bin/OrientContigs: unrecognized option `-minRedundancy' Command line parsing failed Determine contig order and orientation USAGE: OrientContigs -b[ank] [-all] [-noreduce] [-agressive] [-redundancy minLinks] [-repeats fileName] [-skip] The -all option will force initialization of all contigs, including those that have no links to them, otherwise they remain uninitialized The -noreduce option will turn off search for common motifs and recursively remove them, thus simplyfing the graph The -agressive option will not mark edges that move a contig more than 3 STDEVS away as bad and will try to reconcile the positions The -redundancy option specifies the minimum number of links between two contigs before they will be scaffolded The -repeats option specifies a file containing a list of contig IIDs which are considered repeats and whose edges will be unused The -skip option will skip edges that have too low a weight relative to the weights of the other edges connecting their respective nodes. ^^^^^^^^^^^^^^^^ I traced the unknown argument "-minRedundancy" to the configuration file: ./Utilities/config/bambus.spec #### # repeat options [MarkRepeats] # noCoverageRepeats # noPathRepeats redundancy 50 [OrientContigs] # edge threshold redundancy, setting to 0 means automatically set redundancy 10 minRedundancy 5 all # noreduce # maxOverlap 500 ^^^^^^^^ Commenting out "minRedundancy 5" solved the error so that the bambus.spec file now looks like: #### # repeat options [MarkRepeats] # noCoverageRepeats # noPathRepeats redundancy 50 [OrientContigs] # edge threshold redundancy, setting to 0 means automatically set redundancy 10 # minRedundancy is not a option for OrientContigs. # minRedundancy 5 all # noreduce # maxOverlap 500 The error disappeared, but the run_test.sh script does not complete as detailed in my other ticket. Cheer

[skoren]

The minRedudancy option is correct. It was added to AMOS after the 3.1 release. metAMOS will download the correct version of AMOS when running INSTALL.py. You do not need to download and install AMOS 3.1 before running metAMOS.

However, metAMOS should already prefer to use AMOS from its own directory rather than a global instance. Does your metAMOS directory contain an AMOS/Linux-x86_64/bin/ directory? Can you run the programs in that bin directory? I would suggest removing the AMOS directory and re-running INSTALL.py. It should skip installing programs you already have installed and just download/install AMOS. You may also want to remove your AMOS 3.1 install from the path.

[JustGitting]

Dear Dr Koren,

Thanks, I'll remove the AMOS I installed. I had installed it because the previous README.md contained:

"The main prequisite software is python2.6+ and AMOS (available from http://amos.sf.net). Once python2.6+ and AMOS are installed, "

However, the latest README.md does not require AMOS to be installed before running INSTALL.py.

Cheers

[treangen]

Closing this issue, was due to incompatible AMOS version. Current AMOS binaries (included in install) will fix this issue.

[JustGitting]

Hi,

I've deleted the AMOS source directory and the directories in the //local directory which include.

/bin /lib /local /share

I've also removed the manually installed AMOS directory from the PATH and restarted my session.

I then re-installed metAMOS by executing the following commands:

$ cd metAMOS $ git pull $ python INSTALL.py fastqc core imetamos optional ca ale sga freebayes abyss lap frcbam quast metaphyler eautils ray numpy fcp edena uniprot cython cgal mira matplotlib prokka soap2 masurca kronatools sra glimmer-mg kraken psutil phymm setuptools amos phylosift phmmer phymmbl reapr

Notable error: ####### .... Checking whether refseq_protein is complete. Expecting 10 partitions.

Error: MaSuRCA requires gcc at least version 4.4, found version 4.1. Please update and try again tRNAscan not found, optional for Annotate step, download now? Enter Y/N: Y .... ^^^^^^^

At the end I get the following messages,

#########

The file "setup.tRNAscan-SE" has been created. If you type "source setup.tRNAscan-SE; rehash", the paths in your environment will be updated for this session.

To update your environment upon every login, you should add the line:

source //metAMOS/Utilities/cpp/Linux-x86_64/trnascan/setup.tRNAscan-SE

to your ".cshrc" file.

If you prefer to manually update your environment variables, be sure to make the following changes:

1) Add /home/user1/bin to your PATH variable 2) Add /home/user1/bin to your PERl5LIB variable 3) Add /home/user1/man to your MANPATH variable

Run setup.py.. ^[running install_scripts running egg_info writing metAMOS.egg-info/PKG-INFO writing top-level names to metAMOS.egg-info/top_level.txt writing dependency_links to metAMOS.egg-info/dependency_links.txt writing metAMOS.egg-info/PKG-INFO writing top-level names to metAMOS.egg-info/top_level.txt writing dependency_links to metAMOS.egg-info/dependency_links.txt reading manifest file 'metAMOS.egg-info/SOURCES.txt' writing manifest file 'metAMOS.egg-info/SOURCES.txt' running build_scripts copying build/scripts-2.7/runPipeline.py -> //metAMOS copying build/scripts-2.7/initPipeline.py -> /group/metagen/dbtools/src/metAMOS changing mode of //metAMOS/runPipeline.py to 775 changing mode of //metAMOS/initPipeline.py to 775 running build_ext skipping './src/preprocess.c' Cython extension (up-to-date) skipping './src/annotate.c' Cython extension (up-to-date) skipping './src/fannotate.c' Cython extension (up-to-date) skipping './src/mapreads.c' Cython extension (up-to-date) skipping './src/findorfs.c' Cython extension (up-to-date) skipping './src/findscforfs.c' Cython extension (up-to-date) skipping './src/scaffold.c' Cython extension (up-to-date

^^^^^^^^^^^^

However, I don't why setup.tRNAscan-SE wants to add my home directory bin (/home/user1/bin) as I thought metAMOS was setup to install everything within the //metAMOS/ directory.

Is this correct behaviour?

Does your metAMOS directory contain an AMOS/Linux-x86_64/bin/ directory?

No, AMOS/Linux-x86_64/bin/ does not exist, it has something slightly different:

./metAMOS/AMOS/Linux-x86_64/lib

./metAMOS/AMOS/Linux-x86_64/lib/AMOS/ ./metAMOS/AMOS/Linux-x86_64/lib/Statistics/ ./metAMOS/AMOS/Linux-x86_64/lib/TIGR/

and in the AMOS/ directory

$ ls ./metAMOS/AMOS/Linux-x86_64/lib/AMOS AMOS.py* libAMOS.a libAMOSFoundation.a libAlign_poly.a libBundler.a libCelMsg.a libContigUtils.a libGraph.a libSlice.a

I tried to re-run the test scripts.

$ cd Test $ ./run_test.sh

Project dir //metAMOS/Test/test1 successfully created! Use runPipeline.py to start Pipeline !!Sorry, karken is not a supported classification method. Using FCP instead

Starting Task = runpipeline.RUNPIPELINE Starting metAMOS pipeline Warning: AMOS is not found, some functionality will not be available Error: cannot find AMOS in //metAMOS/AMOS or . Please check your path and try again.

[skoren]

I am not sure why your AMOS directory under metAMOS is broken. Have you tried removing it and re-running INSTALL.py? I just ran it on a test system and the bin directory was successfully extracted and available.

The MaSuRCA message is expected, it requires gcc 4.4 to build. If you have an older gcc you will not be able to use MaSuRCA.

[JustGitting]

Dear Dr Koren,

Thanks, I've deleted the AMOS directory and re-run INSTALL.py.

$ rm -rf AMOS/ $ python INSTALL.py amos

The bin directory was created successfully and run_test.sh completed!

$ cd ./Test $ ./run_test.sh .... Job = [proba.asm.contig -> proba.scf.fa] completed Completed Task = postprocess.Postprocess done! pipeline took 5.84 minutes

However, run_test2.sh is stalling during annotation (I can add this as a new ticket if needed).

############## $ ./run_test2.sh ....

Starting Task = annotate.ANNOTATE *\ metAMOS running command: touch /myproject/metAMOS/Test/test2/Annotate/out/proba.annots

*\ metAMOS running command: unlink /myproject/metAMOS/Test/test2/Annotate/in/proba.asm.contig

*\ metAMOS running command: ln /myproject/metAMOS/Test/test2/Assemble/out/proba.asm.contig /myproject/metAMOS/Test/test2/Annotate/in/proba.asm.contig

*\ metAMOS running command: unlink /myproject/metAMOS/Test/test2/Annotate/out/proba.hits

** metAMOS running command: rm -f /myproject/metAMOS/Test/test2/Annotate/out/.hits

** metAMOS running command: rm -f /myproject/metAMOS/Test/test2/Annotate/out/.epsilon-nb_results.txt

** metAMOS running command: rm -f /myproject/metAMOS/Test/test2/Annotate/out/.phymm.out

*\ metAMOS running command: rm -rf /myproject/metAMOS/Test/test2/Annotate/out/PS_temp

*\ metAMOS running command: /myproject/metAMOS/phylosift/bin/phylosift all --threads=8 /myproject/metAMOS/Test/test2/Annotate/in/proba.asm.contig --coverage=/myproject/metAMOS/Test/test2/Assemble/out/proba.contig.cnt

^^^^^^^^^^^^^^^^^^

There is no further activity according to top.

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 5906 user1 15 0 434m 172m 7172 S 0.0 0.2 0:13.41 runPipeline 5937 user1 18 0 265m 29m 1120 S 0.0 0.0 0:00.00 runPipeline 6373 user1 17 0 402m 166m 836 S 0.0 0.2 0:00.00 runPipeline 6374 user1 17 0 402m 166m 836 S 0.0 0.2 0:00.00 runPipeline 6375 user1 17 0 402m 166m 836 S 0.0 0.2 0:00.00 runPipeline 6376 user1 17 0 402m 166m 836 S 0.0 0.2 0:00.00 runPipeline 6377 user1 17 0 402m 166m 836 S 0.0 0.2 0:00.00 runPipeline 6378 user1 18 0 402m 166m 836 S 0.0 0.2 0:00.00 runPipeline 6380 user1 18 0 402m 166m 836 S 0.0 0.2 0:00.00 runPipeline 6381 user1 18 0 402m 166m 848 S 0.0 0.2 0:00.00 runPipeline 6386 user1 18 0 1091m 967m 2748 S 0.0 1.0 0:55.36 perl 7107 user1 15 0 1651m 1.6g 2288 S 0.0 1.7 8:36.66 pplacer 7813 user1 25 0 1651m 1.6g 528 S 0.0 1.7 0:00.00 pplacer

The logs may have something:

$ cat ./Test/test2/Logs/ANNOTATE.log unlink: cannot unlink /myproject/metAMOS/Test/test2/Annotate/in/proba.asm.contig': No such file or directory unlink: cannot unlink/myproject/metAMOS/Test/test2/Annotate/out/proba.hits': No such file or directory

$ tail -9 ./Test/test2/Logs/COMMANDS.log

|2014-01-29 15:29:58|# [ANNOTATE] |2014-01-29 15:29:58| touch /myproject/metAMOS/Test/test2/Annotate/out/proba.annots |2014-01-29 15:29:58| unlink /myproject/metAMOS/Test/test2/Annotate/in/proba.asm.contig |2014-01-29 15:29:58| ln /myproject/metAMOS/Test/test2/Assemble/out/proba.asm.contig /myproject/metAMOS/Test/test2/Annotate/in/proba.asm.contig |2014-01-29 15:29:58| unlink /myproject/metAMOS/Test/test2/Annotate/out/proba.hits |2014-01-29 15:29:58| rm -f /myproject/metAMOS/Test/test2/Annotate/out/.hits |2014-01-29 15:29:58| rm -f /myproject/metAMOS/Test/test2/Annotate/out/.epsilon-nb_results.txt |2014-01-29 15:29:58| rm -f /myproject/metAMOS/Test/test2/Annotate/out/*.phymm.out |2014-01-29 15:29:58| rm -rf /myproject/metAMOS/Test/test2/Annotate/out/PS_temp

What would cause the process to stall, waiting for a file that is not there?

[treangen]

JustGitting,

I've now been able to reproduce the error you observed & opened a new issue #111, thanks for reporting it. As this is in optional component of MetAMOS the test failure can be safely ignored if you use a different classification method. For further updates on this please refer to issue #111 as issue #110 has been closed.