search

CFSAN-Biostatistics / snp-pipeline

SNP Pipeline is a pipeline for the production of SNP matrices from sequence data used in the phylogenetic analysis of pathogenic organisms sequenced from samples of interest to food safety.
Other
57 stars 31 forks source link

Error: "$numAlignThreads" contains no decimal number #2

Closed fabspagnolo closed 9 years ago

fabspagnolo commented 9 years ago

Hello.

I requested that snp-pipeline/0.3.4 be installed onto our bioinformatics HPC as a local/modulefiles so that it is available to all on the cluster. Admin has done so. The cluster uses SGE as the batch system. I modified the .conf file to specify PEname="threaded" as per admin (also tried "openmpi" when issues began). Of course, -Q grid is written into submitted command.

Submitted run_snp_pipeline.sh using standard qsub at command line (also tried specifying at qsub, as in qsub -pe threaded 8 run_snp_pipeline.sh yada yada).

Complete line used is:

qsub -pe threaded 8 ~/scripts/snp-pipeline-0.3.4/scripts/run_snp_pipeline.sh -m soft -c ~/scripts/snp-pipeline-0.3.4/snppipeline/data/configuration/newsnppipeline.conf -Q grid -o ~/data/testAgona/outputDirectory -s ~/data/testAgona/cleanInputs ~/data/testAgona/cleanInputs/reference/NC_011149.fasta

Instantly get following errors:

warning: "-N" option has already been set, overriding previous setting warning: "-j" option has already been set, overriding previous setting Unable to read script file because of error: Numerical value invalid! The initial portion of string "$numAlignThreads" contains no decimal number

Admin thinks the way parameters are assigned in the script are not viable in our environment.

In trying to eliminate any module version conflicts or other module issues, I have also made changes to top of run_snp_pipeline.sh to include the following lines immediately after #!/bin/bash

$ -S /bin/bash

$ -cwd

module load python/2.7 module load sratoolkit/2.3.2-5 module load snp-pipeline/0/3/4 module load samtools/0.1.19 module load bowtie2/2.1.0 module load bwa/0.7.7

But these haven't had any effect. Any ideas would be greatly appreciated.

stevendavis commented 9 years ago

Hello fabspagnolo. Thanks for the feedback. The qsub command is already embedded inside run_snp_pipeline.sh.

Don't do this: qsub -pe threaded 8 run_snp_pipeline.sh -c newsnppipeline.conf -Q grid -s mySamplesDir myReference.fasta

Do this: run_snp_pipeline.sh -c newsnppipeline.conf -Q grid -s mySamplesDir myReference.fasta

You can find documentation here: http://snp-pipeline.readthedocs.org/en/latest/usage.html#high-performance-computing

Please try again without the qsub command.

fabspagnolo commented 9 years ago

Steve,

Thanks for the quick response. I thought you were going to say something along those lines. The admin group here keeps saying I need the qsub, so I was in a bit of a bind. Thank you for the clarification. Sorry to submit a bug report this way, but I thought that the best option. Hope it didn't foul up your day too much.

The pipeline looks like exactly what we need; I'm looking forward to using it.

If I run into major problems (or more cluster admin problems) I'll get back in touch.

Regards, Fab. On Aug 12, 2015 4:16 PM, "Steve Davis" notifications@github.com wrote:

Hello fabspagnolo. Thanks for the feedback. The qsub command is already embedded inside run_snp_pipeline.sh.

Don't do this: qsub -pe threaded 8 run_snp_pipeline.sh -c newsnppipeline.conf -Q grid -s mySamplesDir myReference.fasta

Do this: run_snp_pipeline.sh -c newsnppipeline.conf -Q grid -s mySamplesDir myReference.fasta

You can find documentation here:

http://snp-pipeline.readthedocs.org/en/latest/usage.html#high-performance-computing

Please try again without the qsub command.

— Reply to this email directly or view it on GitHub https://github.com/CFSAN-Biostatistics/snp-pipeline/issues/2#issuecomment-130432663 .

stevendavis commented 9 years ago

Thanks for your interest in the SNP Pipeline. Closing this issue. Feel free to contact us again.