SLURM -mem or --mem-per-cpu

[mattingsdal]

Hi, Im trying to run canu on my local grid using slurm 15.08.8. When I try and run the yeast x20 example data using:

canu -p asm -d yeast genomeSize=12.1m corMhapSensitivity=high corMinCoverage=2 errorRate=0.035 minOverlapLength=499 corMaxEvidenceErate=0.3 -java="/usr/bin/java" -pacbio-raw yeast.20x.fastq.gz -gridOptions='--mem-per-cpu=4G --account=cees --time=02:00:00'

i get the below error.

Starting command on Fri May 13 14:26:04 2016 with 77104.6 GB free disk space

sbatch \
  --mem=4g \
  --cpus-per-task=1 \
  --mem-per-cpu=4G \
  --account=cees \
  --time=02:00:00  \
  -D `pwd` \
  -J "canu_asm" \
  -o /usit/abel/u1/mortema/genome/yeast/canu-scripts/canu.01.out /usit/abel/u1/mortema/genome/yeast/canu-scripts/canu.01.sh
Error: --mem' is not supported! Please only use --mem-per-cpu=4G

Please read http://uio.no/hpc/abel/help/user-guide/
 and 'man sbatch' to learn more about submitting with the SLURM queue system

-- Finished on Fri May 13 14:26:04 2016 (lickety-split) with 77104.6 GB free disk space
----------------------------------------
ERROR (rc=256):
ERROR:  Failed with exit code 1.
ERROR:
================================================================================
Please panic.  canu failed, and it shouldn't have.

Stack trace:

 at /cluster/home/mortema/software/canu-master/Linux-amd64/bin/lib/canu/Defaults.pm line 234
        canu::Defaults::caFailure('Failed to submit script', undef) called at     /cluster/home/mortema/software/canu-master/Linux-amd64/bin/lib/canu/Execution.pm line 860
        canu::Execution::submitScript('/usit/abel/u1/mortema/genome/yeast', 'asm', undef) called at     /cluster/home/mortema/software/canu-master/Linux-amd64/bin/canu line 355

canu failed with 'Failed to submit script'.

`

It seems that canu is parsing --mem=4 to the executing command while slurm is expecting this to be --mem-per-cpu.

Is there a way to hack the code so that the parsed command is "--mem-per-cpu" instead of "--mem"

[skoren]

Yes, you'd have to change two places, line 907 in src/pipelines/canu/Execution.pm:

 907     if (uc(getGlobal("gridEngine")) eq "SGE" || uc(getGlobal("gridEngine")) eq "SLURM") {

and line 80 in src/pipelines/canu/Grid_Slurm.pm:

 80     setGlobalIfUndef("gridEngineMemoryOption",               "--mem-per-cpu=MEMORY");

We can also check if Slurm generally supports the mem per cpu option and use that as the default instead of mem.

[brianwalenz]

Canu will set it's own memory requirement for each component. You don't need to supply it in gridOptions.

Serge - the problem is that --mem and --mem-per-cpu are mutually exclusive.

[mattingsdal]

same error as above with NOT specifying "--mem-per-cpu" in gridOptions

[brianwalenz]

OK, we switched to --mem-per-cpu, and added a tiny bit of logic to allow switching back to --mem (with gridEngineMemoryOption=--mem=MEMORY) for the next user with a bizarre slurm configuration.

Please close the case if its working for you.

[mattingsdal]

Hi again,

Editing the above code does not work im sorry to say. We now expect canu to parse "--mem-per-cpu" instead of "--mem", so I have exluded this option in gridOptions:

canu -p asm -d yeast genomeSize=12.1m corMhapSensitivity=high corMinCoverage=2 errorRate=0.035 minOverlapLength=499 corMaxEvidenceErate=0.3 -java="/usr/bin/java" -pacbio-raw yeast.20x.fastq.gz -gridOptions='--account=cees --time=02:00:00'

ERROR:

-- Starting command on Wed May 18 09:45:17 2016 with 76134.1 GB free disk space

sbatch \ --mem=4g \ --cpus-per-task=1 \ --account=cees \ --time=02:00:00 \ -D pwd \ -J "canu_asm" \ -o /usit/abel/u1/mortema/genome/yeast/canu-scripts/canu.01.out >/usit/abel/u1/mortema/genome/yeast/canu-scripts/canu.01.sh * Error: '--mem' is not supported. * Error: You have to specify --mem-per-cpu (i.e. --mem-per-cpu=4G)

[brianwalenz]

With the most recent version, I'm seeing --mem-per-cpu used by default.

Since I don't know what edits you made, and don't really want to try to debug them, grab the latest version and try again.

[mattingsdal]

Yeah ! working now after updating through git clone.. closing