moiexpositoalonsolab / grenepipe

A flexible, scalable, and reproducible pipeline to automate variant calling from raw sequence reads, with lots of bells and whistles.
http://grene-net.org
GNU General Public License v3.0
94 stars 21 forks source link

SLURM No wall time information given #54

Open RvV1979 opened 2 months ago

RvV1979 commented 2 months ago

Hi Lucas, I am running the latest version of grenepipe (v0.13.1) with snakemake 8.15.2 using SLURM using --executor slurm --profile profiles/slurm options. The file profiles/slurm/cluster_config.yaml includes the following:

__default__:
  time: 120 # Default time (minutes). A time limit of zero requests that no time limit be imposed
  mem: 10G # Default memory. A memory size specification of zero grants the job access to all of the memory on each node.
  cpus-per-task: 1
  nodes: 1
  ntasks: 1

However, when executing the pipeline I get the following warnings and jobs do not get accepted:

No wall time information given. This might or might not work on your cluster. If not, specify the resource runtime in your rule or as a reasonable default via --default-resources.
No job memory information ('mem_mb' or 'mem_mb_per_cpu') is given - submitting without. This might or might not work on your cluster.

It seems the specified default settings are not appropriately transmitted. What may be going wrong? Thanks

lczech commented 2 months ago

Hi @RvV1979,

thanks for reporting this, that seems to be a consequence of the changes in Snakemake concerning their SLURM setup. Similar issues were reported here. I think the cluster config file needs to be updated to fit with the new SLURM executor of Snakemake. Please have a look there - that should explain how to fix this for your run. Once I circle back to working on grenepipe, I'll then look into how our config files need to be adapted as well to make this easier. Unfortunately, bit short on time for maintenance these days, but I'll get to it eventually :-)

Cheers, and feel free to keep me updated on progress or further troubles - I'll try to help as much as I find the time! Lucas