Append top level job name to rule-based job name

[CJREID]

Hi there,

I'm manually submitting multiple runs of the same pipeline for different samples to slurm with this profile to run rules as separate slurm jobs. The problem is every rule for each sample appears the same in squeue and has no clear linkage to the sample. I was wondering if it's possible to manipulate the profile such that the slurm job name for a given rule also includes the name and job id of the slurm job that submitted it.

e.g If sample_pipeline is submitted to slurm and takes main_job_id, can the config.yaml or any of the helper scripts be edited so it then submits jobs with --job-name={sample_pipeline}.{main_job_id}.{rule}.{jobid}?

Thanks,

Cam

[mbhall88]

The correct way to do this with snakemake would be to run a single instance of the pipeline with multiple samples, using sample as a wildcard, which will then be injected into the job name. See this tutorial for an example.

[CJREID]

Hi Michael,

Thanks for the response. Unfortunately the pipeline I'm using is wrapped and doesn't use sample wildcards in the conventional way to handle inputs, it's also very long and prone to fail at different points so it's been easier for me to manage samples separately.

However, I did get a solution for this whereby you can feed any variable you like to snakemake via the --default-resources parameter e.g.--default-resources jobname=jobname id=$SLURM_JOB_ID and then access them via the profile.

E.g.

cluster:
  sbatch
    --job-name={resources.jobname}.{rule}.{resources.id}

Cheers,

Cam