Snakemake now failing with Slurm profile

Hi Job,

I've been trying to run the new release of the pipeline, but unfortunately have not been able to get it to work at all this time. I can download and install Snakemake fine and follow the instructions exactly in the sci-rocket docs, and I have a Slurm-specific Snakemake configuration that has worked perfectly for me before (I have even remade it a few times to try and fix it but it hasn't worked). Now when I try to run the snakemake --use-conda --profile <profile_name> --configfile <path_config> command, I get the error:

snakemake --use-conda --profile embl_slurm --configfile /g/dorrity/sci_rna_expts/medaka_apr2023/Nova/sci-rocket/run2/config_run2.yaml`

Using profile embl_slurm for setting default command line arguments.`

usage: snakemake [-h] [--dry-run] [--profile PROFILE] [--workflow-profile WORKFLOW_PROFILE] [--cache [RULE ...]] [--snakefile FILE] [--cores N] [--jobs N] [--local-cores N]  [--resources NAME=INT [NAME=INT ...]] [--set-threads RULE=THREADS [RULE=THREADS ...]] [--max-threads MAX_THREADS] [--set-resources RULE:RESOURCE=VALUE [RULE:RESOURCE=VALUE ...]] [--set-scatter NAME=SCATTERITEMS [NAME=SCATTERITEMS ...]] [--set-resource-scopes RESOURCE=[global|local] [RESOURCE=[global|local] ...]] [--default-resources [NAME=INT ...]] [--preemptible-rules [PREEMPTIBLE_RULES ...]] [--preemptible-retries PREEMPTIBLE_RETRIES] [--config [KEY=VALUE ...]] [--configfile FILE [FILE ...]] [--envvars VARNAME [VARNAME ...]] [--directory DIR] [--touch] [--keep-going]  [--rerun-triggers {code,input,mtime,params,software-env} [{code,input,mtime,params,software-env} ...]] [--force] [--executor {local,dryrun,touch}] [--forceall]  [--forcerun [TARGET ...]] [--prioritize TARGET [TARGET ...]] [--batch RULE=BATCH/BATCHES] [--until TARGET [TARGET ...]] [--omit-from TARGET [TARGET ...]] [--rerun-incomplete] [--shadow-prefix DIR] [--scheduler [{ilp,greedy}]] [--wms-monitor [WMS_MONITOR]] [--wms-monitor-arg [NAME=VALUE ...]]  [--scheduler-ilp-solver {COIN_CMD}] [--scheduler-solver-path SCHEDULER_SOLVER_PATH] [--conda-base-path CONDA_BASE_PATH] [--no-subworkflows] [--precommand PRECOMMAND] [--groups GROUPS [GROUPS ...]] [--group-components GROUP_COMPONENTS [GROUP_COMPONENTS ...]] [--report [FILE]] [--report-stylesheet CSSFILE] [--draft-notebook TARGET] [--edit-notebook TARGET] [--notebook-listen IP:PORT] [--lint [{text,json}]] [--generate-unit-tests [TESTPATH]] [--containerize] [--export-cwl FILE] [--list-rules] [--list-target-rules] [--dag] [--rulegraph] [--filegraph] [--d3dag] [--summary] [--detailed-summary] [--archive FILE] [--cleanup-metadata FILE [FILE ...]] [--cleanup-shadow] [--skip-script-cleanup] [--unlock] [--list-changes {params,input,code}] [--list-input-changes] [--list-params-changes] [--list-untracked] [--delete-all-output | --delete-temp-output] [--keep-incomplete] [--drop-metadata] [--deploy-sources QUERY CHECKSUM] [--version] [--printshellcmds] [--debug-dag] [--nocolor] [--quiet [{all,progress,rules} ...]] [--print-compilation] [--verbose] [--force-use-threads] [--allow-ambiguity] [--nolock] [--ignore-incomplete] [--max-inventory-time SECONDS] [--latency-wait SECONDS] [--wait-for-files [FILE ...]] [--wait-for-files-file FILE] [--notemp] [--all-temp] [--unneeded-temp-files FILE [FILE ...]] [--keep-storage-local-copies] [--target-files-omit-workdir-adjustment] [--allowed-rules ALLOWED_RULES [ALLOWED_RULES ...]]  [--target-jobs TARGET_JOBS [TARGET_JOBS ...]] [--local-groupid LOCAL_GROUPID] [--max-jobs-per-second MAX_JOBS_PER_SECOND] [--max-status-checks-per-second MAX_STATUS_CHECKS_PER_SECOND] [--seconds-between-status-checks SECONDS_BETWEEN_STATUS_CHECKS] [--retries RETRIES] [--attempt ATTEMPT] [--wrapper-prefix WRAPPER_PREFIX] [--default-storage-provider DEFAULT_STORAGE_PROVIDER] [--default-storage-prefix DEFAULT_STORAGE_PREFIX] [--local-storage-prefix LOCAL_STORAGE_PREFIX] [--shared-fs-usage {input-output,persistence,software-deployment,source-cache,sources,storage-local-copies,none} [{input-output,persistence,software-deployment,source-cache,sources,storage-local-copies,none} ...]] [--scheduler-greediness SCHEDULER_GREEDINESS] [--no-hooks] [--debug] [--runtime-profile FILE] [--mode {subprocess,remote,default}] [--show-failed-logs] [--log-handler-script FILE] [--log-service {none,slack,wms}] [--job-deploy-sources] [--container-image IMAGE] [--immediate-submit] [--jobscript SCRIPT] [--jobname NAME] [--flux] [--software-deployment-method {apptainer,conda,env-modules} [{apptainer,conda,env-modules} ...]] [--container-cleanup-images] [--use-conda] [--conda-not-block-search-path-envvars] [--list-conda-envs] [--conda-prefix DIR] [--conda-cleanup-envs] [--conda-cleanup-pkgs [{tarballs,cache}]] [--conda-create-envs-only] [--conda-frontend {conda,mamba}] [--use-apptainer] [--apptainer-prefix DIR] [--apptainer-args ARGS] [--use-envmodules] [targets ...]

snakemake: error: unrecognized arguments: --cluster-cancel=scancel --cluster=/home/bourn/.config/snakemake/embl_slurm/slurm-submit.py --cluster-status=/home/bourn/.config/snakemake/embl_slurm/slurm-status.py

It seems it is not recognising the Slurm config properly but I am not sure how to fix it given that the exact same configuration has worked perfectly many times before, and I'm downloading the Slurm template with cookiecutter (as before).

I notice that the commands for installing sci-rocket are different from what they used to be. Now you are using Micromamba to create a Snakemade enivornment, where before I used the command micromamba create --prefix /g/dorrity/bin/condaEnvs/sci-rocket -f workflow/envs/sci-rocket.yaml to create a sci-rocket environment. I am guessing this is where my problem is. Do you perhaps have any insight into what might be going wrong? Any ideas would be appareciated!

Best, Jess

odomlab2 / sci-rocket

Snakemake now failing with Slurm profile #23