snakemake / snakemake-executor-plugin-slurm

A Snakemake executor plugin for submitting jobs to a SLURM cluster
MIT License
18 stars 19 forks source link

TypeError: SpawnedJobArgsFactory.general_args() got an unexpected keyword argument 'executor_common_settings' #54

Closed Parkerdow closed 8 months ago

Parkerdow commented 8 months ago

Hi, thanks for taking a look at this.

I'm attempting to use the executor plugin to run a workflow on my cluster via slurm and I'm not sure what exactly is happening. I can see in cli.py the SpawnedJobArgsFactoryExecutorInterface class def has a defined executor_common_settings argument but I'm not sure why its not being passed to my RealExecutor

Trace attached below, happy to add any other information if useful:

Traceback (most recent call last):
  File "/cellar/users/pdow/miniconda3/envs/snakemake/lib/python3.12/site-packages/snakemake/cli.py", line 1978, in args_to_api
    dag_api.execute_workflow(
  File "/cellar/users/pdow/miniconda3/envs/snakemake/lib/python3.12/site-packages/snakemake/api.py", line 588, in execute_workflow
    workflow.execute(
  File "/cellar/users/pdow/miniconda3/envs/snakemake/lib/python3.12/site-packages/snakemake/workflow.py", line 1230, in execute
    raise e
  File "/cellar/users/pdow/miniconda3/envs/snakemake/lib/python3.12/site-packages/snakemake/workflow.py", line 1226, in execute
    success = self.scheduler.schedule()
              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/cellar/users/pdow/miniconda3/envs/snakemake/lib/python3.12/site-packages/snakemake/scheduler.py", line 306, in schedule
    self.run(runjobs)
  File "/cellar/users/pdow/miniconda3/envs/snakemake/lib/python3.12/site-packages/snakemake/scheduler.py", line 391, in run
    executor.run_jobs(jobs)
  File "/cellar/users/pdow/miniconda3/envs/snakemake/lib/python3.12/site-packages/snakemake_interface_executor_plugins/executors/base.py", line 72, in run_jobs
    self.run_job(job)
  File "/cellar/users/pdow/miniconda3/envs/snakemake/lib/python3.12/site-packages/snakemake_executor_plugin_slurm/__init__.py", line 137, in run_job
    exec_job = self.format_job_exec(job)
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/cellar/users/pdow/miniconda3/envs/snakemake/lib/python3.12/site-packages/snakemake_interface_executor_plugins/executors/real.py", line 154, in format_job_exec
    general_args = self.workflow.spawned_job_args_factory.general_args(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: SpawnedJobArgsFactory.general_args() got an unexpected keyword argument 'executor_common_settings'
cmeesters commented 8 months ago

Please indicate:

Parkerdow commented 8 months ago

snakemake: 8.4.12 snakemake-executor-plugin-slurm: 0.4.2

I'm currently using the command: snakemake snakemake --profile test

I believe the profilef folder setup is correct:

.config/snakemake/
└── test
    └── config.yaml

Profile

#slurm settings
executor: slurm
jobs: unlimited
use-conda: true

#default resources
default-resources:
    runtime: 5
    slurm_account: carter-compute
    slurm_partition: carter-co
    mem_gb: 8
    tasks: 2
    account: pdow

However I can recreate the error by skipping the profile and simply calling: snakemake --executor slurm --jobs unlimited --default-resources slurm_account=carter-compute slurm_partition=carter-co

again, thank you for the assistance! I've attached my snakemake venv yaml for full visibility. environment.txt

Parkerdow commented 8 months ago

Apologies Professor, updating from 8.4.12 to 8.10 appears to cleared the problem. Thank you for your continued maintenance of the codebase!

cmeesters commented 8 months ago

@Parkerdow Thank you for the feedback! I could not reply any sooner - I am on a retreat.