Way to make logging go to same logs as when running locally?

snakemake / snakemake-executor-plugin-slurm

A Snakemake executor plugin for submitting jobs to a SLURM cluster

MIT License

17 stars 18 forks source link

Way to make logging go to same logs as when running locally? #123

Open hermidalc opened 2 months ago

hermidalc commented 2 months ago

Is there a way to configure the executor so job logs to the log locations specified in my rules? (i.e. just like when running my workflow locally outside of SLURM). So instead of everything going to .snakemake/slurm_logs

cmeesters commented 2 months ago

no, but it has been asked sometimes. I'll add it on the to-do list.

hermidalc commented 2 months ago

In hindsight it actually makes sense to me the way it is, SLURM specific snakemake logging, job info and e.g. restarts/retries etc go into .snakemake/slurm_logs and the regular rule log output of the successful execution of the rule goes into the rule log path. It's less clutter and you don't need to see the rule regular logging from failed execution attempts mixed in with SLURM specific logging if the job needed to restart for any reason.

cmeesters commented 2 months ago

absolutely, that was the intend. However, the reason, why I leave this issue open is

that the feature was requested multiple times
that i want to feature optional deletion of SLURM outputs (they are mostly redundant and if a workflow suceeds, they can go away as the logging information by Snakemake contains all valuable info, plus the report, etc.)