Change in Snakemake rerun behaviour

[ezherman]

As outlined in this post and the snakemake manual under --rerun-triggers, snakemake >=7.8 now reruns by default after changes in code, parameter values, input files and software environments, in addition to file timestamps.

Consequently, I believe the following statement needs to be updated in episode 4:

"But it doesn’t help us in the situation when rules in the Snakefile change, rather than input files, Snakemake won’t see that the results are out-of-date. For example, if we changed the quality cutoffs within the trimreads rule then Snakemake would not automatically re-run those rules, because it only checks that the output file is newer than the input file."

Perhaps the easiest way forward would be to skip teaching -R, as snakemake should now identify all the cases in which we would have used -R in the course (I think). What are your thoughts Tim?

[ezherman]

There is currently a discussion on the Snakemake discord server on whether to change the default behaviour of --rerun-triggers to something closer to what it used to be. So now is probably not a great time to edit the materials, however learners could be made aware of the current state of --rerun-triggers (which I imagine is not available during the workshop, unless Johnny updates the base conda environment ahead of every workshop run).

[tbooth]

See https://github.com/snakemake/snakemake/issues/2011

[tbooth]

I think this is now dealt with, in that the latest version of the course now deals with the Snakemake 8 behaviour, which I hope is stable. I'm reticent to remove -R from the material. It does make a simpler story to just say that Snakemake will rerun whatever needs running and we don't have to worry, but in practise I have found the -R flag to be useful as the Snakemake change detection is not bullet proof. It's also a nice way to illustrate some ideas covered within ep04.