I want to better handle a situation where you have processed only some of your data through 0.preprocess-sites/1.process-spots.py (e.g. uncaught exception stops processing half way). For config options - perform either processes or skips the whole module while force_overwrite only happens at the file level (i.e. all of the data processing happens and then it decides to save the files or not) so your only option is to run the whole thing over again if you're missing sites.
I'm thinking it needs another config option that allows you to skip complete sites. If this new option is True, at for site in split sites: it should to see if the output folder exists for that site and if so it skips processing that site.
Might also want to do this for 2.process-cells as well.
I want to better handle a situation where you have processed only some of your data through 0.preprocess-sites/1.process-spots.py (e.g. uncaught exception stops processing half way). For config options -
performeither processes or skips the whole module whileforce_overwriteonly happens at the file level (i.e. all of the data processing happens and then it decides to save the files or not) so your only option is to run the whole thing over again if you're missing sites.I'm thinking it needs another config option that allows you to skip complete sites. If this new option is
True, atfor site in split sites:it should to see if the output folder exists for that site and if so it skips processing that site.Might also want to do this for 2.process-cells as well.