This is being raised as a note and perhaps a discussion when things are running more routinely.
When optimising the imager dask routine I removed checks that search for images, essentially removing the image_reuse option. Since I am now routinely cleaning up all non-MFS files this check would be fairly useless in most cases.
There is now a --skip_imager option in the configuration parser. However, I am curious on whether there would be a need to add back in this option? if so, the actual thing to do I think would be to scan for the final cubes. If all 36 (or 72) cubes and corresponding weight files do exist, then the entire imager stage could be automatically skipped, otherwise burn it all down and start again.
This is being raised as a note and perhaps a discussion when things are running more routinely.
When optimising the
imagerdask routine I removed checks that search for images, essentially removing theimage_reuseoption. Since I am now routinely cleaning up all non-MFS files this check would be fairly useless in most cases.There is now a
--skip_imageroption in the configuration parser. However, I am curious on whether there would be a need to add back in this option? if so, the actual thing to do I think would be to scan for the final cubes. If all 36 (or 72) cubes and corresponding weight files do exist, then the entireimagerstage could be automatically skipped, otherwise burn it all down and start again.