Caching image space products

[bennahugo]

Surely this should not make it fall over. Instead if the files don't exist they should be stored under these names

[bennahugo]

Better yet this should be changed to a single prefix parameter as in other imagers? If the file exists and is of the desired shape skip unless forced override?

[landmanbester]

I did initially think that it should fall over if you supply --psf or --dirty with teh wrong headers but I have modified things (cleanup_options branch) so that it will compute (or recompute) them if either the file does not exist or the headers do not match. I do not want to give all the outputs the same file prefix because sometimes (actually very often) I want to rerun with a different optimisation settings that do not affect the dirty or the psf. The same applies to the --x0 and --first_residual options. I should probably add all the relevant information to the fits headers and try to check this more carefully

[landmanbester]

This is really a caching issue. We need to be able to cache products which are expensive to compute (eg. dirty, psf, interpolated beam) so that we can reuse them when possible. The thing that makes this slightly non-trivial is figuring out when the weights or calibration solutions have changed as this necessitates re-computing the dirty image and psf

[landmanbester]

This is now handled in the separate grid worker which computes all data products and writes them to a zarr dataset. In addition, all deconvolution products are written to a separate zarr dataset and all operations performed by deconvolution workers should be reversable