search

mhardcastle / ddf-pipeline

LOFAR pipeline using killms/ddfacet
GNU General Public License v2.0
24 stars 20 forks source link

Apply facet offsets to images during mosaicing #353

Closed twshimwell closed 3 weeks ago

twshimwell commented 5 months ago

Related to issue #351 we need to redo the astrometric corrections as previously poor quality corrections have been applied. Going back to the uv-data for redoing the corrections is prohibitively expensive and thus we need to do it on the images.

An example of applying shifts to an image and having some padding around the facets to avoid creating gaps is in the offsets_compact branch script https://github.com/mhardcastle/ddf-pipeline/blob/offsets_compact/scripts/apply-astrometric-shifts.py. However, we intend to incorporate this shifting into the mosaicing allowing us to avoid doing two regridding steps on the images. So mosaicing needs to correct the astrometry (based on output of offsets.py - #351) of each facet whilst doing the reprojection.

Might as well do this for both low and high resolution images.

mhardcastle commented 4 months ago

A testable version of this is in the offsets_compact branch. mosaic.py and mosaic_pointing.py have been updated along with a bunch of other changes. I am reasonably confident that the shifts are applied in the correct sense (-:

Test with a command line like:

mosaic_pointing.py --do_scaling --directories /scratch/mjh/pointing-images --apply-shift --no-bdsf P121+52

It takes about 20 minutes to run, a little longer if the per-facet noise maps need to be made.

So far this does not do any identifying of bad shifts or bad facets as discussed in #352.

mhardcastle commented 4 months ago

Oh, this works (and should be applying the shifts) for the 20-arcsec mosaics as well.

Because it's parallelized over facets for speed, it needs a few tens of GB RAM for the full-res mosaic making at peak.

twshimwell commented 4 months ago

Oh fabulous! I guess we should test it around dec 90 or RA=360/0 as seems like its easy to go wrong in those regions.

mhardcastle commented 4 months ago

Yes... right now the shifts are applied by adjusting the RA and DEC of the reference pixel in the original image before regridding on to the mosaic. It might make more sense and be safer to adjust the pixel position of the reference pixel, in which case there should be no issue.

mhardcastle commented 4 months ago

OK, so I had a look at the offsets.py findoffsets() code and I think there's no way it's going to work sensibly close to the poles anyway -- it was only designed to be good enough for DR1. To do this properly we would need to choose a projection onto the tangent plane, work in that projection rather than ra and dec to get the offsets, and then be very careful about applying it in exactly the same way in the mosaic code. In other words, don't break out the champagne yet, or run offset finding over the whole sky till we've thought about this...

mhardcastle commented 3 weeks ago

This was fixed and the offsets-compact branch has been merged to master.