search

CosmoStat / autometacal

Metacalibration and shape measurement by automatic differentiation
MIT License
4 stars 1 forks source link

Comparison to NGMIX #17

Closed EiffL closed 3 years ago

EiffL commented 3 years ago

I added a notebook to look at how we do compared to NGMIX. What I'm doing is that I'm making some observations with the ngmix example code, then run metacal on these obs. The input obs give me the input image and PSF, and with the output metacal obs, I have access to the reconvolution PSF.

So, the good news is that the noshear variant seems to be pretty ok: image

The bad news is that I'm seeing some non zero residuals in the sheared variants: image

This doesn't go away easily.... I'm thinking the only part of the code that can introduce these errors is the shearing (given that noshear is ok), and there... I know from experiments with galsim that it may come down to the interpolation method in Fourier space. Unfortunately we only have access to a linear interpolant. When using kspace linear interpolation in galsim and galflow we get close to numerical precision: image but when GalSim uses the quintic interpolant, the errors are several orders of magnitude larger...

So, here is the question... is our linear interpolant a deal breaker here? (in which case we'll need to implement ourselves a different interpolation method) or for the purpose of demonstrating the idea, is linear interpolation good enough for now? Also, still a possibility that the problem can come from elsewhere....

martinkilbinger commented 3 years ago

That looks like a nice dipole in the e2-direction. What do you get for 2p / 2m ?

EiffL commented 3 years ago

here is the 2p residuals image

Also, I have implemented here cubic Fourier interpolation, so I don't think it's a matter of interpolant anymore. With this cubic interpolant, I get close to numerical accuracy results when I just compare the shearing of a profile against galsim (without psf devonv/reconv)

EiffL commented 3 years ago

image Good news everyone, I've tracked down the remaining sources of discrepancies, and now I'm getting super nice agreement between ngmix and our code: image image

The key thing is that we don't yet have the proper tools to deal with off-grid pixel centers, which happen when we use even size images. This is in relation to https://github.com/DifferentiableUniverseInitiative/GalFlow/issues/8 that @b-remy will look into.

Another aspect to this is that we do need the higher order interpolation tools mentioned in https://github.com/DifferentiableUniverseInitiative/GalFlow/issues/10 Using simply a linear interpolant in Fourier space for the 1p shear I get: image I think it's good enough for now, but we see that we get a significant improvement in accuracy if we have access to higher order interpolants.

In the meantime, being very careful with the size of our FFTs and such, I think we can get pretty close to ngmix.

I'll update the demo notebook comparing to ngmix before I forget all the steps needed ^^'

EiffL commented 3 years ago

@andrevitorelli let me know if the demo notebook now makes sense to you: https://github.com/CosmoStat/autometacal/blob/main/notebooks/NgmixComparison.ipynb

You will notice I went back to the basics, doing everything in terms of simple FFT operations, and use odd size images, but I think this is a nice baseline to use for now, until we have more general purpose GalFlow tools.

Note that here we have everything to actually start from the postage stamps themselves, I'm not using GalSim at all to draw kspace image or to do any deconvolution. Let me know if you have any questions about this notebook, or would like me to add anything else.