Ring test, correct image center, enable write PSF object to pickle, record flux zero point in output exposure

[mr-superonion]

1. Enable ring test to reduce error on additive bias (4x 45 degree pairs);
2. Enable dumping PSF object to disk (for reuse);
3. record magnitude zero point in the output exposures;
4. The center of the image was at ndim/2-1 for an image with indexing 1,2..ndim (that is the Gaism indexing). Now fix it to ndim/2+1. For example, a length 3 array, with indexing 1,2,3, the center should be at 2=(3+1)/2.

[esheldon]

Note rather than ring test we use the noise cancelling method of Pujols. Note in the mdet-lsst-sim repo I use both a +g1 and -g1 and store both in the output files

[mr-superonion]

Yes! I think it is better to use both together! (and this is normally what I did) since ring test is very efficient in reducing error on additive bias.

[esheldon]

We need to run a full sim test to show this didn't introduce any shear biases.

[esheldon]

Please don't lint with black. It makes pointless changes (e.g. single to double quotes) and creates conflicts with the flake8 we use for CI

[mr-superonion]

Please don't lint with black. It makes pointless changes (e.g. single to double quotes) and creates conflicts with the flake8 we use for CI

wow, I am wondering Is there any software for the auto linting using the flake8 standard? (fixing the code with one command like black..)

[esheldon]

I don't auto-format my code, I run flake8 within the editor and make the changes by hand.

[mr-superonion]

We need to run a full sim test to show this didn't introduce any shear biases.

Yes!!! Let me make a test for shear estimation with 4 noiseless galaxies (45 degree rotated; each with center offset and the offsets are also rotated; but fix the center of the measurement to the center of distortion/ rotation)

[esheldon]

This might be covered by the CI with shear_meas_tests subdir

[mr-superonion]

Note, the shear estimation test also makes sure that the distortion/ rotation centers are located at the center of the image array, which is the center of the measurement. Otherwise, the shear estimation results are biased!

[beckermr]

What is biased?

[mr-superonion]

shear is biased if the measurement center is not the same as the center of the distortion/ rotation for this test! see here: https://github.com/mr-superonion/descwl-shear-sims/blob/1c2a47496c621b125f57f107d82172e6ae309729/shear_meas_tests/test_shear_meas_ringtest_metacal.py#L93 The measurement center is defined at the center of the image.

[beckermr]

We've done tons and tons of simulations without this PR and found no shear bias. Under what conditions are you talking about? Are you not detecting objects?

[mr-superonion]

This unit test is a special case, which is different from the tests you have done ---- I am using only 4 noiseless galaxies for ring test (each galaxy has an offset to image center and then rotated by i \times 45 degree (i=0,1,2,3) wrt. image center) and run metacal (no detection is run). This test is to make sure that using 4 galaxies successfully removes shape noise, and also make sure that the shear estimator works correctly for the noiseless ring test and the center of symmetry is known. (You know that we can fully remove shape noise with 4 galaxies rotated by 45 degree if the center of measurement is the same as the center of the rotation (assuming that weighted ellipiticity is spin-2, which is generally correct.) Note, the center of the rotation is the center of symmetry. That is, the images are free of spin-4 anisotropy wrt. this center for these four images.)

Since no detection is run and each galaxy has center offset from the image center; in the shear measurement, we must force the center of the measurement to the center of the rotation, otherwise we will find a multiplicative bias due to the offset of the measurement center from the rotation center. However, this is because that the ring test does not fully remove shape noise with respect to the measurement center (which is not the same as the rotation center..) I call this multiplicative bias centroid offset bias.

However, if we run the full detection center estimation, and include the shear response of the detection and center estimation, this centroid offset bias is included and corrected in the shear response of detection !

[esheldon]

Where do we stand on this PR?

[mr-superonion]

Where do we stand on this PR?

Sorry I have been busy with some other projects. I am waiting for the review from you and Matthew!!

[esheldon]

can you point that out?

[esheldon]

Ahh, ok. I thought you meant a breaking change.

[beckermr]

Right. Not a breaking change but could be broken in some cases.

[esheldon]

Yes, it needs to be passed here

https://github.com/LSSTDESC/descwl-shear-sims/blob/230922ad3f381fcb13f2a357187c39f26f951f45/descwl_shear_sims/galaxies.py#L81

[esheldon]

@mr-superonion can you pass on that keyword indicated above?

[mr-superonion]

I think now it is consistent!!

[esheldon]

Thanks for this PR, I think this will be very useful.