search

esheldon / metadetect

Library for meta-detection, combining detection and metacalibration
Other
12 stars 6 forks source link

pair sims are showing bias #130

Closed esheldon closed 3 years ago

esheldon commented 3 years ago

For separation 1.35 arcsec I see bias with and without deblending

no deblending

0.00475472 < m1 < 0.0135739 (99.7%)

with deblending

0.0635627 < m1 < 0.0767297 (99.7%)
esheldon commented 3 years ago

At 4.0 arcsec things look ok

-5.47526e-05 < m1 < 0.0011639 (99.7%)

all results are for wmom

beckermr commented 3 years ago

I assume this is for lsst?

esheldon commented 3 years ago

Yeah, I've tested this with regular metadetect that uses sep and it has much better accuracy.

esheldon commented 3 years ago

forgot to mention that, with the deblending, the R value is actually negative

So something is going completely off the rails

esheldon commented 3 years ago

I'm seeing a bias running sx on these sims as well. So it has something to do with the simulation data produced

beckermr commented 3 years ago

forgot to mention that, with the deblending, the R value is actually negative

Do you see this with sx?

esheldon commented 3 years ago

No, the R value looks normal for sx and for lsst without deblending

beckermr commented 3 years ago

What bias do you see with SX? I don't think we expect mdet to be perfect for pairs near where the detection rate for two objects is 50%. In a real universe we get to average over many separations.

esheldon commented 3 years ago

0.0024349 < m1 < 0.00695481 (99.7%)

with my other sims I saw a maximum m of about 0.0005

beckermr commented 3 years ago

with my other sims I saw a maximum m of about 0.0005

Which other sims?

esheldon commented 3 years ago

There was the code from the metadetect paper, you know those sims.

I also reimplemented that recently for the pair tests I showed you in slack. But these sims are so simple I doubt there was any substantial differences between those two implementations.

substantial differences in the sims

  1. pixel scale change from 0.263 to 0.2
  2. I used moffat psf for the other sims, gaussian for these
  3. the other sims didn't try to use postage stamps, rather the entire scene was drawn at once
esheldon commented 3 years ago

Also these galaxies are smaller relative to the psf hlr/psf_fwhm of 0.6 rather than 1.1 for the other sims

esheldon commented 3 years ago

I think the bias difference is mainly due to the smaller size of the objects used in the newer sims, hlr=0.5 vs hlr=1.0

I ran the old sim code with small galaxies and I also see a bias.

Here is sx, hlr=0.5, fixed separation 1.35 arcsec but as a function of the input shear, with 3-sigma error bars. (note the sims that started this thread were at 0.02)

m-vs-shear