regression with grid sims

[esheldon]

This sim is now giving 0.15% bias

# same as v1 but now we are saving the mask frac
epochs_per_band: 1

gals_type: "exp"

gals_kws:
    mag: 22

layout_type: "grid"
layout_kws:
    dim: 8

wcs_kws:
    dither_range: [-0.5, 0.5]
    position_angle_range: [0, 360]
    scale_frac_std: 0.01
    shear_std: 0.01

psf_type: "ps"
psf_kws:
    variation_factor: 1.0

[esheldon]

bias goes away with gauss psf

[beckermr]

ah - we have to stack enough PS PSFs to reduce the PSF variation

how many epochs?

[beckermr]

3 is too few IIUIC

[esheldon]

we already tested this before, so this is a regression

[esheldon]

note variation_factor: 1.0

[beckermr]

Are you sure?

[esheldon]

The only difference was that the other test used brighter objects.

[esheldon]

Here is the old result with the ~17.5 mag exp gals

5.46647e-05 +/- 0.000511308 (3 sigma) 

[esheldon]

New results mag=22 objects (still high s/n)

0.0012705 < m1 < 0.00178272 (99.7%)

[beckermr]

Does the new code work with brighter objects?

[esheldon]

not to mention, we tested PS psf with full wldeblend galaxies and it worked perfectly

[esheldon]

Well I'll be damned, it did work for the brighter objects

0.000262936 < m1 < 0.000404232 (99.7%)

[beckermr]

Wow

[esheldon]

I had run the brighter one with wider grid spacing. Running the faint one with wider grid spacing also works

-4.08625e-05 < m1 < 0.00044735 (99.7%)

[esheldon]

wider is 6x6 grid rather than 8x8 grid

[esheldon]

Well, I decided to burn some time verifying this is OK with random placement of the galaxies, and it looks ok

0.000151968 < m1 < 0.000755387 (99.7%)

so I think the bias for the close packed grid was in fact due to the close spacing, such that every single object was blended and the blending was of a similar degree and character in most of the cases (different near the edge of the grid)

I can't say I really understand it, but I don't see how it could affect a realistic scenario.

[esheldon]

I ran with small galaxies, hlr=0.1, mag 17.5 on a 6x6 grid and did get some bias.

    0.000441331 < m1 < 0.000957044 (99.7%)

So this I think connects back to the bias we were seeing in the gals+stars run with PS/moffat psf. There are a lot of small galaxies in the wldeblend sims. Seems to be an issue of moffat + small galaxies. There may be some additional bias due to having stars.

My intuitioin is that the bias is from deconvolutions/reconvolutions with the moffat. I do see artifacts for very bright stars with the moffat PSF and psf stamp size of 53x53, but I had hoped it was only a problem if there were really bright stars in the image. But I think it bites us for small galaxies too

[beckermr]

So fitgauss is too aggressive and needs to widen the PSF a bit more?

[esheldon]

Maybe.

I see the artifacts for psf: gauss too...

[esheldon]

If I run with variation_factor: 0.001 the bias goes away

[esheldon]

So I think your guess might be correct, that we need more epochs.

[beckermr]

which test? the original one?

[esheldon]

I ran the test with mag 17.75, hlr 0.1 but with variation_factor 0.001 rather than 1

the bias for variation_factor 1 I got

    0.000441331 < m1 < 0.000957044 (99.7%)

with variation_factor 0.001 I got

-4.69865e-05 < m1 < 0.000262134 (99.7%)

[esheldon]

Running with 3 epochs and variation factor 1

2.17194e-05 < m1 < 0.000422611 (99.7%)

[beckermr]

Great! I'd say this is solved?

[esheldon]

I think so.

[esheldon]

It was wrong to call it a regression; I don't think we specifically tested some of these scenarios before

[beckermr]

No worries!

[esheldon]

Is it possible to mock up the effect of 3 epochs by modifying the variation_factor?

[beckermr]

OFC, but IDK how to calibrate that except to do actual sims

[beckermr]

This PS PSF "model" is VERY crude. It is a way to generate variation quickly. I cannot promise it relates to other things much more than this.

[esheldon]

Before we do any more runs with PS psf I think we'll need to calibrate this some how.

[beckermr]

Yup. And we should figure out if the correlation structure is right. I built that to show that in principle PSF variation can be averaged down.