Allow large blends. Add error fields

esheldon commented 3 months ago

set the deblend config to allow large blends
set afw_table.CoordKey.addErrorFields(schema) to avoid getting spammed by warnings

This deblending change is important. It should result in a lot more detections in blended regions, and it potentially requires a full retesting.

We might be happy retesting in "no warp" mode, direct to coadd simulations, which would speed it up by a factor of perhaps 2.5

esheldon commented 3 months ago

It seems detection may have changed, either due to stack or something we did. Previously objects were detected in zero weight (inf variance) regions but not now.

This test test_lsst_zero_weights might need to change to look for one less detection.

esheldon commented 3 months ago

The lsst-tests metacal exposures test will fail until the new ngmix shows up on conda forge

esheldon commented 3 months ago

summary of additional changes

the change to detection was in the stack, defaults in config changed
set all detection config values explicitly
make code for metacal images consistent with ngmix (bug fix)
make sure no detection in areas marked BRIGHT
deal with extra sum field in AM, added for rho4

esheldon commented 1 month ago

@beckermr can you have a look at this one?

beckermr commented 1 month ago

Shall we merge @esheldon?

arunkannawadi commented 1 month ago

I saw a resolved conversation after I posted this comment and understood the motivation.

On Wed, Aug 28, 2024, 1:01 PM Matthew R. Becker @.***> wrote:

@.**** commented on this pull request.

In metadetect/lsst/tests/test_lsst_dm_configs.py https://github.com/esheldon/metadetect/pull/218#discussion_r1735021017:

'tempLocalBackground',

'doTempLocalBackground',

'tempWideBackground',

'doTempWideBackground',

'nPeaksMaxSimple',

'nSigmaForKernel',

'statsMask',

'excludeMaskPlanes'

]

config = SourceDetectionConfig()

keys = config.toDict().keys()

for key in keys:

assert key in expected_keys, f'found unexpected config key {key}'

The point is that if something changes, we need to understand what that change is and ensure it works. We shouldn't simply let things float.

— Reply to this email directly, view it on GitHub https://github.com/esheldon/metadetect/pull/218#discussion_r1735021017, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABG56HRODSPWNO5VHD6DNNDZTX643AVCNFSM6AAAAABJ76S5U2VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDENRWHAZDKNJZHA . You are receiving this because you commented.Message ID: @.***>

esheldon / metadetect

Allow large blends. Add error fields #218

@.**** commented on this pull request.