SNANA reports extremely faint KN as detected

[deepchatterjeeligo]

Slack discussion: https://lsstc.slack.com/archives/C6FHSS7GW/p1635441741072800

Input file: $SNANA_DEBUG/deep-chatterjee-bulla-debug/SIMGEN_INCLUDE_BULLA_M2_2COMP.NOBATCH.INPUT

An example of detection at redshift of ~0.3

VARLIST: MJD BAND FIELD FLUXCAL FLUXCALERR PHOTFLAG GAIN ZPT PSF SKY_SIG SIM_MAGOBS 
OBS: 63094.0347 u VOID  8.8005e+00  1.001e+01    0  1.000 28.550  2.31 3.150e+00  30.0994 
OBS: 63095.0426 r VOID  3.2192e+00  2.964e+00    0  1.000 32.300  1.85 3.751e+01  27.5353 
OBS: 63099.0503 i VOID  4.0048e+00  4.552e+00    0  1.000 32.070  1.76 4.903e+01  29.2671 
OBS: 63111.1519 z VOID  5.9928e+00  8.681e+00 6144  1.000 32.540  1.64 1.549e+02  33.0027 
TRIGGER:  trigger logic satisfied at MJD_TRIGGER=63111.1523
OBS: 63112.0258 Y VOID  2.2917e+01  2.940e+01    0  1.000 30.060  1.89 4.631e+01  32.3187 
OBS: 63112.9787 Y VOID -9.7717e+00  2.040e+01    0  1.000 31.210  1.63 1.076e+02  32.6717 
OBS: 63113.1637 i VOID -6.5317e+00  5.751e+00    0  1.000 32.120  1.46 7.830e+01  35.3023 
OBS: 63113.9802 z VOID -1.8218e+01  2.528e+01    0  1.000 30.980  3.15 5.582e+01  33.7187 
OBS: 63114.9829 z VOID  1.4493e+01  1.455e+01    0  1.000 31.740  2.32 8.779e+01  34.1492 
OBS: 63116.9791 z VOID -1.3524e+01  1.650e+01    0  1.000 30.990  1.89 6.129e+01  34.4849 
OBS: 63117.0114 g VOID  4.4754e+00  1.947e+00    0  1.000 32.310  1.97 2.324e+01  44.6057 
END_PHOTOMETRY:

[deepchatterjeeligo]

I see a similar case with Kasen models. So I think this is tied to something else. This is an instance from the Kasen model.

# ============================================ 
# TEXT LIGHT CURVE OUTPUT: 
#
NOBS: 8 
NVAR: 11 
VARLIST: MJD BAND FIELD FLUXCAL FLUXCALERR PHOTFLAG GAIN ZPT PSF SKY_SIG SIM_MAGOBS 
OBS: 61118.1968 u VOID -1.8351e+01  1.073e+01    0  1.000 28.640  2.80 3.080e+00  33.8883 
OBS: 61122.0176 u VOID -4.9489e+00  6.809e+00    0  1.000 28.840  2.00 3.290e+00  37.2710 
OBS: 61125.1761 r VOID  2.8968e+00  2.010e+00    0  1.000 33.070  1.83 5.226e+01  32.8651 
OBS: 61129.0087 Y VOID -7.5660e+01  3.421e+01    0  1.000 30.100  2.31 4.581e+01  29.3446 
OBS: 61130.0658 z VOID -4.9603e+01  1.873e+01    0  1.000 31.790  3.01 9.127e+01  30.5067 
OBS: 61132.0635 Y VOID -1.9480e+01  3.133e+01    0  1.000 30.110  2.00 4.890e+01  29.9703 
TRIGGER:  trigger logic satisfied at MJD_TRIGGER=61135.1133
OBS: 61135.1151 i VOID  2.4980e+00  7.440e+00 6144  1.000 32.560  2.44 9.088e+01  33.4630 
OBS: 61136.1519 i VOID -4.8064e+00  4.469e+00    0  1.000 33.090  1.93 1.125e+02  33.4145 
END_PHOTOMETRY: 

[RickKessler]

My hunch is that you are seeing rare random fluctuations combined with a single-detection trigger. Take a look at the detection Efficiency vs. SNR in $SNANA_LSST_ROOT/models/searcheff/SEARCHEFF_PIPELINE_LSST_DC2_Run2.2i_WFD.DAT and note that EFF falls to about 0.001, but not quite zero. We should probably set EFF=0 for SNR < about 1 or 2 to avoid these artifacts, but these artifacts could be real. While EFF ~ 0.001 at low-SNR seems small, there are many epochs so eventually it triggers a detection. The analysis should require a more reasonable SNR (e.g. SNR>5).

[deepchatterjeeligo]

The distribution makes more sense if I add the cut on SNR

CUTWIN_SNRMAX: 5 ugrizY 1 0 20

@RickKessler from examples in the manual, the above will impose at least 1 detection above SNR 5?

[deepchatterjeeligo]

Same as above but with at least 2 detections above SNR 5

[deepchatterjeeligo]

I think the SNR cut makes sense to me and we should impose one. I'll close this issue since I understand the reason why I was getting the faint mags.