strange choice of reference band in Run 1.2i object catalogs

[rmandelb]

I have been using the DPDD-like Run 1.2i and Run 1.2p object catalogs. While investigating some oddities in the total number counts and magnitude distribution of galaxies, I was advised to check the reference band for the galaxies. The results were surprising:

For Run 1.2p, if I check how many galaxies in a particular tract (4849) have ugrizy as their reference band, the result is 335 1037 3759 108119 3235 2116 -- so really mostly i band, which is consistent with what we'd expect for an S/N criterion.

For Run 1.2i, if I do the same test in the same tract, the result is 626 51700 0 0 0 0 -- almost entirely g band, no rizy at all. Based on a S/N criterion, this should not happen. And lots of objects that are perfectly fine should be undetected in u and/or g, so clearly this will severely impact the forced photometry for many objects.

In case it's useful, here are the selection criteria I've been imposing on these object catalogs before computing the numbers above:

I'm also happy to share other plots that may be helpful. @rearmstr had suggested this could occur if there is some issue with the PSF flux S/N in rizy that affects the selection of the reference band, so I computed the ratio of PSF flux to PSF flux error in i-band and found that the distributions in Run 1.2i and Run 1.2p for the objects passing those selection criteria do differ, but not in a way that is catastrophic (e.g., it's not like everything in Run 1.2i has PSF flux S/N<10 or something really bizarre like that):

I'm happy to do other diagnostics if it would be helpful.

[wmwv]

My hypothesis is that the merged detection catalog for the coadd merged detection reference was generated when only the u and g band coadds were available.

1. The merged detection reference catalog should be generated.
2. The forced-source measurements on the coadds and on the per-visit images should be regenerated. @heather999 and @johannct are aware of this and are reprocessing.

[johannct]

Yes, this is a pilot error, and fully understood.

[rmandelb]

@johannct - if this is fully understood, then shall I close this, or would you like me to run my script on new object catalogs when they available and close once the fix is confirmed?

[fjaviersanchez]

@rmandelb as a follow-up. I checked the depth in a single visit in i-band and the depth looks about right (the sky-brightness in this particular visit was ~19.68). I am plotting below the magnitude histogram for visit 204594:

I checked for visits with a background dimmer than 20.2 in i-band but I couldn't find any here: /global/cscratch1/sd/desc/DC2/data/Run1.2i/rerun/20181007_h/.

[johannct]

@rmandelb having the same scripts produce a more correct output on the new dataset is probably best. Should be at NERSC late today CET if @airnandez is available.

[rmandelb]

@fjaviersanchez - that looks better than the distribution of i-band magnitudes I was finding in the coadd object catalogs! Thanks for confirming.

@johannct - thanks. I will await a notification that there are updated object catalogs that are readable by GCR and rerun my script before closing this.

[johannct]

@rmandelb we let that slip over. I believe the CC-processing is available at /global/projecta/projectdirs/lsst/global/in2p3/Run1.2i/w_2018_39. Happy 2019!

[rmandelb]

Thanks @johannct ! I believe the full set of object catalogs for that new processing is not yet ready, but Yao pointed me to a single one for a single tract that I could play with, and when using my default flag cuts etc. the number counts of objects passing the i<25 mag cut increased by 50% with this new processing. The shape of the number counts as a function of magnitude is also closer to reasonable. This is a positive sign, but a more rigorous/quantitative check will require truth catalogs that include extinction, which we don't have yet.

[wmwv]

Addressed and resolved in the in Run 1.2i re-processing. /global/projecta/projectdirs/lsst/global/in2p3/Run1.2i/w_2018_39