Create individual-image lightcurves for 33 DR1 supernova

[wmwv]

1. [ ] ~Assemble lightcurves from calexp and src files.~
2. [x] Run forced-position photometry based on coadd position for SN.
3. [ ] ~Run forced-position photometry based on RA, Dec position.~
4. [x] Compare these lightcurves with DR1 lightcurves. Remember Vega<->AB

[wmwv]

Had to re-run forced-photometry at coadd positions because I hadn't saved the updates to dr1_coadd_actual.list with the new set of field, tract numbers. These were used to generate the makeDiscreteSkyMap.py, I just hadn't save them clearly.

This fragility of field-tract mapping will likely persist as an issue. The LSST code really wants to use tract and I really want to think in terms of field.

[wmwv]

sn_lightcurve.py is now smart enough to look up tract numbers and dataIds for a given field. Lightcurves are being produced and visualizing the SN location on the coadds and the catalogs verify that things look reasonable.

The actual lightcurves, however are of somewhat questionable quality. There's the issue of getting more images included #16 , that may in tern improve the calibration of those images that are currently being processed. But that's somewhat separate. I should likely next generate lightcurves for 2MASS stars and verify their stability and compare to the correct (AB) magnitudes.

[wmwv]

Two fields have 0 objects in their forced-src calexp catalogs

PTF11qzq
SN2012go

The key errors for PTF11qzq were:

[...]
forcedPhotCcd.references INFO: Getting references in {'tract': 28, 'patch': '0,0'}
forcedPhotCcd FATAL: Failed on dataId={'field': 'PTF11qzq', 'tract': 28, 'filter': 'H', 'seq': 'A', 'night': 20121007, 'expnum': 408}: TaskError: Reference {'tract': 28, 'patch': '0,1'} doesn't exist
[...]
forcedPhotCcd.references INFO: Getting references in {'tract': 28, 'patch': '0,0'}
forcedPhotCcd FATAL: Failed on dataId={'field': 'PTF11qzq', 'tract': 28, 'filter': 'J', 'seq': 'A', 'night': 20130925, 'expnum': 475}: TaskError: Reference {'tract': 28, 'patch': '0,1'} doesn't exist

SN2012go:

root INFO: Running: /ssd/lsstsw/stack3_20171021/stack/miniconda3-4.3.21-10a4fa6/Linux64/meas_base/15.0-2-g2010ef9+6/bin/forcedPhotCcd.py /home/wmwv/SweetSpot/DR1_data/repo/test_dr1 --rerun forcedPhot --id field=SN2012go tract=59 filter=H^J --clobber-versions
forcedPhotCcd.references INFO: Getting references in region with corners ((340.5, 35.01)), ((340.5, 34.932)), ((340.39, 34.932)), ((340.39, 35.01)) [degrees]
forcedPhotCcd.references INFO: Getting references in {'tract': 59, 'patch': '0,0'}
forcedPhotCcd FATAL: Failed on dataId={'field': 'SN2012go', 'tract': 59, 'filter': 'H', 'seq': 'A', 'night': 20121125, 'expnum': 121}: TaskError: Reference {'tract': 59, 'patch': '0,1'} doesn't exist
forcedPhotCcd.references INFO: Getting references in region with corners ((340.51, 35.008)), ((340.51, 34.936)), ((340.42, 34.936)), ((340.42, 35.008)) [degrees]
forcedPhotCcd.references INFO: Getting references in {'tract': 59, 'patch': '0,0'}
forcedPhotCcd FATAL: Failed on dataId={'field': 'SN2012go', 'tract': 59, 'filter': 'H', 'seq': 'A', 'night': 20121130, 'expnum': 207}: TaskError: Reference {'tract': 59, 'patch': '0,1'} doesn't exist
forcedPhotCcd.references INFO: Getting references in region with corners ((340.51, 35.005)), ((340.51, 34.933)), ((340.42, 34.933)), ((340.42, 35.005)) [degrees]
forcedPhotCcd.references INFO: Getting references in {'tract': 59, 'patch': '0,0'}
forcedPhotCcd FATAL: Failed on dataId={'field': 'SN2012go', 'tract': 59, 'filter': 'J', 'seq': 'A', 'night': 20121125, 'expnum': 125}: TaskError: Reference {'tract': 59, 'patch': '0,1'} doesn't exist
forcedPhotCcd.references INFO: Getting references in region with corners ((340.51, 35.01)), ((340.51, 34.937)), ((340.42, 34.937)), ((340.42, 35.01)) [degrees]
forcedPhotCcd.references INFO: Getting references in {'tract': 59, 'patch': '0,0'}
forcedPhotCcd FATAL: Failed on dataId={'field': 'SN2012go', 'tract': 59, 'filter': 'J', 'seq': 'A', 'night': 20121130, 'expnum': 223}: TaskError: Reference {'tract': 59, 'patch': '0,1'} doesn't exist

[wmwv]

So these fields likely span two patches somehow. Will flag here to address later: #25

[wmwv]

Runs and generates 31 lightcurve files (which are how many fields had successful processCcd->coadd LSST reductions and were in the DR1 paper SN lightcurve list):

. run_all_calexp_lightcurves.sh > all_calexp_lighcurves.log 2>&1

[wmwv]

I've added a ecsv_to_Wstd.py to convert to the Wstd.dat format I think in.

I've plotted the lightcurves and they look within relam of being lightcurves, but they're clearly significantly off in calibration and have much less coverage due to missing images from failed processCcd.py runs.

[wmwv]

A run of a lightcurve for P161-D show large scatter of ~0.5 mag in the reported values from the lightcurve.

[wmwv]

Aha! I wasn't generating lightcurves of the correction objects at all to extract_photometry. sn_lightcurve.py now passes the object ID from the caodd deep reference catalog and extract_photometry now matches against this to find the correct row. I had been assuming that the forced-photometry generating rows for every object in the source table, but my guess is that it only generates rows for objects that were on the individual calexp. But the objectId lists the original id from the deep coadd catalog.

[wmwv]

The lightcurves now agree with the DR1 ones to within ~0.2 mag. Notable disagreements

LSQ13cwp  0.5 mag
LSQ13cmt  0.5 mag
iPTF13dkl 0.5 mag (in J)

[wmwv]

Nothing particularly stands out about these. In all three cases the SN is in the outskirts of a galaxy. There's certainly some detectable contribution of galaxy light there, but it's subdominant compared to the SN light.

[wmwv]

Will investigate LSQ13cwp, LSQ13cmt, iPTF13dkl further in #26