Assess the DM full focal plane imaging capabilities

[cwwalter]

We need to talk to the appropriate DM people to understand if full focal planes can be assembled, co-adds can be generated for full focal planes, and if photometry across multiple-sensors works.

[jchiang87]

I think we'll need full focal plane sims at some point to test out this functionality. @cwwalter Do you know of any that we could use? If not, I've asked Tom Glanzman about running a couple of r-band visits from the Twinkles Run 1.1 selection. I can generate the instance catalogs for those. Any reason we shouldn't use the physics overrides here as for Twinkles Run1.1?

[cwwalter]

Sorry... I don't know of any full focal plane example data sets. @danielsf you don't know of any do you?

I think no problem using twinkles output for exploring this.

[danielsf]

If any full-focal plane images were simulated, it would have been back in 2012 for an AAS poster. I would recommend re-running the simulations, regardless, to capture all of the development that has been done since then.

[cwwalter]

Any status report on this @jchiang87? Do you need others to help?

[jchiang87]

We don't have full focal plane data yet, but Tom is working on it. I did run the Level 2 pipeline tasks on 5 visits of the R22 raft (comprising 9 sensors). Operationally, everything worked. Since the Stack code divides everything into overlapping "patches" (for these data, there happens to be one patch per sensor) and then treats each patch as a separate coadd, I don't think anything will be different operationally when the doing the full focal plane except scaling up by the relative number of patches (guessing around a factor of 20). So as far as just running the Level 2 pipeline on the full focal plane goes, I don't see any problems.

I do see a couple anomalies, however. Using the default focalplanelayout.txt file, one gets ~100 pixel gaps between the sensors. Here's one visit that shows the gaps:

This is in addition, of course, to the problem with the serial and parallel transfer directions being swapped. The readout direction issue isn't relevant here since I'm running on eimages (I have no idea what to do with post-readout data, so if someone wants to tackle that, then that would be good). The 100 pixel gaps aren't necessarily a problem regarding the science, since we will be masking the edge rolloff regions anyways, but we should try to model the actual focal plane configuration if possible, and these gaps are bigger than the proposed edge roll off masks.

The second anomaly is that I'm seeing inconsistent photometry for brighter objects. Here is the photometric repeatability plot that we had been making for Twinkles:

Having stdev(flux)/median(flux) >> 0.5 mag seems really bad to my uninformed eye. Looking at a handful of the ones with the largest variability, they appear to result from a combination of source position offsets and variation induced by the saturation wings rotating with field position angle. I'm working on getting some examples together showing this.

[sethdigel]

LCA-13501 describes a pixel coordinate system for the full focal plane based on LSST-13381 (LSST Camera Detector Plane Layout.

Table 8 of LCA-13501 has a summary of various dimensions for e2V and ITL layouts. In a raft of ITL sensors the gaps are nominally 27 pixels between sensors, although the active area of the a sensor is of course smaller than its physical size. In pixel units, the physical size of an ITL sensor is 4198x4198. So in the x direction* the gap between active areas of adjacent sensors is 4198 - 4000 + 27 = 225 pixels. In the y direction the gap is 4198 - 4072 + 27 = 153 pixels. This is not so far off from the Phosim default. There's a 26 pixel gap between the edges of the outer sensors of a raft and the physical edge of the raft, and the rafts have a pitch of 12700 x 12700 pixels.

Based on LCA-13501 I think I could make a version of focalplanelayout.txt that corresponds to the layout in LSST-13381, if there's any chance that it would be used.

* This is the x direction in the Camera coordinate system, which is rotated by 90 deg from the serial direction of the CCDs.

[jchiang87]

Here are a couple of examples of objects associated with the saturation wings of bright objects that have the variability I describe: These movies are amimations of the 10x10 arcsec cutouts of the warped visit images that were used for the force photometry. The red point in each image is the location of the object in question (using the coadd catalog position), and the red point and vertical dotted line in the associated light curve plot are synced with the cutout that's being displayed.

I'll look deeper into the objects with excess variability to see if I can find any that aren't of this type, but if there aren't any and given that the gaps between chips are basically what we expect, I'd say there aren't any show-stoppers to running the Stack on the full focal plane.

[cwwalter]

@jchiang87 Great. Thanks for this. If you confirm as above "there aren't any show-stoppers to running the Stack on the full focal plane" and you think we are at the point we can start to do tests can you go ahead and close this issue?

[jchiang87]

I can't think of any other tests I can do at this point. I'll assume that proceeding with eimages will be sufficient for DC1, though I'm sure we will want to be able to run the actual ISR code at some point.

[cwwalter]

@jchiang87 You are running the stack on the e-images now right? Our plan is to only turn on saturation and blooming since that is all we are confident in the stack now for correction. Do you know if those are handled as part of ISR or in detection and measurement?

[jchiang87]

I'm running v12_0 of the Stack. From what I can tell, it is applying a saturation correction in the processEImage.py task. See here and here. Presumably, it would also be handled in the normal ISR processing, i.e., also prior to detection and measurement. @SimonKrughoff may wish to comment.

[cwwalter]

Ah, OK I didn't realize there was a "e-image ISR task".

I guess we will be fine then.

[jchiang87]

@slosar @cwwalter I've uploaded the "Level 2" outputs from the Stack from this investigation to NERSC. The data are in

/global/project/projectdirs/lsst/phosim_deep/feasibility_study/single_raft

Everything is there except for the original phosim eimages, which I don't think you would want to use. The calibrated images are available in the output_repo/calexp subdirectory. There are five visits and I ran the full Twinkles Level 2 pipeline, generalized to do a full raft or focalplane. I put a set up file for edison and example script using the data butler, which should verify that everything is set up ok. Here is how I would set up and run it:

[edison10] bash --rcfile edison_setup.sh 
setting up lsst_apps
[edison10] python butler_test.py 
CameraMapper: Loading registry registry from /global/project/projectdirs/lsst/phosim_deep/feasibility_study/single_raft/output_repo/_parent/registry.sqlite3
CameraMapper: Loading registry registry from /global/project/projectdirs/lsst/phosim_deep/feasibility_study/single_raft/output_repo/_parent/registry.sqlite3
1414156 2,2 0,0
1414156 2,2 0,1
1414156 2,2 0,2
1414156 2,2 1,0
1414156 2,2 1,1
1414156 2,2 1,2
1414156 2,2 2,0
1414156 2,2 2,1
1414156 2,2 2,2
1648025 2,2 0,0
1648025 2,2 0,1
1648025 2,2 0,2
1648025 2,2 1,0
1648025 2,2 1,1
1648025 2,2 1,2
1648025 2,2 2,0
1648025 2,2 2,1
1648025 2,2 2,2
1668469 2,2 0,0
1668469 2,2 0,1
1668469 2,2 0,2
1668469 2,2 1,0
1668469 2,2 1,1
1668469 2,2 1,2
1668469 2,2 2,0
1668469 2,2 2,1
1668469 2,2 2,2
1973403 2,2 0,0
1973403 2,2 0,1
1973403 2,2 0,2
1973403 2,2 1,0
1973403 2,2 1,1
1973403 2,2 1,2
1973403 2,2 2,0
1973403 2,2 2,1
1973403 2,2 2,2
921297 2,2 0,0
921297 2,2 0,1
921297 2,2 0,2
921297 2,2 1,0
921297 2,2 1,1
921297 2,2 1,2
921297 2,2 2,0
921297 2,2 2,1
921297 2,2 2,2

The script just loops over the datarefs and prints visit, raft, and sensor id for each. I can offer some help in using these data, but I'm not an expert, so detailed questions probably should be posted some place like https://community.lsst.org/

[slosar]

@fjaviersanchez Javier, do you want to have a look at these outputs to see if you understand everything, or maybe anything at all to start with? :)

[slosar]

@jchiang87 @cwwalter What is the difference between eimages and "postreadout" data? Are eimages what an ideal CCD would see? Presumably we want to use the postreadout data for actual DM processing, no?

[jchiang87]

eimages still have sensor effects like saturation, as well as, potentially, brighter-fatter, tree rings etc.. They lack things like crosstalk, dark current, read noise, and cte. I understand that the main reason we have been using eimages is that the Stack doesn't have full ISR yet for phosim images. You would have to inquire with DM about that. However, except for read noise, I think the other post readout effects can be taken out in ISR effectively. The read noise acceptance spec for LSST sensors is <8 e-/pix whereas sky background in r is several hundred, typicallly.

[cwwalter]

As Jim says you can think of the e-image files (which is really the electron signal before the electronics) as being the result of ISR being perfectly applied to the normal output.

Effects that act on the electrons (saturation, BF, tree-rings etc) will also be in the e-image file if we turn them on.

Because of the current state of DM ISR corrections we have decided to look at e-image files and basically only turn saturation on. Things like BF correction are not yet at the point we can use them. There is a still some ISR to handle the full well etc (I think + somethings like CRs) when the stack is run.

Twinkles is using the e-image files now also. If ISR works well in the future all of the gains etc will be equalized and the outcome should look similar. Does that make sense?

[slosar]

Ok, great, we just need to keep track of these things, down to the details.

[cwwalter]

Yes, these options are being tracked in the LaTeX file in this directory. We should update add more detail as necessary. Please feel free to contribute!

[jchiang87]

@slosar I put the instance catalogs (these are created by catsim and contain the sources) used for these 5 phosim runs in

/global/project/projectdirs/lsst/phosim_deep/feasibility_study/single_raft/phosim_inputs

Note that the sources in all five are identical, only the phosim commands, setting the observing conditions, etc., differ.

[fjaviersanchez]

@jchiang87 Thanks for putting this together! @slosar: I am downloading this and I'll take a look at these catalogs.

[fjaviersanchez]

I was just playing around with the catalogs and I compared the input sources (red +) to the detected sources (blue x). The axes are just the pixel numbers in the X and Y axes. I represented a small region of the file calexp-0-0,0.fits. The number of detections looks kind of low but I'll check it again (I expected something around 40% of the total input sources since CatSim is deeper than the actual data). I have one question:

Will we get the information about DC1 inputs in this format or in a different one? This format would work for us but, maybe we would also want some extra information about colors without having to do a query in the CatSim DB, it only has MAG_NORM: The normalization of the flux of the object in AB magnitudes at (500 nm)/(1+z) (which is roughly equivalent to V (AB) or g (AB)).

I can post somewhere the script that I used to generate that plot, if needed.

[fjaviersanchez]

Same plot but now distinguishing between input stars (red +) and input galaxies (cyan +). Detected sources are still blue x.

[jchiang87]

We should be able to provide the magnitudes for each source in the given band.