search

LSSTDESC / SSim_DC1

Configuration, production, validation specifications and tools for the DC1 Data Set.
4 stars 2 forks source link

Validate full focal plane output #12

Closed cwwalter closed 7 years ago

cwwalter commented 8 years ago

Repeat the exercise of #11 with full focal plane output.

cwwalter commented 7 years ago

Issue #7 was closed in August after @jchiang87 determined that DM should work properly with fill focal planes. But, it is now being used by @fjaviersanchez and others to discuss validation of the test output of the pipeline provided by @jchiang87.

That should be discussed in this issue and in issue #11.

I forgot the other issue was closed and spent 5 minutes trying to find it this morning. Feel free to continue the discussion here, or to close these and reopen #7 and continue the discussion there. If you do reopen the other one add the DC1 validation milestone to the issue.

jchiang87 commented 7 years ago

The discussion on these sorts of tests is being continued at the desc-lss channel

cwwalter commented 7 years ago

The discussion on these sorts of tests is being continued at the desc-lss channel

OK. All: Please be sure to at least summarize here for posterity and then close this when appropriate. I joined the channel so I can follow.

jchiang87 commented 7 years ago

I've been meaning to follow up on the comparison of input vs observed magnitudes that Javier posted on Nov 6 for the PhosimDeep precursor data, which were generated with phosim v3.4.x (see https://github.com/LSSTDESC/SSim_DC1_Roadmap/issues/11#issuecomment-258301455), so here's what I have so far.

Like Javier, I've used a kdtree to get positional associations between the object entries in the instance catalogs and the objects found by the execution of our Level 2 pipeline implementation using v12_0 of the LSST Stack. Here are plots for the central chip (S11) of raft R22 (the only raft simulated in the precursor data): v1414156-fr_r22_s11_comparison

There seems to be a rotation in the coordinates for the measured positions vs the instance catalog values. Here is the offset plot for the full raft: v1414156-fr_r22_offsets

Looking at the Twinkles Run3 data (phosim v3.5.3, Stack v12_1), I'm seeing a similar pattern for the offsets, though they are much larger (by about a factor of ~10): v230-fr_offsets Note that Twinkles only simulates the central chip, R22_S11. I really haven't tried to chase down the origin of this rotation, but it seems to coming from the Stack since an overlay of the instance catalog positions match the phosim eimage sources fairly well, and I'm seeing the same thing when I analyze imSim-generated data.

fjaviersanchez commented 7 years ago

Thanks for this @jchiang87! For the unmatched objects I found that they can be not only CRs but also just fake detections. These sources appear especially close to very bright sources. I am surprised about the rotation, I didn't expect to see such a large effect. I don't have a clear idea of what it can be for the rotation. What positions/centroids are you using to get the differences between the input and the measured sources? base_Sdsscentroid_x/y or others (RA/DEC and then converted to x,y via WCS)?

jchiang87 commented 7 years ago

For the measured positions, I'm using the coord_ra and coord_dec values in the forced source catalogs, e.g., in /global/project/projectdirs/lsst/phosim_deep/feasibility_study/single_raft/output_repo/forced/0/v1414156-fr/R22/S11.fits. I think these should be the same object coordinates as in deepCoadd-results/merged, but I actually haven't verified that. For the comparisons, I'm using the ra, dec values directly, i.e., not doing any conversion to x, y.

Another thing to note is that I ran the Level 2 analysis just on the single Twinkles Run3 visit, whereas the PhoSim Deep precursor analysis included 5 visits in the coadds, so that may have something to do with the smaller rotation amplitude for those data.

fjaviersanchez commented 7 years ago

Yes, I think those should be the same but I haven't tested them either. The pattern looks PSF-esque... I would say that one RA/DEC is PSF-convolved and the other isn't. I will check that as well, maybe we can use the centroid measurements (base_Sdsscentroid_x/y were as good as GalSim centroid estimates if I recall correctly), convert them to RA/DEC and compare to the inputs.

jchiang87 commented 7 years ago

Using the WCS from deepCoadd-results/r/0/0,0/calexp-r-0-0,0.fits to convert base_SdssCentroid_[xy] from deepCoadd-results/merged/0/0,0/ref-0-0,0.fits, the RA, Dec values I find are same as in the coord_ra, coord_dec values in that same file.

fjaviersanchez commented 7 years ago

I have no clue then. Maybe it is worth to post something at community.lsst.org?

slosar commented 7 years ago

Yes,stuff like that should really be understood. Offsets are nearly one arcsec at the edges, so clearly some coordinate frame issue, not a PSF bias. I like the idea of asking at the community.lsst.org or even just hassling the correct people on slack.

cwwalter commented 7 years ago

@jchiang87 I looked around and I think this is the issue where you reported on the observed rotation in the coordinate system. Jim and Scott just figured out what caused this so it would be good to report it here.

jchiang87 commented 7 years ago

Yes, from recent conversations with Simon, Scott, and Seth, I've understood that the apparent rotation arises from my having used an "instance" catalog to generate the astrometry index files which contains geocentric coordinates for the objects. This induces a rotation in the solution that results from accounting for Earth precession and nutation. So ultimately it was pilot error on my part in generating the index files. @danielsf can correct anything I misstated here.

cwwalter commented 7 years ago

These problems were all addressed and we are using (trimmed) full focal planes. However after the DC1 production it was discovered that there is a different rotation convention between CatSim and PhoSim which caused errors. This is documented in #36.