Dome flat checks

[fjaviersanchez]

Hi all,

I computed the dome flats for several rafts in the u-band and the g-band. The ones that are labeled WFD are the ones that are in CALIB_WFD directory, whereas the others are in the CALIB directory.

Do these look reasonable @rmjarvis @jchiang87?

[rmjarvis]

I'm not sure. I would have expected it to be higher, both a small scales and then dropping slower than this with increasing k.

I tried to compute the power spectrum of a pure Poisson noise image with mean = 1.e4 as a comparison to see what we should be expecting, and I got something that doesn't seem to drop as quickly as what you found. I just scatter plotted the absolute value of the 2d FFT vs k, so my plot doesn't look as nice as yours, but it seems to be quite a bit shallower than what you found. Although admittedly, it's hard to do the averaging by eye, since the density of points gets a bit lost in the scatter plot.

Is the mean of the image around 10,000? I think that was supposed to be the target mean value for these, but that's the k=0 value of the power spectrum which isn't shown.

Also, I guess we should check the variance as well, which should be roughly mean / gain. I don't know what the gain was though.

[wmwv]

I don't have an immediate intuition for what the overall scale should be, but without having thought in much mathematical depth about this, I would think that the dome flat and regular image should roughly match.

@fjaviersanchez Could you make both real-space and power-spectrum plots of a one-D trace in x, y, and radially from focal-plane center for both the flats and the regular image? I think my simple brain can reason better about things in 1D.

[rmjarvis]

I think the regular image should have much more high frequency power. All those nearly delta functions all over the place...

[fjaviersanchez]

@wmwv @rmjarvis, sorry, I should have provided more details.

The images are first rebinned (1 pixel in the image that I compute the P(k) is a 16x16 pixels patch in the original), I ignore the gaps between sensors and the image in which I compute the power-spectra is just a juxtaposition of the 9 sensors that are part of the raft. After this, I divide the pixel values by the mean value in the image and subtract 1. Then, I compute the 2D power spectrum, and finally, I compute the radially averaged power-spectra using the 2D.

The image that I called "regular" image is the number 9 listed in issue #140. It used PhoSim's full background calculations with sources in it (although I checked that 35 that has no sources also has the same amplitude in the P(k)). I think that the rebinning takes away the small scale power since I am averaging while I rebin. I believe that in that case we have more power because of the sky-background large-scale correlations.

Should I check the same without rebinning and forget about the comparison with the "regular" image? Or compare without rebinning with the image with no sources (no. 35 that I mentioned above)? What would be more useful for you? I can also compute the 1D power in the x or y directions as @wmwv suggested or not subtract the mean value from the images. @rmjarvis I'll check the variance of those images. Thanks!

One thing that I was worried of is that, when I saw the "regular" images that were near the edges of the focal plane, they had a lot of high-scale power due to vignetting. However, the flats here, don't show as much power (maybe I should go more towards the edge than R02 too).

[jchiang87]

The "flats" in the /global/cscratch1/sd/jchiang8/desc/calibration_products/dome_flats/CALIB[_WFD]/flat are generated by the constructFlat.py task. This task does some amount of ISR on each raw dome flat image (bias and dark current subtraction, CR repair, crosstalk correction) and presumably medians the stack of images after some kind of per image rescaling. I would assume that those flats do not have a background model subtracted, so it's not clear to me that a direct comparison to a sky image (either raw or calexp) makes sense.

It might be useful to do statistics on the eimage dome flats in /global/projecta/projectdirs/lsst/groups/CI/DomeFlats_1.2p/flats/*/5*/output or even do direct comparisons with sky eimages. One could also look at the raw files in /global/cscratch1/sd/jchiang8/desc/calibration_products/dome_flats/v5*/* and compare to raw sky images, but those are multi-extension FITS files with one amp per HDU, so somewhat more work to analyze.

[wmwv]

I meant to compare against a calexp with the background model added back in.

[wmwv]

But I realize that this is already far more involved than I can follow in a combination of GitHub issues. I will step back unless you specifically want further input from me. If you do, I fear that it will likely only be efficient if there's a several page writeup that I can read so that I'm up to speed on all of the detailed work that has gone into investigating these issues. Instrument signature removal is intricate and complicated, and me wandering in and asking questions on just one particular issue perhaps isn't the most effective use of other people's time.