search

Chandra-MARX / marx

Chandra X-ray Observatory ray-trace simulator
http://space.mit.edu/cxc/marx/
5 stars 4 forks source link

marx PSFs don't match observed PSFs #28

Open hamogu opened 8 years ago

hamogu commented 8 years ago

From a user: As you know, the MARX machinery has one "dial": aspectblur and one "switch": pixadj parameter in marx2fits [EXACT, EDSER, NONE, RANDOMIZE].

The MARX documentation clearly states that aspectblur=0.07 arcsec and is an observatory quantity, not an instrument-specific quantity. Nonetheless, you can "dial" aspectblur up or down, which introduces more or less gaussian random noise to the simulation, broadening or tightening the PSF.

The pixadj switches add uniform random noise, broadening the PSF in a slightly different way.

So Pat built a tool that allows us to manipulate the dial and the switch and compare to actual point source data. We have done this for several sources at both the I3 and S3 aimpoints (FI CCD and BI CCD, respectively). We find that the MARX settings that best reproduce the sources near the two aimpoints are:

I3 (FI) aimpoint

aspectblur=0.07, pixadj=NONE


aspectblur=0.15, pixadj=EDSER

NOTE -- we are NOT able to reproduce good PSFs using Jonathan's team's findings, aspectblur=0.20, pixadj=EDSER.  This makes PSFs that are broader than our data.
## S3 (BI) aimpoint

aspectblur=0.07, pixadj=EDSER

NOTE -- we are NOT able to reproduce good PSFs using our favorite settings for I3 above, namely aspectblur=0.07, pixadj=NONE.  Again these settings produce PSFs that are broader than the data.

**_THUS WE FIND THAT DIFFERENT MARX SETTINGS ARE REQUIRED FOR S3 DATA THAN FOR I3 DATA!**_
I think the thread needs to be clearer about this, once you guys confirm our findings.

MARX is not reproducing grade distributions correctly for either I3 or S3 data, so I can't imagine how its pixadj=EDSER can ever work.  Pat speculates that perhaps there are other dials/switches under the hood in MARX (as Moritz mentioned) that John Davis might have tuned to get MARX to reproduce the PSF on S3.  I think perhaps MARX was never tuned/optimized for the I3 aimpoint, given the results we find above.  I hate to think what hidden parameters might be lurking under the hood that were tuned for S3 but are totally inappropriate for I3.  I'm amazed that we can get good PSFs out of MARX for I3 at all.

I have no idea how MARX performs for sources on a FI CCD using the ACIS-S aimpoint (sources on S2 for example).  More explicitly, I don't know if the current MARX tuning adjusts things that are aimpoint-related (ACIS-I on I3 vs. ACIS-S on S3) or things that are CCD-type related (FI vs. BI).  

Incidently, the 14-year-old paper with tables of example grade distributions for I3 and S3 is here:
http://adsabs.harvard.edu/abs/2002NIMPA.486..716T
hamogu commented 8 years ago

Hi,

this is a follow-up with more details about how MARX currently handles fltgrade and what that means for the width of the PSF. We did not define a clear process of how I communicate progress, so I've decided that I'll just occasionally send email with a summary to this list of people until I'm ready to make a formal write-up.

Here, it is important to note that the marx workflow of a typical user in fact contains two programs: marx and marx2fits. marx performs the ray-tracing and outputs a bunch of binary vectors with values for each photons like true arrival time, true position, true energy, DETX, X, PHA, fltgrade, grade, and some more. marx2fits turns this into a CIAO compatible fits file.

MARX

marx assigns grades to photons based on a very simple scheme: Each pixel is split into 3*3 sub-pixels. Photons hitting the middle one get assigned fltgrade 0 = grade 0, those hitting one of the 4 corners get randomly assigned one of four appropriate corner-split grades (e.g. 10, 11, 138, 139 for the bottom left corner), and those hitting a side one of four vertical or horizontal split grades (e.g. 2, 34, 130, 162 for the middle bottom sub-pixel). That makes almost all fltgrades equally likely, and grade 0 four time more likely than the rest. That's consistent with the pie charts I showed you (bottom of http://nbviewer.jupyter.org/github/Chandra-MARX/marx-test/blob/b85ec667e01382e0539631bd279f1d607737ce37/notebooks/AspectBlur.ipynb ), but, in the words of Cathrine Grant from the ACIS team, any scheme that's not energy dependent and does not distinguish between FI and BI "is just pure fiction". For example, for BI chips that low energies, photons will generate only small charge clouds and even events that hit fairly close to the pixel boundary do not generate split events (see e.g. https://iopscience.iop.org/article/10.1086/421866/fulltext/ - Fig. 1).

marx2fits

With pixadj=EDSER marx2fits uses the fltgrade and looks up the sub-pixel event repositioning dx, dy in the appropriate CalDB file (we ship and install a copy of this file with marx) and repositions the event accordingly. I believe that this is what a_p_e also does, so that is consistent behavior with CIAO, but not with the way marx assigns the fltgrade.

What does that mean for the marx PSF?

It's obvious that marx2fits or a_p_e with "pixadj=EDSER" will not match the observed PSF, except by chance. However, that does not mean automatically that marx2fits pixadj=NONE or EXACT or RANDOMIZE are any better. The marx PSF is determined by several parameters: The "aspectblur" parameter several people have played with, but also mirror blur parameters ("P1Blur", "H1Blur", "P2Blur", ...) which are found in the marx.par below the "Do not edit anything" line. Those blur parameters are calibrated by the marx developers (John Davis did that last of that in marx 5.0, I have not touched them yet) to make the marx PSF match something else (I don't know if John compared to observations or SAOTrace simulations or both). We also have to remember that the marx mirror model is intentionally much simpler than SAOTrace, so there is no way that marx matches SAOTrace everywhere perfectly; there will always be trade-offs (do I tune to match the core? Or the wings? On-axis or off-axis? At what energy? Since marx was developed as the HETG calibration model, we have traditionally concentrated on getting the line spread functions right.)

A similar tuning process must have been done for the SAOTrace internal parameters at some point, but I don't know details.

So, what can we do?

1) Add a real CCD model to MARX to give it predictive power for fltgrades. PennState has such a model, which might be useful. In this case, predicted fltgrade values would actually mean something. However, that's not a quick fix; maybe something for the marx 6.x series. 2) Reverse the subpix CalDB file and assign fltgrades in marx such that the a_p_e or marx2fits finds exactly (possibly with some random scatter) the current dx, dy. Possibly messy to implement, but conceptually easy. However, that makes "subpixel work, because we programmed it such that it works". No predictive power, just a feel good for users who can play with grades if they wish. 3) Recommend marx2fix pixadj=EXACT with some extra blur to compensate the fact that EDSER on the real data is not exact. Recommended blur value will depend on energy and FI/BI, thus messy for users. No need to upgrade marx. 4) Put out a warning along the lines of "These are the limits of the marx/saotrace mirror and detector calibration. We have not been very forthcoming with these numbers in the past and we observe that some users put too much trust into simulation results. See this link for the current limits and wait for future version to improve that."

For now, I'm leaning towards option (4) for putting out something soon, but I still need to run more comparisons to gather all the relevant numbers for the write-up we would link to.