AstarVienna / METIS_Simulations

Scripts for simulating METIS with ScopeSim
1 stars 0 forks source link

Add script to compare header keywords with the DRLD #37

Closed hugobuddel closed 3 months ago

hugobuddel commented 4 months ago

Compare headers keywords as defined in recipes.yaml with those used in the DRLD.

Better would be to (also) verify the generated FITS files. (Because at first I got confused by the header keywords that are fixed in updateHeaders.py.)

To run the test you first need to install the METIS_DRLD as a Python package. E.g.

METIS_DRLD$ pip install -e .

Then you can run the script from the METIS_Simulations directory:

METIS_Simulations$ python -i ESO/compare_yaml_with_drld.py  

I've fixed most of the problems. @JenniferKarr could you please check that I didn't accidentally make things worse? (It's past midnight, so a 2nd pair of eyes would be useful.)

Currently it only complains about the SKY observations:

LM_IMAGE_STD_RAW has DPR.TYPE SKY in yaml but STD in DRLD
N_IMAGE_STD_RAW has DPR.TYPE SKY in yaml but STD in DRLD
LM_LSS_STD_RAW has DPR.TYPE SKY in yaml but STD in DRLD
N_LSS_STD_RAW has DPR.TYPE SKY in yaml but STD in DRLD
LM_IMAGE_SCI_RAW has DPR.TYPE SKY in yaml but OBJECT in DRLD
N_IMAGE_SCI_RAW has DPR.TYPE SKY in yaml but OBJECT in DRLD
N_LSS_SCI_RAW has DPR.TYPE SKY in yaml but OBJECT in DRLD
LM_LSS_SCI_RAW has DPR.TYPE SKY in yaml but OBJECT in DRLD
LM_IMAGE_SCI_RAW has DPR.TYPE SKY in yaml but OBJECT in DRLD
LM_IMAGE_SCI_RAW has DPR.TYPE SKY in yaml but OBJECT in DRLD

We only explicitly defined SKY observations for the IFU, named IFU_SKY_RAW, so I've included those. But we do not define any SKY observations for the other modes. It seemed that these were deliberately added to the simulations, so I kept them in the yaml file. We should define them in the DRLD too if we want them.

Are some of these SKY images taken by the acquisition templates?

By the way, it also happened that recipes.yaml was correct, and the DRLD wrong, thanks: https://github.com/AstarVienna/METIS_DRLD/commit/a49db9866c2c8e049d4032dd12024f130fb39a06

JenniferKarr commented 4 months ago

The sky observations are all blank-sky images taken with the same DIT/NDIT/filters as the corresponding object. We need to think about how the recipes find the sky images, and what the telescope will produce with chopping and nodding observations.
With header keywords of STD or OBJECT they'll look identical to an actual standard or object observations, but won't have a source in the frame. We're not currently defining a source name in the header, and don't have RA/Dec coordinates that would distinguish between a source and sky. I used the generateSummary.py routine to dump the filenames and header keywords to a summary CSV file, then did some tidying on that to generate the summaries for the reports.

hugobuddel commented 4 months ago

We can keep the DPR.TYPE=SKY header, if we want use these sky observations, but then we should also give the data item a new name/DO.CATG. That is, there should (I think) be a 1-to-1 mapping between DPR keywords and DO.CATG.

But we don't mention these sky observations in the DRLD at all, so (the way things are now) we don't need to simulate them. As in, if there is a need for these observations (and I trust your judgement), then we should add them to the DRLD first (with their own name and DPR keywords), and only then add them to the simulations.

That is what we did with IFU_SKY_RAW, which has its own name, has DPR.TYPE=SKY, and is used in the metis_ifu_reduce recipe. We should do the same for the other SKY images, if indeed they are needed. I did not add SKY images for the other modes, as it didn't seem necessary, but I can very well be wrong, in which case we should add them.

But in this PR I explicitly left those SKY ones untouched (except renaming the IFU one, since that had a name). So if the rest of this PR is good, then I propose we merge this first, and then discuss the SKY observations separately.