RIX R7M : METIS-2010 : Add additional input regarding LSF and PSF QC parameter for IFU

[astronomyk]

https://jira.eso.org/browse/MET-2010 page 10, item METIS-6923

Email conversation with Wolfgang Brandner

All: Ralf Siebenmorgen voiced his “concerns” about LMS PSF and Line Spread Function (LSF) determinations.

PSF: as a response to the AIs we got in the Nov ’22 FDR, I associated the names of the pipeline recipes with the calibration procedures. For LMS PSF determination, I selected metis_lms_std_process w/o checking/verifying that the “imaging” box is part of this recipe (see attached page 30).

Iirc, the decision here was to determine the PSF at the central wavelength of a observation wavelength range (so basically the slice at the middle of the reconstructed data cube)

Would it be fine with both of you to add this functionality to metis_lms_std_process?

This sounds like the logical place to put this functionality. @hugobuddel and @sesquideus do you agree? Just a side note - I fear you may have an older version of the DRLD with the recipes listed. Hugo and Martin put in a lot of effort re-organising the LMS recipes over summer. The new LMS workflow can be found in the latest version of the DRLD on polarion (I've added it to this email for convenience). The new name of "metis_lms_std_process" is "metis_ifu_reduce".

QC parameters could be the FWHM (minor axis) of the PSF at the central wavelength (assuming that this does not fall into a strong telluric absorption band), the ellipticity of the PSF, and the angle of the major axis in the x-y coordinate system of the 3D data cube.

These QC parameters sound reasonable, although we currently don't have a function defined in the DRLD for doing these calculations. Hugo, Martin - is this something we will need to add for FDR? I guess, only if one of the reviewers pick up on it. Otherwise, we'll make a note of it. Another side note - @Hugo, @Martin shall I add a github issue to write / include this to the DRLD repo?

LSF: this is only mentioned as an afterthought in the CAL plan (see attached page 44). The easiest way to get this funtionality included might be in metis_lms_wavecal?

Again, seems reasonable to me. @Hugo, @Martin?

We could provide, e.g., one LSF determined from a “suitable” calibration line. Pipeline output would be a LSF (normalized between 0 and 1) and a FWHM as QC? Do we have to worry about LSF varying with wavelength?

[astronomyk]

Here's how I read the needs from Wolfgang Brandners LMS calibration plan update. We should add the following functions to the IFU recipes:

metis_ifu_reduce ->

• ifu_psf_characterisation : generates QC parameters
  • QC : IFU_PSF_FWHM_1, float : major axis of the PSF at the central wavelength (assuming that this does not fall into a strong telluric absorption band),
  • QC : IFU_PSF_FWHM_2, float : minor axis of the PSF at the central wavelength
  • QC : IFU_PSF_ELLIPTICITY, float : the ellipticity of the PSF, and
  • QC : IFU_PSF_ANGLE, float : the angle of the major axis

metis_ifu_wavecal

• ifu_generate_lsf : generates LSF calibration data
  • LSF (normalized between 0 and 1) : a series of weights vs wavelength in an ASCII(?) file
  • QC : IFU_FWHM, float : the fwm of the LSF at the cube's central wavelength

Edit 2023-10-10 @Rumpelstil I only realised today that you are part of this repo. We can therefore keep the conversation going here. Are the above additions to the DRL-D in line with what you expect regarding the MET-2010 ROX from Ralf Siebenmorgen?

[hugobuddel]

My email reply:

Hi Kieran, Wolfgang, Roy, Martin,

Thanks for relaying these concerns about the LMS/IFU point spread
function and line spread function Wolfgang.

I think we can fulfill the requests by Ralf, they sound reasonable and
useful. Perhaps we should already incorporate it in the DRLD before
FDR, that would make an even stronger response.

On Thu, 28 Sept 2023 at 13:05, Kieran Leschinski
<[kieran.leschinski@univie.ac.at](mailto:kieran.leschinski@univie.ac.at)> wrote:
>
>
> Just a side note - I fear you may have an older version of the DRLD with the recipes listed. Hugo and Martin put in a lot of effort re-organising the LMS recipes over summer. The new LMS workflow can be found in the latest version of the DRLD on polarion (I've added it to this email for convenience).

The latest DRLD should always be on Overleaf [1] or Github [2];
Wolfgang has access to both, Roy to overleaf.

[1] DRLD on overleaf: https://www.overleaf.com/project/5f1abb4137d7690001f8aeb1
[2] DRLD on github: https://github.com/AstarVienna/METIS_DRLD

> Am Fr., 22. Sept. 2023 um 08:02 Uhr schrieb Wolfgang Brandner <[brandner@mpia.de](mailto:brandner@mpia.de)>:
>>
>> All: Ralf Siebenmorgen voiced his “concerns” about LMS PSF and Line Spread Function (LSF) determinations.
>>
>> PSF: as a response to the AIs we got in the Nov ’22 FDR,

Another side note: I realize now that I should have taken a closer
look at the Action Items from the previous FDR to see whether any of
them are relevant for the pipeline. I did not do so, and did therefore
not take the action items into account in the pipeline. I will look
through the action items anyway in order to be prepared for FDR.
However, I cannot (quickly) find these particular action items about
the LMS PSF and LSF determination.

> I associated the names of the pipeline recipes with the calibration procedures. For LMS PSF determination, I selected metis_lms_std_process w/o checking/verifying that the “imaging” box is part of this recipe (see attached page 30).
>
> Iirc, the decision here was to determine  the PSF at the central wavelength of a observation wavelength range (so basically the slice at the middle of the reconstructed data cube)
>
>> Would it be fine with both of you to add this functionality to metis_lms_std_process?
>
> The new name of "metis_lms_std_process" is "metis_ifu_reduce".
> This sounds like the logical place to put this functionality. @Hugo Buddelmeijer and @Martin Baláž do you agree?

The metis_ifu_reduce recipe would indeed be the best place to measure
the PSF, as the cube is reconstructed in that recipe.

metis_ifu_reduce is used to process both standard star observations
and science observations. It should be relatively easy to derive the
PSF for the standard observations, but for science observations this
might fail if there are no isolated stars. I suppose this is fine.

>> QC parameters could be the FWHM (minor axis) of the PSF at the central wavelength (assuming that this does not fall into a strong telluric absorption band), the ellipticity of the PSF, and the angle of the major axis in the x-y coordinate system of the 3D data cube.
>
>
> These QC parameters sound reasonable, although we currently don't have a function defined in the DRLD for doing these calculations. Hugo, Martin - is this something we will need to add for FDR? I guess, only if one of the reviewers pick up on it. Otherwise, we'll make a note of it.

It would be fine to add these to the DRLD. Note that we already define
"QC IFU STD FWHM" in Chapter 11 (QC Parameters) and indeed claim that
it is derived in "metis_ifu_reduce", but we don't actually list it in
the recipe table in Chapter 8...

> Another side note - @Hugo, @Martin shall I add a github issue to write / include this to the DRLD repo?

I noticed you created
https://github.com/AstarVienna/METIS_DRLD/issues/210 , thanks. Let's
discuss the details there. I'll copy my reply there as well.

>> LSF: this is only mentioned as an afterthought in the CAL plan (see attached page 44). The easiest way to get this functionality included might be in metis_lms_wavecal?
>
>
> Again, seems reasonable to me. @Hugo, @Martin?

I suppose we could do something similar in the LMS/IFU recipes as we
do in the LSS recipes. There we claim that "[f]or the determination of
the LSF we primarily rely on the possibilities as offered by the
telluricorr/molecfit package", which would mean we could do this in
the telluric correction recipe metis_ifu_telluric. metis_ifu_wavecal
only determines the central location of the lines, not the shape.

>> We could provide, e.g., one LSF determined from a “suitable” calibration line. Pipeline output would be a LSF (normalized between 0 and 1) and a FWHM as QC? Do we have to worry about LSF varying with wavelength?
>
>
> @Wolfgang, a question for you - how will the LSF be determined?
>  I had a quick look through the LSS part of the DRLD, and it seems that Innsbruck is assuming the LSF will be provided as a calibration product that was determined externally during AIT. This seems inconsistent with what we should do here.

I think we determine the LSF twice: once during AIT, and then we use
that LSF as a first guess in order to determine the actual LSF in the
pipeline.

> @Hugo, @Martin, I guess we add this as a To-do for including a function to determine LSFs in the DRLD?

Yes.

>> Please let me know if this sounds ok to you. I could then suggest this to Ralf as our solution in order to get his comments. He might still file this as a RIX, and you could then present the above as our response at PIP FDR.
>
> I'll see what Hugo and Martin have to say then we can decide what to propose to Ralf.

We need to work out the details, but it seems we are on the right track.

[Rumpelstil]

metis_ifu_reduce and metis_ifu_wavecal additions as suggested above, look fine to me. One additional concern voice by Ralf was a spatially varying LSF over the detector, and how to characterize and calibrate it. I would consider this the "typical" (central?) "series of weights vs wavelength in an ASCII(?) file" as a very good first step.

During AIT we will then try to check for any spatial variations of the LSF (I think that we should not invest PIP time+resources now into a potential effect, whose magnitude and importance we cannot easily quantify at the moment)

[astronomyk]

To keep everything in one place, here's my response:

Sorry for the delay in responding to this RIX. Indeed we have discussed this item at length with the Wolfgangs (Brandner and Kausch) and Roy. The calibration plan will now include an extra section on the determination of the LSF. In summary, the first guess LSF will be determined during AIV, and a refined LSF will be determined (and returned to the user) as part of the LSS and IFU telluric correction recipes. For this the functionality available in the molecfit package will be used. (A more detailed description can be found in teh answer to MET-2164, incl. mention of how to account for a wavelength-dependency in the LSF shape).

In any case the descriptions of the additional algorithmic steps, data products, and QC parameters will be added to the DRLD in coordination with the update to the Calibration plan.