Standardise EXTNAMEs for HDUs

[cosimoNigro]

While EXTNAMEs for Event lists and GTIs are uniform in different data productions - it is always EVENTS and GTI, the same is not true for IRF components. For example, in the H.E.S.S. 1st data release they are in the abbreviated form AEFF, EDISP, while in the CTA 1DC data they are instead in the extended form EFFECTIVE AREA, ENERGY DISPERSION. Would it be a problem to add to the specs a mandatory EXTNAME? The only recommendation I found is this on the effective area and no other.

Any thoughts on this? Thanks

[cosimoNigro]

It's mostly for science tools, If I want to load an IRF directly from a HDU I have to open the header and search in the specific HDUCLAS* keywords. But if the IRF component had a standardised EXTNAME then I could directly fetch it from the HDU list.

[maxnoe]

Actually, there is no difference between looking for a specific extname and specific HDUCLAS keywords. You have to loop through the file until you find it.

A FITS file is not a mapping with random access, it's a sequence of HDUs.

I dont think we should prescribe extnames and science Tools should not rely on them.

[cosimoNigro]

Ok I see your point, but beside the specific structure of the files and how we read it, wouldn't it be good to provide also mandatory values for EXTNAME to the specs? The specifications we provide go beyond science tools and if people were to explore the CTA and the H.E.S.S. data they would find different HDU names. Of course they would be able to figure out what an AEFF is, but still, why not to provide uniform HDU names? Why there is one recommended for the effective area and not for the others HDUs?

[maxnoe]

@cosimoNigro I don't think it makes sense to prescribe EXTNAMEs, since extnames must be unique but people might want to store multiple HDUs of the same type in the files.

E.g. Both point-like and FE IRFs or for different cut values, or different methods of background estimation as @AtreyeeS mentioned here https://github.com/gammapy/gammapy/pull/3514.

There is already an issue about making the recommended extnames consistent: #165

However, I don't think there is a reason for science tools to rely on extnames other than when using HDU index tables. If there is only one HDU per IRF component it is identifiable via the HDUCLAS header entries. If not, that's a case that is solvable by creating a hdu index table pointing to the one the science tools should use (using an arbitrary EXTNAME).

[TarekHC]

Hi @cosimoNigro and @maxnoe

This is something that was discussed in the past, and I believe the conclusion was we don't want to "impose" it. The issue was that looking at other instruments, each calls their IRFs with different EXTNAME (even if their meaning is identical), so if we added a fixed name to the specifications, then every single high-level IRF would not be GADF compliant.

I don't see any harm with suggesting one name, though. If in the future several effective areas are added to one FITS file, it will be easier for the user to understand their content if their name is descriptive, and will also guide the DL3 "producer". If several HDUs of the same type are added to a single file, they could have "*_N" labels.

I would perhaps add "recommended" EXTNAME values, different for each IRF component type (as I believe we have for the effective areas). But I don't have a strong opinion, as the HDUCLASn keyword family is the method we decided to use to identify HDUs.

[maxnoe]

@TarekHC exactly, so as we have the other issue about providing recommendations (#165) and we don't want to require extnames, I think this issue can be closed, right?

[TarekHC]

If @cosimoNigro and others think it is ok to add recommendations, definitely close it. If others support Cosimo's suggestion of imposing the EXTNAME, feel free to re-open and farther justify the change.

My feeling is that following up on #165 would be easy, and a "good enough" solution.

[cosimoNigro]

Ok, then I am closing it and I will follow-up on #165.