Proposal for a FITS convention for region based "skymaps"

[adonath]

In Gammapy we already use the GADF sky maps formats for HEALPix and WCS. In Gammapy there is now the use-case to handle "spectral" data, that was integrated / averaged over a certain region with additional extra axes, e.g. time.

This pull request proposes a new format for these "region based" maps. Here is the core of the proposal:

• Store the definition of the axes in the same "Bands HDU" as defined here: https://gamma-astro-data-formats.readthedocs.io/en/latest/skymaps/index.html#bands-hdu
• Allow for an additional "DATA" column in this HDU to store the data
• Allow for an optional region HDU, storing the region definition in the standard FITS convention described here: https://fits.gsfc.nasa.gov/registry/region.html

I hope the general need for such as format is uncontroversial, as it will allow to store e.g. time dependent counts spectra together with complex region definitions. Already now there are many use-cases where this format would be useful.

Currently the description is rather short, but as noted above the proposed format relies almost exclusively on already defined HDU definitions.

[adonath]

I have requested reviews from the people, that have been active in this repo for the last time. Possibly I forgot someone...

[adonath]

For now I have added it to the skymap section, however it could possibly be a sub-section of spectra as well. The "skymap" section seemed more natural to me, as the format definition relies on the "bands HDU" and has the support for region definitions as well. So it's a format proposal for an N-dimensional map with a single spatial bin, defined by the region.

The main argument for me is that the format does not strictly apply to a "spectrum" in a classical sense. E.g. it can be used as well to store a reduced instrument response function, e.g. a PSF with a true energy and rad axis or an RMF matrix with a true and reconstructed energy axis.

[adonath]

Following up on a private discussion with @registerrier (see also the PR with the prototype implementation in Gammapy https://github.com/gammapy/gammapy/pull/3231), I updated the proposal as following:

• The data is now stored in a separate data HDU and not coupled to the bands HDU anymore
• The data HDU has required header keywords to point to the bands HDU and the (optional) region HDU and thus is consistent with WCS and HPX conventions.

This allows to store multiple "ND spectra" with the same axes specification and let them share a single bands HDU.

[registerrier]

The addition of this format is indeed useful. The OGIP format only supports parts of the use cases that this supports as it is limited to purely spectral information (counts or counts rate as a function of channel).

It is not totally clear to me where this belongs, though. This convention and the one describing skymaps (https://github.com/open-gamma-ray-astro/gamma-astro-data-formats/blob/master/source/skymaps/index.rst) are not necessarily fully connected to gamma-ray format. They simply describe a FITS storage convention for multidimensional quantities.

Should we describe them here or propose them as a set of conventions for the registry of FITS conventions that is maintained by the IAU FITS working group? For instance in the section "Conventions applicable to specific types of data" . Any opinion about this @cboisson and others?

[PaulKuin]

It might be useful to check out the standard for MOC's at CDS. What you proposed is, of course, a general problem, and using the already developed MOC approach seems to me the way to go.