Observation index table

[cdeil]

@mimayer - Let's use this issue to discuss details about the obs index table.

I've changed the formatting a bit and also the names or required status of some of the columns: http://gamma-astro-data-formats.readthedocs.org/en/latest/data_storage/obs_index/index.html

Any objections / further suggestions so far?

Should we make it a requirement that ALT_PNT is computed at the observation mid-time, and not in some other way? I think that would be good, because people will use this for observation selection, and it would be confusing if the observation selection differs between HAP and PA because we fill mean ALT / AZ / ZEN differently.

@mimayer - Could you please take care of the TODO for ZTRGRATE?

TODO: define what “zenithed averaged mean” means or remove this column.

[mimayer]

I've changed the formatting a bit and also the names or required status of some of the columns:

Thanks, looks much better now.

Any objections / further suggestions so far?

On first look, I saw you removed the "OBJECT" column. In my view this column should at least be optional. There are two use cases for this:

• A user wants all observations on "Crab Nebula" (of course this depends on the outcome off the HESS software)
• When building observation definition files I currently use the "OBJECT" column to assign a name to the observation. This is not necessarily required but very nice and convenient to have.

Should we make it a requirement that ALT_PNT is computed at the observation mid-time, and not in some other way? I think that would be good, because people will use this for observation selection, and it would be confusing if the observation selection differs between HAP and PA because we fill mean ALT / AZ / ZEN differently.

I agree that the we should use the ALT/AZ/ZEN at mid-time of the observation. Nevertheless, at least from the PA exporter, I am not sure how to do this. I will have a look but it should be trivial I hope.

Could you please take care of the TODO for ZTRGRATE?

I have looked into the PA code and could not find directly where and how this is filled. I will check again and remove it if I cannot the definition.

[cdeil]

Update based on the feedback by @mimayer in the last comment: https://github.com/gammapy/gamma-astro-data-formats/commit/6cbab957ae8307f05fa01c07d8bcc0f70df33b16

• OBJECT as optional string column (I deleted it by accident, completely agree it's useful) I even added recommentations on values to put ... OK?
• Require that ALT/AZ/ZEN is at mid-time of the observation. This is also something we still have to fix / change in the HAP-HD exporters.

New question: should we include info about the array location? It will be useful to have an obs lists that can contain runs from HESS, VERITAS, CTA south / north, and a way to select out a given array, no? Having this info is also useful for consistency checks (RA / DEC matches ALT / AZ) Drawback is that it's an extra 3 columns (array lon, lat, height). Maybe we can put a "OBSERVATORY" string as just one extra column and then the location can be looked up? https://github.com/gammapy/gammapy/blob/master/gammapy/obs/observers.py

[kosack]

sorry I closed this by accident

[mimayer]

OBJECT as optional string column (I deleted it by accident, completely agree it's useful) I even added recommentations on values to put ... OK?

Sounds good. Nevertheless, I would keep this as simple as possible. In PA exporters we will write to "OBJECT" whatever is stored in the HESS database. I don't think it is worth to test the object name for sanity on 'SESAME' on export run time. For example, many off observations are targeted on e.g. globular cluster with a certain name which is stored in the database. Still, of course it is useful to put this as recommendation.

New question: should we include info about the array location? It will be useful to have an obs lists that can contain runs from HESS, VERITAS, CTA south / north, and a way to select out a given array, no? Having this info is also useful for consistency checks (RA / DEC matches ALT / AZ) Drawback is that it's an extra 3 columns (array lon, lat, height). Maybe we can put a "OBSERVATORY" string as just one extra column and then the location can be looked up?

I am not sure about this. Since we have obs-index files per cut config, I doubt we will ever mix it for several instruments (different instruments always will have different cut configs). At the end the observatory should be handled on the level of the master index file, right?

[cdeil]

I am not sure about this. Since we have obs-index files per cut config, I doubt we will ever mix it for several instruments (different instruments always will have different cut configs). At the end the observatory should be handled on the level of the master index file, right?

I just remembered that the EVENTS header has OBSERVER, GEOLON, GEOLAT and ALTITUDE, i.e. exactly the info I proposed to have in the obs index. See http://gamma-astro-data-formats.readthedocs.org/en/latest/events/index.html#id2

What info to put in the index files is always a matter of convenience versus extra effort to specify / fill, i.e. to a large degree a matter of taste. Let's think about that decision a bit more ... I'm not adding it for now.

Any other comment about the obs index here welcome ... I'll leave this issue open for at least a few days ...

[mimayer]

I just have one more comment which probably affects more the the Event specs: http://gamma-astro-data-formats.readthedocs.org/en/latest/events/index.html We should probably add the same optional header keywords to the event specs as for the obs-index file. This would certainly simplify the extraction of such information (as AIRPRESS, RELHUM etc.) and fill it into the obs-index file.

[cdeil]

@mimayer - :+1: Can you adapt the events spec?

It's very convenient to be able to build the index files just from info in the FITS header, without having to inject extra info. I plan to move this script from the HD-HAP exporter to Gammapy so that it can be re-used for HAP-Fr (or even PA if you like).

[mimayer]

@mimayer - :+1: Can you adapt the events spec?

Done.

It's very convenient to be able to build the index files just from info in the FITS header, without having to inject extra info. I plan to move this script from the HD-HAP exporter to Gammapy so that it can be re-used for HAP-Fr (or even PA if you like).

Agree this is very convenient. I am having also a script which does that. It is probably best to share this stuff somehow. Maybe this can also be independent of any high-level python package such as ctools or gammapy and could be added to the HESS software (which of course is not public). We need to discuss a strategy for this eventually - best at a phone call.

[cdeil]

It is probably best to share this stuff somehow. Maybe this can also be independent of any high-level python package such as ctools or gammapy and could be added to the HESS software (which of course is not public). We need to discuss a strategy for this eventually - best at a phone call.

Well, we have host in HESS, and Gammapy and ctools as open-source packages, and Gammapy already has the DataStore class implemented (needs to be updated once our discussions here are settled). I don't think it makes sense to start another repo. Your main argument for not wanting to use Gammapy was that it's hard to install, I think. But the only required dependency is Astropy and Numpy, and I don't want to write such scripts without using the Astropy Table and SkyCoord class. So from my point of view there's absolutely no point in starting another repo outside Gammapy for scripts for data handling, background modeling or whatever.

[mimayer]

As I said, this is a separate topic...

Are there any other comments for the observation table?

[cdeil]

There's TODOs for you to say what exactly ZTRGRATE, NSBLEVEL and RELHUM is. Probably it's not possible / worth the effort to specify it so that it could be filled exactly equivalently by different FITS file producers in HESS (and VERITAS, ...). So maybe instead of trying to pin this down exactly, add a comment, something like "Some HESS chains export this at the moment and this quantity can be useful for data selection. Comparing values from different chains or other telescopes would require a more specific specification."

[cdeil]

There's a few other things above the obs spec: I have to write the time specs.

If you have time, improving the BKG_SCALE description at http://gamma-astro-data-formats.readthedocs.org/en/latest/data_storage/obs_index/index.html#notes to a point where it's clearer how it's supposed to be used would be good. Maybe it's enough to link to this section and say that it's supposed to be multiplied to those rates to get a better estimate? http://gamma-astro-data-formats.readthedocs.org/en/latest/irfs/background/index.html

[mimayer]

Done.

[cdeil]

@lmohrmann opened #95 and we're finalising the HESS test data release which will have an observation index file, so I think we should try to conclude this discussion on observation index in this issue also.

Previously it was just @mimayer and me, looking up I'm not sure if there's still useful bits of discussion that should be implemented.

@jknodlseder - Do you plan to keep using the observation index table format in ctools? If yes, do you have any opinions on the current format? http://gamma-astro-data-formats.readthedocs.io/en/latest/data_storage/obs_index/index.html#obs-index

@cosimonigro and @TarekHC are starting to produce index files for MAGIC - maybe you could have a look as well and comment?

---

My suggestion would be that we clean up http://gamma-astro-data-formats.readthedocs.io/en/latest/data_storage/obs_index/index.html now without regard for backward compatibility, and then adapt our scripts / tools to a clean, from then on more stable format.

The question for time-related info is being discussed in #95 .

Otherwise - my main comment would be that I think there are way too many "required" columns. Making a valid observation index file should be as simple as possible, really I would suggest to have "OBS_ID" as only required key, i.e. to just say that at the top, but remove the sub-section distinction between required and optional columns. This is what we use internally in HESS - often time runlists are used that just have the OBS_ID as only column, so I think that should be a valid observation index table.

Alternatively, I could be persuaded that start and stop time could be declared required. Already with pointing position there might be use cases where it's not appropriate, e.g. slew runs of IACTs of other kinds of observations by telescopes that aren't fixed on a single pointing position. Don't get me wrong - I think we should keep the definition of the pointing-related columns, so that if the info is available, it's filled in a common way instead of everyone inventing their own column names. I'm just advocating to not declare them "required" so that it becomes easier to make valid obs index tables, and the concept becomes usable for more use cases.

[TarekHC]

Hi Christoph,

I think I already said this in the past, but I personally never liked too much these tables, neither their addition to the open specs. I see them too related to the specific implementation of each science tool. Of course, I understand why they were added, as this documentation can be useful as a reference.

I would just suggest to keep in mind that the objective is to create single FITS files containing events + IRFs. In the specific case that the user wants to use another set of IRFs, then it is up to the science tools how to handle that.

[cdeil]

@TarekHC - I disagree, there needs to be some index file in DL3 declaring what data is available.

Just look at the first CTA data that was released: 1DC

There's 1000s of runs, so any analysis has to start with run selection, and both Gammapy and ctools use index files that are roughly equivalent, but incompatible in format (and FITS as documented in this spec, the other XML that's documented in the ctools docs):

• http://cta.irap.omp.eu/ctools/users/tutorials/1dc/first.html
• https://nbviewer.jupyter.org/github/gammapy/gammapy-extra/blob/master/notebooks/cta_1dc_introduction.ipynb#Select-observations

It's not just up to the analyser / science tools, these index files are released with the DL3 data, and IMO there is value in having them and there would be even more value to agree on a format.

So the HESS test data release is coming, and it will have index files in this format and no XML index files so that ctools people can go analyse it straight away. I'm not even attached to the formats we have here, there's several things I dislike about them. But it seems to me that interest in collaborating on these open specs has stalled from the ctools side, so we just continue with what we have here. If the ctools XML formats were documented here and they are better in some way, we could consider deprecating / discouraging the existing formats at http://gamma-astro-data-formats.readthedocs.io/en/latest/data_storage/index.html and move to simpler index files at the DL3 level. But I don't think we can get rid of them completely, i.e. CTA and other observatories just release 1000s of FITS files, letting the user scrape the FITS header infos to find the data they are interested in.

I also don't think the open spec should be limited to DL3, e.g. there's http://gamma-astro-data-formats.readthedocs.io/en/latest/skymaps/healpix/index.html which is already partly in use and continues to be extended in the Fermi science tools, and is now being implemented in gammapy.maps. This is DL4, but it would be great if ctools and others adopt and extend these formats instead of just implementing the same thing in a different format.

I would just suggest to keep in mind that the objective is to create single FITS files containing events + IRFs.

I agree with this goal. It's what we do for the upcoming HESS test data release. I strongly advocated for this for CTA 1DC, but there they put a CALDB. And apart from needing to still convince people to do this, there is the problem of duplicating IRFs and space. Especially for background models, but also for EDISP and PSF tables the responses quickly dominate over events, even if they are almost the same for many runs. So in the end I would say it's still an open question if CTA will deliver DL3 in this one file per obs, including background model, or if a way of linking to separate files will be needed.

Anyone willing to really pick up the work on the DL3 specs and organise another call to kick this off?

[TarekHC]

Yes, we have discussed this several times. :)

There's 1000s of runs, so any analysis has to start with run selection, and both Gammapy and ctools use index files that are roughly equivalent, but incompatible in format (and FITS as documented in this spec, the other XML that's documented in the ctools docs):

I understand that science tools need them, but I disagree with the fact that CTA should provide them. If you have 1000s of runs, it is extremely simple, fast and efficient to run a script (at the science tool level) to select the runs where a given source appears, or within a given time period, etc...

If science tools want to agree on a specific format for this intermediate step, of course I understand that it could be technically convenient, but not much more than that... We would still need a tool to create new data selections, so the gain is not that clear to me.

I also don't think the open spec should be limited to DL3, e.g. there's http://gamma-astro-data-formats.readthedocs.io/en/latest/skymaps/healpix/index.html which is already partly in use and continues to be extended in the Fermi science tools, and is now being implemented in gammapy.maps. This is DL4, but it would be great if ctools and others adopt and extend these formats instead of just implementing the same thing in a different format.

Yes, you have a point there. Although I see stronger (technical & scientific) reasons why we should adopt standards within higher level products as compared with adopting a standard in these "data selection" tables.

I agree with this goal. It's what we do for the upcoming HESS test data release. I strongly advocated for this for CTA 1DC, but there they put a CALDB. And apart from needing to still convince people to do this, there is the problem of duplicating IRFs and space. Especially for background models, but also for EDISP and PSF tables the responses quickly dominate over events, even if they are almost the same for many runs. So in the end I would say it's still an open question if CTA will deliver DL3 in this one file per obs, including background model, or if a way of linking to separate files will be needed.

Ok good. If you agree, then I'm totally ok with all this.

I would say that one file per obs/run is pretty clear, and will adjust the file size depending on the amount of time you want to add to each file (adding start/stop to the IRFs, you can have flexible DL3 file size). Having bulky IRFs is not optional, it is required to have small systematic uncertainties.

But with the background models I agree that it is not that clear: a simple model should be provided, but more complicated ones should be optionally generated using science tools. But I see the actual implementation of how to link each background model to each analysis not that much as a standard, but as an implementation.

Anyone willing to really pick up the work on the DL3 specs and organise another call to kick this off?

Very good question. I will see if I continue working on this topic in the middle of November. If I don't, then I really don't know who will take care of it...

[cdeil]

If you have 1000s of runs, it is extremely simple, fast and efficient to run a script (at the science tool level) to select the runs where a given source appears, or within a given time period,

This is not true. Opening up all the FITS files and reading the headers is not lightning fast. I did it for the CTA data challenge and also for HESS: https://github.com/gammasky/cta-dc/blob/master/data/cta_1dc_make_data_index_files.py If the FITS files are gzipped (which I currently plan to do for the HESS test data release, see #46) it's even much more inefficient, because you have to read the complete files and decompress them just to get at the FITS headers.

If you don't provide an index file with DL3 data, then what you'll get is just lots of scripts flying around between 100s of users, with endless variants of doing glob to find files and extracting infos from FITS headers.

I agree 100% that index files are a convenience thing! It is and should be possible to analyse DL3 data without the index files! But I don't think we can or should get rid of them at the DL3 level and keep a nice user experience, I think instruments need to hand out index files of what data exists. In HESS also every analysis stars with a run selection from an observation table (in that case querying a MySQL database).

@TarekHC Maybe I just stop arguing and wait for you to do your first data release or two in MAGIC, with 100s or 1000s of runs? I'm almost certain you would also provide an observation table of what data is there, or if not most users would ask you for it. :-)

[TarekHC]

This is not true. Opening up all the FITS files and reading the headers is not lightning fast. I did it for the CTA data challenge and also for HESS: https://github.com/gammasky/cta-dc/blob/master/data/cta_1dc_make_data_index_files.py If the FITS files are gzipped (which I currently plan to do for the HESS test data release, see #46) it's even much more inefficient, because you have to read the complete files and decompress them just to get at the FITS headers.

Define lightning fast? It is something that you are going to do once per analysis, and it will not be that common to have that many files in the CTA era (not that many proposals with huge amount of hours).

If you don't provide an index file with DL3 data, then what you'll get is just lots of scripts flying around between 100s of users, with endless variants of doing glob to find files and extracting infos from FITS headers.

If science tools provide these scripts, then I don't see a reason why that should happen.

@TarekHC Maybe I just stop arguing and wait for you to do your first data release or two in MAGIC, with 100s or 1000s of runs? I'm almost certain you would also provide an observation table of what data is there, or if not most users would ask you for it. :-)

You might be right there... You have way more experience than I do. But even if that was the case, the debate to add those tables within the DL3 specs would still be open.

But anyway, sorry to open this debate. I do not propose to remove them from the specs, so sorry for adding noise. If they are there, we should improve/update them! Although I'm afraid I won't be able to add much to the discussion.

[cdeil]

If science tools provide these scripts

We had this in Gammapy, but removed it ~ 2 years ago. The problem was that the data format (folder and file and HDU names) kept changing / evolving for each new FITS production in HESS (done by 3 different people / chains), and it was annoying to have to add a new "scheme" (that's what we called it) to access the data to Gammapy for every new FITS production that came out, i.e. to touch the science tool code every time.

I would go so far as to say that archiving / releasing FITS data without index tables has been tried and was found to be very unpleasant, and this goal to have them be part of the DL3 spec (as an optional, convenience thing for data selection / access) is already based on a lesson learnt.

That said, I'm not really against re-adding parts of https://github.com/gammasky/cta-dc/blob/master/data/cta_1dc_make_data_index_files.py to Gammapy, to make it easier for people doing FITS productions for HESS, CTA, MAGIC to generate index files and checks on produced DL3 data. The problem with that is that it's hard to write this code in a re-usable way, because there's always some specific formatting that needs to be done to get to the common format. So at the moment we have the copy & paste of such index file generation scripts by the ~ 5 people in the world that do FITS productions, which is annoying but I think acceptable, as long as it's not ~ 500 CTA users that are copying around variants of that script for the FITS productions that are coming over the coming years.

But anyway, sorry to open this debate. I do not propose to remove them from the specs, so sorry for adding noise. If they are there, we should improve/update them!

I think it's OK to spend some time to re-discuss this. It's an important question and there is no great solution or concensus yet. I'm 100% agreed with this final statement: let's not remove them, but try to improve the spec a bit, since some instruments (HESS & CTA data release) and tools (Gammapy and partly ctools) support them.

[kosack]

it's even much more inefficient, because you have to read the complete files and decompress them just to get at the FITS headers.

Is that really true? I know libcfitsio can handle reading gzipped fits as a stream (without decompressing the full file), and I would assume the astropy.io.fits does the same. You never need to uncompress the file manually, and the libraries only decompress the part they need (so would stop after the header).

[kosack]

For the question of index tables, I tend to agree with Tarek - science tools could just provide a tool to generate what is needed. Remember in real CTA analysis, run-selection will be done mostly at the GTI level, though it is useful to have a first pass to identify which files should be loaded and which IRF to use. For that, one could have something much simpler than what is there. Also, is there a good reason why we are not basing it on the CALDB index format (e.g. https://heasarc.gsfc.nasa.gov/docs/heasarc/caldb/data/asca/sis/)? That would keep compatibility, and also defines how to name the IRF tables. I notice for example that this is still inconsistent between ctools and gammapy (or at least not clearly defined).

[cdeil]

Is that really true?

I think yes. Please read and comment in #46 , which is specifically about that question.

For the question of index tables, I tend to agree with Tarek - science tools could just provide a tool to generate what is needed.

Well, then please get involved with CTA 1DC and make these comments now https://forge.in2p3.fr/projects/data-challenge-1-dc-1/boards and then again when the 1DC close-out document gets written / circulated.

If you don't, then 2DC and future CTA data releases will keep releasing index files in multiple formats.

:-)

[TarekHC]

I am strongly against using a CALDB approach. In the future, we won't have like now, 1000s of runs represented by a simple IRF. Our IRFs are changing very fast, and the use of incorrect IRFs could be catastrophic. The whole DL3 approach, ideally, goes in the direction of using a single FITS file with events + IRFs, so the bond between which events are used with which IRFs is stronger (even if that link is not perfectly clear now).

[TarekHC]

In my mind, CALDB should be used as its name indicates, as a database. For Fermi-LAT the database can be included within the science tools, as the total size of all Fermi-LAT IRFs is small. For CTA, the IRF database will definitely be larger.

We could use CALDB internally to store all IRFs, but when providing the DL3 files, I would push to provide merged events+IRFs.

[cdeil]

I am strongly against using a CALDB approach.

Me too. I was arguing we should distribute DL3 data for 1DC in a more realistic way and put run-wise IRFs. This would have shown people that IRF size matters and they need to work on good IRF formats.

I think it's important that people knowleageable about data comment on 1DC, so that there's a chance of things getting better in the future. Probably commenting in https://forge.in2p3.fr/projects/data-challenge-1-dc-1/boards would be best. Or we could gather our comments on the 1DC dataset a bit and then post one thread listing our ~ 5 complaints with 1DC.

Roberta and I started this wiki page, where we had planned to gather such comments: https://github.com/gammasky/cta-dc/wiki/1DC-comments-from-gammapy-team Feel free to add things there any time if you think it's better to first gather comments and submit them all together.

[TarekHC]

I think it's important that people knowleageable about data comment on 1DC, so that there's a chance of things getting better in the future. Probably commenting in https://forge.in2p3.fr/projects/data-challenge-1-dc-1/boards would be best. Or we could gather our comments on the 1DC dataset a bit and then post one thread listing our ~ 5 complaints with 1DC.

That is actually a good idea. I have not been involved in the 1DC at all, but if I ever find the time to play with it, I will add something to the discussion.

Me too. I was arguing we should distribute DL3 data for 1DC in a more realistic way and put run-wise IRFs. This would have shown people that IRF size matters and they need to work on good IRF formats.

I can understand why these were not included for the 1DC, but in the close future we will be able to interpolate different IRFs for different azimuth/zeniths. In fact, the next data challenge could even propose the CTA MC group the challenge of generating minimal MC productions for each run following a certain path in the sky, to really encounter the problems we will find in the future.

[jknodlseder]

We have no plans to support the observation index table format in ctools. As you know, I never was a friend of the index tables and I'd prefer a scheme of on-the-fly index generation which is much more flexible. Hence there should be tools that allow for example via an auto discovery to generate an index file, but the index file itself should not be part of the data format.

Le 19 oct. 2017 à 12:46, Christoph Deil notifications@github.com a écrit :

@lmohrmann https://github.com/lmohrmann opened #95 https://github.com/open-gamma-ray-astro/gamma-astro-data-formats/issues/95 and we're finalising the HESS test data release which will have an observation index file, so I think we should try to conclude this discussion on observation index in this issue also.

Previously it was just @mimayer https://github.com/mimayer and me, looking up I'm not sure if there's still useful bits of discussion that should be implemented.

@jknodlseder https://github.com/jknodlseder - Do you plan to keep using the observation index table format in ctools? If yes, do you have any opinions on the current format? http://gamma-astro-data-formats.readthedocs.io/en/latest/data_storage/obs_index/index.html#obs-index http://gamma-astro-data-formats.readthedocs.io/en/latest/data_storage/obs_index/index.html#obs-index @cosimonigra and @TarekHC https://github.com/tarekhc are starting to produce index files for MAGIC - maybe you could have a look as well and comment?

My suggestion would be that we clean up http://gamma-astro-data-formats.readthedocs.io/en/latest/data_storage/obs_index/index.html http://gamma-astro-data-formats.readthedocs.io/en/latest/data_storage/obs_index/index.html now without regard for backward compatibility, and then adapt our scripts / tools to a clean, from then on more stable format.

The question for time-related info is being discussed in #95 https://github.com/open-gamma-ray-astro/gamma-astro-data-formats/issues/95 .

Otherwise - my main comment would be that I think there are way too many "required" columns. Making a valid observation index file should be as simple as possible, really I would suggest to have "OBS_ID" as only required key, i.e. to just say that at the top, but remove the sub-section distinction between required and optional columns. This is what we use internally in HESS - often time runlists are used that just have the OBS_ID as only column, so I think that should be a valid observation index table.

Alternatively, I could be persuaded that start and stop time could be declared required. Already with pointing position there might be use cases where it's not appropriate, e.g. slew runs of IACTs of other kinds of observations by telescopes that aren't fixed on a single pointing position. Don't get me wrong - I think we should keep the definition of the pointing-related columns, so that if the info is available, it's filled in a common way instead of everyone inventing their own column names. I'm just advocating to not declare them "required" so that it becomes easier to make valid obs index tables, and the concept becomes usable for more use cases.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/open-gamma-ray-astro/gamma-astro-data-formats/issues/7#issuecomment-337871010, or mute the thread https://github.com/notifications/unsubscribe-auth/AC2oV2Dio5hWgNrfRAwlZZUtkuVWpTJPks5styhygaJpZM4GxrWD.

[cdeil]

I've removed the "master index table" and improved the description of the obs and HDU index table in #109 . See the new version: http://gamma-astro-data-formats.readthedocs.io/en/latest/data_storage/index.html

I'm closing this old issue now.

There's several things that could / should be improved, but given that the index tables will likely be replaced by something new completely, I think it's OK to leave the spec for this prototype format (that has been and is in use in HESS and by other IACTs) as-is.