cta-observatory / ctapipe

Low-level data processing pipeline software for CTAO or similar arrays of Imaging Atmospheric Cherenkov Telescopes
https://ctapipe.readthedocs.org
BSD 3-Clause "New" or "Revised" License
64 stars 268 forks source link

Update trigger event types to match R1 data model #2094

Open maxnoe opened 1 year ago

maxnoe commented 1 year ago

Describe the bug

The trigger event types definition in ctapipe.containers currently does not match the released R1 data model document.

Specifically, it distinguishes multiple different kinds of pedestal events, where the R1 data model only knows a single PEDESTAL event type.

Additional context

In simulations at least (and probably we also need this for observed data) these different kinds of pedestal events are known. Where should we store this information if there is no event type for it?

kosack commented 1 year ago

Can you give more information? What types of pedestal events are distinguished in simtel?

The R1 document currently says:

So missing types could still be added - It would be good to just know what they are and why they are needed

maxnoe commented 1 year ago

Basically the one we have currently in the enum:

    0: EventType.DARK_PEDESTAL,
    1: EventType.SKY_PEDESTAL,
    2: EventType.SINGLE_PE,
    3: EventType.FLATFIELD,

So it distinguishes between pedestals that are taken with closed or open shutter and between single_pe (internal illumination) and flatfield (external illumination).

kosack commented 1 year ago

Hmm. that enum came directly from the data model originally (but perhaps an older version). I didn't actually notice it was re-numbered or lost the "dark pedestal" type. That may actually be a problem in the R1 Doc.

kosack commented 1 year ago

Ah, I see the change:

Note that there are different types of pedestals: sky, dark, and electronic. The distinction between these pedestal types is handled at DPPS, based on monitoring data information transmitted to ACADA via the Monitoring system interface.

That was because at the R1 level, only the camera server is involved, and it has no idea what the external light sources are doing. It only knows "I got a pedestal trigger and will send a pedestal event" However, it could be that we push ACADA to assign the pedestal type at the DL0 level - that should be possible I think, but it may come down to how they split up subsystems and what info each has access to.

So indeed, we can use a different event type model than the R1 document, with these extra types, and it is ok. We could have an event_type and event_subtype field, for example

kosack commented 1 year ago

So I would say we can close this: there is no need to match the R1 model for event types, as it explicitly says we can make more detailed types at the DL1 (and maybe DL0) level with more information available at those levels.

The question may be if we keep a single event type (which exactly matches R0) and add a subtype, or if we simply re-define/re-map the event types to a more detailed model

maxnoe commented 1 year ago

@kosack ok, good. But we still need to adapt an important thing: these trigger types are defined per telescope event, not per array event.

E.g. a physics trigger event can have LST_HARDWARE for the LSTs and STEREO for MSTs and STEREO_LONG for long events in some MSTs.

kosack commented 1 year ago

Yes, those types are only per-telescope, so should probably go in the TelescopeTriggerContainer as the telescope trigger type. Stereo events in observed data can only come from SWAT, which combines the stereo LST triggers and individual telescope triggers into a single subarray trigger. So in the end, the only type that should be received there would be a "subarray stereo" type (and I think any info on the hardware trigger is only in the telescope trigger data). Have to confirm in the DL0 model, as that is where the SWAT info is really stored (at R1, we are still really only at telescope-level).

So I think we should just have different Enums for telescope trigger types and subarray trigger types.