Open Cosmics Common Data Format Use Case OC-UC-001

[twhyntie]

Open Cosmics CDF Use Case OC-UC-001

(NB: All names/numbering/etc temporary for the purposes of drafting!)

Summary

This document presents an example use case for the Open Cosmics Common Data Format (CDF). Generally speaking, these use cases should help establish what the CDF needs to contain to be useful for scientific analysis across cosmic ray projects.

The Use Case

• A user from an OC project has detected something in one of their detectors at time t_event.
• The user wishes to check if any other detectors in the Open Cosmics Collaboration recorded any data in a time window of t_window = t_event - Delta_t/2.0 to t_event + Delta_t/2.0.
• The user wishes to receive a list of data points containing:
  • The location of all detectors in the network;
  • An indication of whether the detector recorded something in t_window,

in such a format that they may be plotted on some form of open mapping software.

Discussion

• This is a very, very basic use case to get things going.
• What requirements should there be on the resolution in time and space?
• Should we have a way of indicating resolution on the data output by the query?
• ???

[Tontonis]

Physics questions: Do we want >t_window to be something human readable or suitable for coincidence checks for cosmic showers?

I like this idea as it would be a nice example to say... have a higher set of lists of (possible) showers above events to visualise.

[twhyntie]

@Tontonis re. t_window - great question, what would the difference be? As t_window is a function of the query parameters Delta_t and t_event my naive guess would be human-readable; the coincidence checks would be done in the back-end, presumably...

That said, what would the coincidence checks for cosmic showers would require? :-)

[pingud98]

I think t_window should be user selectable in the query? There's an added element here, the distance between the detectors (in 3D space) and the speed of light. And the timing precision of each detector.

Let's leave the geometry out of this for now and just focus on timing. If we have an api, I think it should work like this: find_time_events(t_event, t_window, t_precision) where t_precision is the self declared timing accuracy of each experiment/data source, ultimately it's going to be a number in ns or ms. The 'worst case' timing is probably going to be someone using ntp at around 100ms (a figure of merit from here https://en.wikipedia.org/wiki/Network_Time_Protocol), with the best case in the <10s of ns.

find_time_events returns a list of events found by all accessible detectors, within the time window t_window either side of the event time, where the t_precision value of the detector is equal to or bettter than the specified criteria.

(As a side note, NTP without enhancement is nowhere near good enough for cosmic timing precision, 100ms @ 3*10^8 = 30,000km! But we can work on improving this as a separate issue)

[ozel]

There's not just NTP, the PTP / IEEE 1588 protocol allows even ps precision if there's a master clock somewhere on the local network (i.e. only few network switches between the devices). In general both protocols benefit a lot if the timing source is local and based on a percise oscillator which is occasionally synced with an internet time server or a GPS clock. That may sound fancy but is quite often done and not extremly expensive.

I think we need to define what is t_event. IMHO it should be an absolute timestamp stored with a certain precision. Do we want to propose already a data size? I tend to "yes". E.g.: The NTP timestamp counts seconds since 1.1.1900 and stores this in 32-bit unsigned seconds plus 32-bit fractions of a second (resulting in a precision of 232 ps). However, used like that, it will overflow in 2036 => use 64-bits for the seconds? Alternatively we can define t_event start counting from the unix time epoch, 1.1.1970, which many systems use internally. What precision for t_event would be usefull to analyze muon showers properly? Would the NTP timestamp precision be enough?

[pingud98]

What does PTP use for it's timestamp format? Why don't we use that - I had a quick look but couldn't find it.

My starting point for muon showers is that the muons travel at the speed of light, so basically we can use that and assumed path length differences to work out the useful time from the distance between detectors. 232ps gives us a spatial resolution of about 7cms, still not good enough for multiple events on the same timepix, but more than adequate for most other detector technologies.

Also, I think we should consider/state explicitly somewhere that we'll use UTC (via GPS) time as our 'central' reference, when it comes to time, with all events referenced to UTC (GMT or Zulu time), and queries made with respect to UTC...?

[twhyntie]

@pingud98 Re. the Timepix resolution - this really helps nail down the sort of thing we should be looking at recording at this level. For the "event" level data it looks like we're talking about, it looks like single pixel-based events (i.e. 55 micron pitch) would be too much information to record (even if the Timepix electronics readout time allowed). And if you really wanted, you could retrieve the pixel information later.

Perhaps the "atomic unit" of event data could be each "sensitive detector" volume of the detector, a la GEANT4. 7cm would probably be enough then, particularly for scintillator-based detectors (briefly looking at the data formats linked to @RaoOfPhysics, it seems most projects record their detector geometries too).

[ozel]

PTP uses by default the unix time epoch as reference (can be changed though) and neglects leap seconds (as apposed to NTP which includes those, as it is fully UTC referenced). PTP is based on the international atomic time (TAI, in reverse since it's a french acronym ;) minus an offset in seconds (TAI-10s) making it comparable to UTC (but neglecting the leap seconds). Similar to GPS, but since this was introduced later, GPS has a larger offset to UTC (TAI-19s). I guess we really want a monotonic time reference for t_event and ignore leap seconds and time zones all together. some refs: http://www.madore.org/~david/computers/unix-leap-seconds.html http://www.ptfinc.com/wp-content/uploads/2015/10/NTP-and-PTP-Time-Transfer-Part-2.pdf

@pingud98 so yes, PTP seems like a good fit. Esp. as it seems easy to convert GPS's "raw" time values. The PTP timestamps consist of 48 bits for seconds, 32 bits for nano seconds.

[pingud98]

I had a chat with one of the timing experts here at CERN last week.

Summary: From his perspective there are two kinds of time - 1) UTC and 2) UTC + a small offset that's not necessarily exactly known.

In practice 1 can only be determined by sending the data from your atomic clock to your national metrology lab at the end of each month, and one month later they will give you the corrected values.

2 is basically using GPS. Over 24 hours you can assume it has the correct number of seconds (let's forget leap seconds for now), and as a general figure of merit are accurate to 100ns or better.

I would therefore suggest that as a starting point, any generic data that someone puts into Open Cosmics has a 100ns accuracy (+/-50) unless otherwise specified. This equates to a spatial resolution of 30m, which I think is reasonable for typical use cases between different physical detectors.

In terms of how to express it, I think we should stick with UTC, expressed in PTP timestamp bit format (48 bits for seconds, 32 for nano seconds), so that we can go to higher precisions in future if required.

Here's yet another guide to timing standards I found... https://confluence.qps.nl/display/KBE/UTC+to+GPS+Time+Correction

James

On 20 May 2016 at 22:23, Oliver Keller notifications@github.com wrote:

PTP uses by default the unix time epoch as reference (can be changed though) and neglects leap seconds (as apposed to NTP which includes those, as it is fully UTC referenced). PTP is based on the international atomic time (TAI, in reverse since it's a french acronym ;) minus an offset in seconds (TAI-10s) making it comparable to UTC (but neglecting the leap seconds). Similar to GPS, but since it was introduced later, it has a larger offset to UTC (TAI-19s). I guess we really want a monotonic time reference for t_event and ignore leap seconds and time zones all together. some refs: http://www.madore.org/~david/computers/unix-leap-seconds.html

http://www.ptfinc.com/wp-content/uploads/2015/10/NTP-and-PTP-Time-Transfer-Part-2.pdf

@pingud98 https://github.com/pingud98 so yes, PTP seems like a good fit. Esp. as it seems easy to convert GPS's "raw" time values. The PTP timestamps consist of 48 bits for seconds, 32 bits for nano seconds.

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/OpenCosmics/opencosmics.github.io/issues/19#issuecomment-220709393