search

European-XFEL / karabo_data

Python tools to read and analyse data from European XFEL
https://karabo-data.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
13 stars 7 forks source link

Timestamp to UTC time #104

Open zhujun98 opened 5 years ago

zhujun98 commented 5 years ago

It would be useful to get the UTC time for each train ID. Sometimes the scientist would like to know exactly when the data is taken.

As far as I understood, a timestamp is only attached to slow data and each slow data has a different timestamp. Also, some slow data may only be updated several days before this experiment and we need to figure out which timestamp is the correct one.

A possible solution in karabo_data would be scan all the slow data for a train and sort the timestamp and convert the latest one to UTC time. Now we can only do it by hand, for example, at https://www.epochconverter.com/.

takluyver commented 5 years ago

I think Python can do the conversion easily enough, e.g. https://docs.python.org/3/library/datetime.html#datetime.datetime.utcfromtimestamp

The bit we really need is a reasonable timestamp in the files. Taking the most recent control data timestamp is not a robust way to get a meaningful timestamp. We've asked ITDM to record information from the time server in the data files, but I don't know where that's got to.

zhujun98 commented 5 years ago

Yes, we can easily do that in Python. I simply copied the email content that I sent to SCS people since they are trying to get the timestamp from HDFView, so a web tool would be an easy way for copy-and-paste.

tmichela commented 5 years ago

You can do as you say @zhujun98 as a workaround, but we are depending on an update of the file format to have this feature reliably usable. I contacted IT again on the matter, wait and see...