search

desihub / desisurvey

Code for desi survey planning and implementation
BSD 3-Clause "New" or "Revised" License
2 stars 7 forks source link

UTC vs. MJD timestamps in survey output #76

Closed sbailey closed 3 years ago

sbailey commented 6 years ago

From @weaverba137 in desihub/two_percent_desi#6:

I'd really like to get back to using full UTC timestamps instead of MJD. Databases like timestamps. Again, not an issue for this notebook.

Years ago we agreed to use UTC as the primary time metric (see https://desi.lbl.gov/trac/wiki/Pipeline/FormatsAndNumbering), but MJD has crept into desisurvey/surveysim as the defacto primary time metric. My understanding is that ICS will provide both UTC timestamps (ISO 8601 format) and MJD float in the raw data headers, with the UTC timestamp being the canonical reference time and the MJD float provided for convenience.

The primary issue with MJD is the ambiguity with definition on days with leap seconds. Astropy handles this self-constently, but it isn't clear that there is any cross package standard for doing this, or how ICS will handle this. UTC is explicit in how leap seconds are handled.

I'm not convinced that a sub-second ambiguity on the start time of a 1000 second exposure actually matters for anything, and the UTC timestamp will still be available for the rare cases where it really does. Continuing to use MJD may be pragmatic.

If we want to store a UTC timestamp in a FITS binary table, it is unclear to me what the right column format is. A 19-character text string in ISO-8601 format is probably not particularly practical. What's the right solution here? TAI float is unambiguous for time differences, but I also haven't found a canonical standard for what TAI=0 means.

dkirkby commented 6 years ago

I have no allegiance to MJD but let's not change anything in the next few weeks or adopt a standard that cannot be unambiguously represented with a float.

dkirkby commented 5 years ago

The scheduler and ETC only care about relative time, so the introduction of a UTC leap second during a night should have no impact.

The pre-tabulated ephemerides assume no new leap seconds during the survey, but are only used for forecasts which do not care about an extra second. I suggest that we recompute the ephemerides file during the first full moon after a leap second is introduced, to minimize confusion.

I propose to continue using MJD internally in desisurvey since this can be unambiguously converted to a UTC timestamp (using astropy.time), when needed, for external use.

schlafly commented 3 years ago

Given the lack of activitity in the past three years and the fact that things have largely been baked in by survey start, I'm closing this; we don't want to change formats now.