Interoperability with hifitime for leap second support

[ChristopherRabotin]

I'm quite new to Rust, and just wrote my first crate hifitime, which I will be using for precise time measurement calculations (which must include leap seconds) between TAI Epoch, UTC, and Julian days. I must admit that chrono has some pretty good timezone management and formatting possibilities, and it would take me some time to incorporate all that into hifitime.

Since I'm new to Rust, I was wondering what the best approach is here, and I am seeking advice from the chronotope team for this. Should I do one of the following (or something else entirely)?

1. Open a pull request adding hifitime in the chrono dependencies and have the hifitime to chrono DateTime function exist in the chrono code base?
2. Create a new crate whose sole purpose is to convert a date time from hifitime to chrono?

Moreover, what will happen if I try to create a Utc datetime in chrono with 60 seconds?

[quodlibetor]

@ChristopherRabotin

I would recommend adding a chrono dependency to hifitime and making it optional, and implementing your conversion functions there.

The simplest way to do that would be to do something like:

[dependencies]
chrono = { version = "0.4", optional = true }

#[cfg(chrono)]
mod chrono_compat {
    // all your conversion/compat items
}
#[cfg(chrono)]
pub use chrono_compat::*;

Then your users can use your chrono compatibility layer by using hifitime as hifitime = { version = "*", features = ["chrono"] } in their Cargo.toml. You can also give the feature a more custom name (which is a bit more future-proof as well, because you're not allowed to group features under the default feature name of an optional dependency).

If you run into any issues with chrono that make it unpleasant to use please open issues or pull requests!

[quodlibetor]

Details about chrono's leap second handling is documented in the NaiveTime docs.

Generally, chrono represents leap seconds as a nanosecond part of a DateTime or NaiveDateTime in the range 1,000,000,000 1,999,999,999. These leap seconds pass through to the other fragment specifiers (micros and millis). You cannot create leap seconds by passing in a time of 60, instead you do it with a second of 59 and a milli (or other fraction) greater than reasonable (e.g. 1,000 for millis, 1,000,000 for micros, etc).

Chrono parses strings with 60 as containing leap seconds, and will emit leap seconds as having a number of seconds == 60.

We don't track historic leap seconds, it's currently recommended to treat all timezones as TAI, rather than UTC. Honestly I haven't thought about this in great detail recently and I'm not sure how critical of a bug we should consider this.

I don't believe that completely rewriting our internal time representation to use hifitime is in the cards, but improving our leap second support is enthusiastically desired. If you want to help us flesh out support in chrono I'd personally love it and be happy to work with you to make it happen. In return I can offer you nothing except for the thrill of contributing to one of the most depended-on crates in the rust ecosystem ;-)

[ChristopherRabotin]

Thanks for the information and details. Let me start by trying to integrate chrono as an optional dependency (I didn't know those existed), and we can move on from there.

Adding leap second support to chrono would be pretty awesome indeed. I started by doing an independent crate because it seemed easier to tackle than the beast-sized code base of chrono with its huge amount of features. I don't actually know how time is handled in chrono, but what I can say is that if there's a way to set a time reference at midnight on 01 January 1900, then adding leap second support from hifitime shouldn't be that hard. And, of course, I would be thrilled to work on that effort!

[quodlibetor]

What's the overall strategy that hifitime takes to implement leap second management? A cursory look at the code suggests that it's pretty similar to chrono (basically count the number of days since the epoch) except that chrono uses some tricks to optimize the calculation a bit more. Possibly just including the Leap second months in chrono would allow us to get leap seconds, but I'd much rather pull those leap seconds from the operating system if it's possible. And if we're pulling it from the tzdata files then that probably belongs in some interop with chrono-tz.

From another glance around the world:

• Java does not include leap seconds by default (1, 2)
• python does not (also on SO) include leap seconds by default (handled by the tzinfo package)
• Go doesn't appear to handle them (there's a package for them, the standard library's time package doesn't mention them, and there's even a bug about the fact that they can't be parsed)
• All the evidence I can find is that C/C++ is the same way, except more painful

So we're in good company not exposing leap seconds by default. Which makes sense, because we're not continuously broken all the time.

Something that chrono would like to be able to do would be to grow the ability to handle multiple calendar systems (preferably by providing a basis for them so that they can be implemented in external crates), perhaps a leap-second-respecting calendar would be a good testbed. So I still think it comes down to the right way to handle this is for you to provide an optional dependency on chrono, and for you to file bugs against chrono for anything that makes it more difficult than you think it should be.

[ChristopherRabotin]

Leap seconds are announced usually on 28 December of a given year by the IETF. In recent years, there has been one leap second every other year on 31 December right before midnight of 01 January, cf. the official list of leap seconds.

The strategy of hifitime is quite dumb: a human (me) listens to IETF announcements and if there is one, the code base is updated with that new leap second day. More specifically, leap seconds are by definition either between June and July or December and January, and always at the last second of the month. So the code base has those year hardcoded. There's an argument to be made that it's not the best practice to hard code these years, but the update frequency is so slow that an automated updating by fetching and parsing the IETF file isn't far too much of an overkill.

You're right: almost no mainstream language supports leap seconds by default. In C, I usually rely on the NASA NAIF Leap Second "kernel" which needs to be loaded through the legacy-code of the SPICE Tookit. It's a real mess... And effectively, my desire to rewrite a number of SPICE features in rust (with Python scripting through C-FFI) led to me start hifitime. As someone who write simulation software, I actually see a lot of potential for Rust to enhance that field, and I think that a native Rust implementation of leap seconds would do a good poster child, especially if in the standard library or close to (i.e. either in std::time or in chrono given the ubiquity of chrono).

Using the machine's time zone information to determine if a leap second has happened is valid, as long as the machine is synchronized with NTP (or a similar service). If not, it won't reflect the leap second change since these aren't predictable.

Where can I find more information about chrono's plan to support multiple calendars? All I really need for my simulation software is leap second support if converting dates to and from UTC, and a specific calendar system called the Modified Julian Day.

[ChristopherRabotin]

BTW, I agree that hifitime probably doesn't use as many short cuts as it should or could: I encountered a number of issues when implementing the conversion between UTC and number of seconds since TAI Epoch, so I do use quite a few loops for the conversion to make sure I don't forget anything: these can probably be simplified. The test suite is now exhaustive with specific odd time computation examples, so speed enhancements should be a focus of mine promptly.