Leap second statement is confusing

[perennialmind]

At the time of writing, the documentation reads

represents the number of seconds that have elapsed

and

does not take leap seconds into account

I found this wording confusing. At first reading, I questioned whether the calculation treated leap seconds differently from absolute monotonic seconds, as it should. Based on cursory tests, I can see that calculations are correctly performed in the calendar domain (UTC), not the time domain (TAI). The best wording I have seen to date is this, by Markus Kuhn:

The POSIX second count ignores inserted leap seconds (and does not even provide an encoding for them) and counts deleted leap seconds

In practice, unix time represents a UTC date and time as a set of fields packed into an integer value according to a formula set by POSIX. The companion definition of unix time as a measure of elapsed time cannot be reconciled with this formula; it is effectively a false correspondence rooted in history and is a common source of error. It appears that ToUnixTimeSeconds uses the latter definition and conforms because the DateTimeOffset.Ticks shares the same limitation regarding leap seconds. The documentation there is more clear:

It does not include ticks that would be added by leap seconds.

(edited to fix blockquote)

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: 265528c3-9318-aa3c-6c32-a6a2b2b2cedd
• Version Independent ID: 92ea6e5d-653f-ac43-a3bf-9348fc3a0a0f
• Content: DateTimeOffset.ToUnixTimeSeconds Method (System)
• Content Source: xml/System/DateTimeOffset.xml
• Product: dotnet-api
• GitHub Login: @dotnet-bot
• Microsoft Alias: dotnetcontent

[Thraka]

@perennialmind Hi!

Are you saying that the "does not take leap seconds into account" should be a bit more clear about where those seconds come from, as the ToUnixTimeSeconds method explains it?

[perennialmind]

The wording of the documentation for DateTimeOffset.Ticks is short, clear, and precise. Ticks and UnixTime are aligned conceptually and by implementation, so using the same wording makes sense to me. The other wording I quoted is also clear and precise, with additional information for context. Either would be an improvement.

Knowing that leap seconds are a potential source of inaccuracy is important. If the contract is narrowed, the platform interface may be reliably combined with external data (a leap second table) to make faithful calculations in both true elapsed time and calendar time.

[perennialmind]

@Thraka, I think I may have missed your point. I would not expect the documentation to define leap seconds, just how they are handled. I had to review the code to find out. (The specs take leap seconds into account insofar as they are explicitly excluded from the count). The implementation is free to ignore them because Windows date/time functions have historically ignored them; that's changing, and I hope to know how that will impact DateTime. That's where the docs come in :-)

It might even be worth calling out the behavior during a leap second. I gather that the latest Windows release will report second '59' for two seconds in GetSystemTime for processes that don't opt-in to leap second support. Does GetSystemTimeAsFileTime follow suit? Hold constant for a full second, as a literal reading of the docs would imply? I don't know, but it might be worth a few words, even if only to say the behavior is "system dependent".

[tarekgh]

@perennialmind here is some clarification how the .NET (version 4.7.2) work on the version of Windows that support the leap seconds (i.e. Windows 10 RS5 release):

• DateTime (DT) and DateTimeOffset (DTO) are not changed in how it stores the time units and how operate on such units. These types just store ticks and the tick is 100 nanoseconds. When converting between ticks and date/time parts (e.g. year month, day, hour, minute, second, millisecond) it always assumes the minute is 60 seconds and cannot be 61 seconds. i.e. no leap seconds counted in the ticks or in the conversion.
• When calling Now property on DT and DTO, we'll end up calling Windows API (e.g. GetSystemTimeAsFileTime). GetSystemTimeAsFileTime has the leap seconds counted there. So, .NET is doing extra step when running on the leap seconds enabled system to get the exact time by calling more Windows API which can report the system time to ensure .NET reported time is synchronized with the system. .NET still calling GetSystemTimeAsFileTime for the sake of getting more precise time (which is 100 nanoseconds accuracy).
• In case Windows report to us a second number 60 (which is a leap second), .NET will assume this is the last second in that minute and use it as a second 59 to make it work seamlessly with DT and DTO as these types not aware of leap seconds.
• If someone try to create a DT or DTO with a leap second (60), .NET will check first by calling Windows API if this is a valid leap second then convert it to second number 59. If it is not valid leap second, then we'll throw exception.
• .NET didn’t change how DT and DTO work for the sake of application compatibility as we know many users doing the same assumptions in their code that ticks always has the minute is 60 seconds. And ticks in different system cannot mean different time.

Let me know if you have any more questions or you need more clarification

[Thraka]

@tarekgh Thank you for the great explanation. Would you be interested in updating the docs to make it clearer?

[tarekgh]

I would wait a little if there is any change will happen in the near future.

CC @rpetrusha our technical writer who I think can help with the docs.

[rpetrusha]

I hope you don't mind, @Thraka if I assign this issue to myself. I just met with @tarekgh to discuss the documentation needed for leap seconds.

[perennialmind]

@tarekgh, thank you! I'm sorry for taking so long to respond.

So Ticks will stay on the original windows "timeline", using the terminology from Dan Cuomo's "Leap Seconds for the IT Pro: What you need to know" blog entry – whether or not the ProcessLeapSecondInfo flag is in effect. That strikes me as generally desirable.

I went looking through the 4.7.2 and coreclr code available to better understand the logic you described and came up empty. If I understand your description correctly, DateTime.UtcNow will:

• acquire both a FILETIME (GetSystemTimeAsFileTime) and a corresponding SYSTEMTIME (FileTimeToSystemTime)
• construct Ticks from the SYSTEMTIME
• extract and apply the sub-ms component from the original FILETIME to Ticks
• while treating SYSTEMTIME wSecond=60 as a special case

Do I have that right?

That only leaves me with one question. When handling wSecond=60 and adjusting down to Seconds=59, is the sub-ms component still added as-is or rounded up to 999900 ns? In other words, during a leap second, do Ticks accumulate as:

▁▂▃▄▅▆▇█▁▂▃▄▅▆▇█

or

▁▂▃▄▅▆▇█████████

?

[tarekgh]

So Ticks will stay on the original windows "timeline", using the terminology from Dan Cuomo's

What you mean by Ticks here? are you referring to DateTime.Ticks?

Do I have that right?

Yes, you got it right. just to be explicit, we'll also add the fractions of milliseconds (in 100-nanoseconds) to the date time ticks for the sake of the time precision.

When handling wSecond=60 and adjusting down to Seconds=59, is the sub-ms component still added as-is or rounded up to 999900 ns?

it is the later, we round it up to the max 100-nano seconds to ensure always the reported time is progressive and be in order.

[perennialmind]

What you mean by Ticks here? are you referring to DateTime.Ticks?

Just so. Although, in my mind, I'm already aligning "Ticks" with "classic non-leap FILETIME".

Do I have that right?

Yes, you got it right. just to be explicit, we'll also add the fractions of milliseconds (in 100-nanoseconds) to the date time ticks for the sake of the time precision.

Yup. That is clearer than my wording.

When handling wSecond=60 and adjusting down to Seconds=59, is the sub-ms component still added as-is or rounded up to 999900 ns?

it is the later, we round it up to the max 100-nano seconds to ensure always the reported time is progressive and be in order.

Clear, consistent, and comprehensible behavior. I like it. I now know just what to expect.

Now if only it were possible to smuggle out the difference between Ticks and leap-FILETIME. 😏

[tarekgh]

Now if only it were possible to smuggle out the difference between Ticks and leap-FILETIME.

Just to mention, DateTime.ToFileTime API will return the leap-FILETIME ticks. so at least you'll have easy way to go back and force between DateTime ticks and system ticks :-)

[perennialmind]

Oh that is so perfect. Perfect! Heh. Maybe I should have asked for two ponies. 😁