search

tc39 / proposal-temporal

Provides standard objects and functions for working with dates and times.
https://tc39.es/proposal-temporal/docs/
Other
3.3k stars 149 forks source link

ISO 8601-2:2019 #819

Closed justingrant closed 3 years ago

justingrant commented 4 years ago

I just learned that a new iteration of the ISO 8601 standard that was released last year. https://www.iso.org/standard/70908.html

Are there any changes in the standard that are significant for Temporal?

justingrant commented 4 years ago

Spec abstract is below. I bolded the parts that looked most interesting.

This document specifies additional representations of dates of the Gregorian calendar and times based on the 24-hour clock that extend the basic rules and composite elements of those defined in ISO 8601-1. These representations are specified as character strings for use in information interchange. It is also applicable for representing times and time shifts based on Coordinated Universal Time (UTC).

These extensions include: — uncertain or approximate dates, or dates with portions unspecified; — extended time intervals; — divisions of a year; — sets and choices of calendar dates; — grouped time scale units; — repeat rules for recurring time intervals; and — date and time arithmetic.

justingrant commented 4 years ago

BTW, downloading the standard costs a few hundred dollars. (!!!!)

ptomato commented 4 years ago

I hope @sffc doesn't mind me reposting this from the chat, where he listed some highlights from his reading of the standard:

ptomato commented 4 years ago

And here are my opinions on whether the above changes are relevant...

Negative durations are supported. They use the popular "-P2M1D" syntax. However, they also support mixed signs in the duration. They lay out an algorithm for how to resolve the mixed signs.

As long as we have negative durations, then we should probably check whether we can conform to their algorithm to resolve mixed signs.

Durations allow fractional values; for example, "P1M2.5D" is valid.

It doesn't seem like we would be able to support this on years, months, weeks, or days, and still be consistent with what we have.

New syntax for dates and times: "1985Y4M12D" == "1985-04-12"

Does this seem like something that we want to support? Maybe it would be good to see if it gains traction as an interchange format first. It's something that could always be added as a follow-on proposal. (If we don't support this, then a lot of the other items become moot.)

New support for centuries ("16C" = 1600-1699), decades ("196J" = 1960-1969), and day of year ("350O" = 350th day of the year)

We don't have separate types for centuries or decades, so I don't think this is actionable. Maybe ordinal days would be relevant, although we already didn't support the existing 2019-350 for Temporal.Date (350th ordinal day of year 2019).

New negative sign for relative days and weeks: "-3W" = third to last week of the year, such that you can do "1985Y-3W" to mean "third to last week of 1985"

We don't have Temporal.YearWeek, so I don't think this is actionable. 2019Y-10O might be relevant if we support the new syntax, but then again we already didn't support constructing a Temporal.Date from the existing year/ordinal day notation.

New syntax for BCE: "12YB" = 12 BCE = -11Y in ISO-8601

Would only be relevant if we supported the new syntax. I'm indifferent about it otherwise.

Uncertainty: use "X" to stand in for a digit to signify that you are uncertain about it. '1390YXXM' expresses a two-digit calendar month in 1390. You can also express uncertainty using significant digits, tildes, percentages, and a number of other methods. A large portion of the spec is focused on this subject.

I thought the X had actually been in the previous version of the standard. Anyway, we don't have this concept in Temporal (although I suppose we might say that things like Temporal.YearMonth.from('2020-08-XX') are OK?)

Exponential notation (scientific notation) is allowed for representing years

Seems trivial to support, why not?

Month numbers 20 and up are defined to mean year subdivions; for example, "2001-21" represents "Spring of 2001". Subdivisions include seasons, quarters, quadrimesters, and semestrals.

I don't think this is relevant to Temporal since we don't have things like Temporal.YearSeason and Temporal.YearQuarter.

Grouped units: allows for more complex expressions such as "the second fortnight of February 2018"

Ditto.

Syntax to support sets and ranges of dates: "{1667, 1668, 1670..1672}" is a set of years

Ditto, we don't have this concept either.

Date intervals (start date plus duration) make a comeback New syntax for recurring date intervals

Intervals were already in the previous edition and we do not support them. (Although we have had one request that it would be useful, there are some practical concerns: #205)

justingrant commented 4 years ago

As long as we have negative durations, then we should probably check whether we can conform to their algorithm to resolve mixed signs.

@sffc - could you post an excerpt of the algorithm here? I'd like to understand how the spec expects constrain-balancing to work if the starting point date and time zone isn't known.

sffc commented 4 years ago

Section 4.4.1.9 defines the term "negative duration" as follows:

A duration in the reverse direction, called a "negative duration" in this document, can be expressed using the following representation based on the [duration] representation specified in ISO 8601-1:2019, 5.5.2. In this case, all time scale components within the duration representation shall be positive.

duration(m) = [!]["-"][positiveDuration]

where [positiveDuration] is the representation of a positive duration.

However, Section 11, entitled "Explicit duration and extensions", is an entire chapter dedicated to the subject of durations with mixed signs. It defines a new term for it, "composite durations". Here is an excerpt:

11.1 General

Duration can be expressed by a combination of time scale units in explicit form — years, months, weeks, days, hours, minutes and seconds. The representations of duration in this clause are based on CalConnect CC 18011, Clause 7, "Explicit duration".

These time scale components are specified in 4.3 and are used to specify positive duration.

11.2 Durational units

Time scale unit components are represented by a time scale component used to compose a duration expression and are defined as "durational units".

...

11.3.2 Composite representation

The composite representation of a duration is a more flexible and relaxed specification for duration than that of ISO 8601-1:2019, 5.5.2. It accepts all expressions of the duration representation given in ISO 8601-1:2019, 5.5.2 and is given as follows.

[!]["-"]["P"][durationUnits(m)]

where [durationUnits(m)] contains time scale components for expressing (positive or negative) duration (see 11.2).

That section goes on to define the syntax, which is the same as the syntax for "normal" (ISO 8601-1 style) durations, except that it explicitly allows mixed signs, such as:

EXAMPLE 5 'P-3M-3DT1H5M', duration of three months and three days in the reverse direction, with one hour and five minutes in the original direction. ... EXAMPLE 9 '-P8M-1D', duration in reverse, "eight months minus one day", is equivalent to 'P-8M1D', "eight months ago with a day ahead"

The next section, "11.3.3 Precedence representation", further defines syntax with multiple durations in a sequence. It calls them "precedence durations". Example:

EXAMPLE 4 'P-3MP2DP-1Y' describes a duration of three months in the reverse direction, two days in the original direction and one year in the reverse direction, to be evaluated in the order from left to right.

The algorithms for arithmetic are introduced in Section 14 and explained in detail in Annex D. Here is an example of duration-duration arithmetic from Section 14.2:

EXAMPLE 6 'P1Y10M3D - P2Y5MT10M' results in 'P3Y15M3DT-10M'

EXAMPLE 6 doesn't look right to me, but that's what it says.

Here is an example of datetime-duration arithmetic from Section 14.4, with a precedence durational unit, but not mixed signs:

EXAMPLE 3 The expression '2018Y9M10DT9H10M + P2MPT50S' is identical to the effect of '(2018Y9M10DT9H10M + P2M) + PT50S', where the parentheses indicate precedence order.

Annex D is the part I think we're most interested in. It specifies the algorithm for datetime-duration arithmetic.

It defines two new operations on datetime types: "Carry-over of overflow in time scale components" (section D.3.1) and "Truncation at time scale component boundaries" (section D.3.2). Those two operations are then used in the algorithms, of which there are three: ones for simple durations (ISO 8601-1 style, but with only one time scale component; section D.4.1), composite durations (ISO 8601-2 style; section D.4.2), and precedence durations (section D.4.3).

Here is the algorithm for composite durations, from section D.4.2:

— For all values of time scale components in the composite duration, directly modify the values of the corresponding time scale components in the date and time representation. — In the resulting date and time representation, starting from the lowest order overflowed time scale component, perform carry-over on all overflowed time scale components until there is no more overflow in the representation. — If any time scale components have become invalid, apply truncation to them. — The resulting date and time representation is complete

"Carry-over" is what we call "balance", and "truncation" is what we call "constrain". They have special rules for leap months, leap seconds, etc.

I encourage you to obtain your own personal copy of ISO-8601-2 if you want more details. You can read the license agreement here:

https://www.iso.org/terms-conditions-licence-agreement.html

To my great disappointment, Annex D does not give any examples of how to resolve situations in which the time scale components become negative, even though my careful reading of the document strongly suggests that such a situation is possible. However, the spec references CalConnect CC 18011 multiple times when discussing these algorithms, so maybe that spec has some answers.

justingrant commented 4 years ago

Because I wasn't ready to pay $200 of my family's money for the ISO spec, I read CalConnect CC 18011 instead. ;-) Here's some notes:

Honestly I found this spec disappointing. It doesn't seem to offer resolution for any of the big open issues that we're trying to resolve with Temporal.

sffc commented 4 years ago

Okay, so it sounds like CalConnect CC 18011 has pretty much the same scope as ISO-8601-2, with the same types of answered and unanswered questions.

We have a liaison relationship with CalConnect in the works. Once that is established, then we can send our questions directly to the authors of that spec.

ptomato commented 4 years ago

I didn't realize until now that the 2019 edition split ISO 8601 into ISO 8601-1 and ISO 8601-2, where part 1 is "Basic rules" and part 2 is "Extensions". All the things above are from part 2, if I'm not mistaken, so it seems appropriate that we should be able to pick and choose which extensions we support.

ronaldtse commented 4 years ago

Hi guys, editor of ISO 8601-2 and CC 18011/18012 here, from CalConnect.

ISO 8601-2 is effectively a combination of three specs: Library of Congress' EDTF, CalConnect's CC 18011 and 18012. All of these are open specs so you're not missing out much on the ISO copy. (caveat below...)

Some replies:

I didn't see any mention of how subtraction should be handled, e.g. by reversing the order for subtraction like Temporal does.

Clause 14.2 of ISO 8601-2 (which is supposed to be CC 18011) deals specifically with temporal arithmetic including subtraction. I apologize here -- because 18011 was not updated to the latest text we contributed to 8601-2. We will update 18011 to reflect the missing changes. I can't just paste the images here due to copyright restrictions, but the text is from CalConnect.

A preview...

Screen Shot 2020-09-05 at 4 28 37 PM

Absolutely nothing in the spec related to DST or time zones, other than how to append the offset. ;-( None of the DST-unsafe examples have any notes indicating that they're not safe for DST

Those examples were chosen to "hint" at the problem without naming it, because DST and timezones are out of scope of ISO 8601-2. In ISO 8601-1 and 8601-2 the only concept that exists is "time shift". This is intentional, because another project ISO 34200 (and the associated ISO 8601-3) is scoped to deal with that particular problem.

Last bit about some potential confusion -- ISO 8601-1 and ISO 8601-2 support the concept of "profiles" -- no one needs to support all the representations used in these documents. In ISO 8601-2, there is an Annex that provides the EDTF profile built upon ISO 8601-2 primitives (happy to share that via the CalConnect liaison).

ronaldtse commented 4 years ago

Answering @sffc

EXAMPLE 6 'P1Y10M3D - P2Y5MT10M' results in 'P3Y15M3DT-10M' EXAMPLE 6 doesn't look right to me, but that's what it says.

I can't figure out why that was the case either. Probably was supposed to be P1Y10M3D + P2Y5MT-10M. Will mark this down for a future amendment.

To my great disappointment, Annex D does not give any examples of how to resolve situations in which the time scale components become negative, even though my careful reading of the document strongly suggests that such a situation is possible.

I just re-read 8601-2. Indeed, Annex D does not describe situations where time scale components become negative, but Clause 14 does provide examples of subtraction where they can be performed on the same units.

Could you briefly describe the issues encountered with subtraction? In the document we were very concerned about the comparisons between nominal durations vs realized durations (e.g. with positive / negative leap second), and would love to know how you deal with them.

sffc commented 4 years ago

Hello, @ronaldtse!

Could you briefly describe the issues encountered with subtraction? In the document we were very concerned about the comparisons between nominal durations vs realized durations (e.g. with positive / negative leap second), and would love to know how you deal with them.

The algorithm essentially says to take your composite duration, apply it field-by-field to your time scale unit, and then resolve problems in the resulting intermediate time scale unit by performing carry-over and truncation as necessary.

So, what happens with 2018Y01M10D minus P2M15D? You get an intermediate 2018Y-01M-5D. The algorithms for roll-over and truncation say what to do when the values are too big, but, as far as I can tell, not when the values are too small, unless I'm missing something.

Also see #857 for some more cases and related questions.

sffc commented 4 years ago

I just re-read 8601-2. Indeed, Annex D does not describe situations where time scale components become negative, but Clause 14 does provide examples of subtraction where they can be performed on the same units.

Section 14 shows examples of subtraction between two composite durations, but not between a time scale and a duration. Section 14.4, "Date time modified by duration", is the one I'm most interested in, and it only gives a few examples (none of which involve subtraction) and defers you to Annex D for more details.

ronaldtse commented 4 years ago

Since ISO 8601-2 Annex D omitted the algorithm and examples for dealing with underflow units, here's a draft supplement. As editor I apologize that this wasn't provided in the standard.

This will be contributed back to CC 18011, and the ISO group may perhaps consider as an amendment. (https://github.com/CalConnect/csd-datetime-explicit/issues/12)

Please feel free to let us know what you think!

(insert) ISO 8601-2:2019, 14.4

In date-time arithmetic, some time scale units can be converted into one another through deterministic relationships, which are transitive. These freely convertible relationships include:

The following time scale unit conversions are not deterministic in abstract, the actual durations of these time scale units depend on contextual information:

These rules are considered as the "time scale unit conversion boundaries".

The result of a date time arithmetic expression does not change with time scale unit conversion as long as the conversion boundaries are not violated.

When the full context of a date time expression is provided, these boundaries can be relaxed accordingly.

EXAMPLE 1 The date time expression '2023Y59O' can be freely converted into '2023Y2M28D', because the 59th ordinal day of a Gregorian year is always February 28th.

EXAMPLE 2 The date time expression '2020Y60O' can be freely converted into '2020Y2M29D', because the 60th ordinal day of the Gregorian year 2020 is February 29th.

EXAMPLE 3 The date time expression '2016Y12M31DT61S' can be freely converted into '2017Y1M1DT-1S', because a leap second was inserted into the last clock minute on 2016-12-31.

(add) ISO 8601-2, D.3.3 Borrow to resolve underflow in time scale components

The composite duration form was meant to delay the resolution of date time duration until application because some time scale units require contextual information.

In some cases, it is possible to convert a duration expression that contains a negative time scale component into an equivalent duration expression where the previously negative time scale component becomes positive.

This process of conversion involves two operations:

NOTE: If exact duration is not necessary or is considered too complex to calculate, one could simplify to omit some boundaries to adopt "nominal duration", such as ignoring the leap second (i.e. ignore rule 3) and treat 1 minute = 60 seconds.

EXAMPLE 4 The expression 'P1Y10M3D − P2Y5MT10M' results in 'P3Y15M3DT-10M'. The negative sign at '-10M' can be converted into a positive expression by "borrowing" from a time scale unit of a higher order. Since the "hour" time scale unit is 0, the next time scale unit to borrow from is the "day" unit. One "day" unit expands to 24H, resulting in 'P3Y15M2DT24H-10M'. One "hour" unit can expand into 60M, resulting in 'P3Y15M2DT23H(60-10)M', which ends up being 'P3Y15M2DT23H50M'.

EXAMPLE 5 'P1Y-10M3D + P2Y-5M' results in 'P-1Y-15M3D'. The negative sign at '-1Y' cannot be resolved because there is no higher order unit to borrow from. The negative sign at '-15M' can be resolved by borrowing from the "year" unit. In order to pad the "month" unit to a positive number, the minimum number of years to borrow from "year" is 2, resulting in 'P-3Y(24-15)M3D', which equals 'P-3Y9M3D'.

EXAMPLE 6 'PT1H60S - PT122M' results in 'PT1H-122M60S'. The negative sign at '-122M' can be resolved, because the conversion between "hour" and "minute" does not cross any conversion boundaries. In order to resolve the negative sign at '-122M', a total of 3 hours need to be borrowed, resulting in 'PT(1-3)H(180-122)M60S', which concludes as 'PT-2H58M60S'. This expression can also be represented as 'PT-62M60S', since "hour" and "minute" can be freely converted (-2H = -120M; -120M + 58M = -62M).

EXAMPLE 7 'PT5H120S - PT1M' results in 'PT5H-1M120S'. The negative sign at '-1M' can be resolved by borrowing from the "hour" unit without crossing conversion boundaries. The borrowing of 1 hour to minutes results in 'PT4H(60-1)M120S', which concludes as 'PT4H59M120S'. The '59M120S' portion cannot be simplified as the "minute to seconds" conversion requires contextual information.

(new) ISO 8601-2, D.4.6 Resolving negative time scale units in an expression ("underflows")

The steps are as follows:

  1. Starting at the rightmost side of the composite duration expression, process every negative durational unit one by one as follows: a) From the negative time scale component (the "underflowed component"), find in the expression a higher order time scale component that is freely convertible (i.e. does not violate conversion boundaries) and has a positive value. There could be multiple higher order units that can be freely convertible, such as minute <> hour and minute <> day. b) Borrow units from the higher order value until the current time scale is positive. Ensure the time scale components being modified are properly updated. c) Once the underflowed component becomes positive, or if there are no more freely convertible components (even if the underflowed component is still negative), continue with step 1 on the resulting duration expression.
  2. Once all durational units have been processed, the resulting date and time representation is complete.

EXAMPLE 1 Calculation of '2022Y2M2D - P1Y10M3D':

EXAMPLE 2 Calculation of '2025Y59O - P20DT1H30M':

EXAMPLE 3 Calculation of 'T10H10M10S - PT5H30M20S':

sffc commented 4 years ago

Thanks for writing this up! I commented on it in https://github.com/CalConnect/csd-datetime-explicit/issues/12.

ptomato commented 3 years ago

Is there anything remaining to do on this issue? @justingrant Does the Duration arithmetic that we have, correspond with https://github.com/CalConnect/csd-datetime-explicit/issues/12 ?

ptomato commented 3 years ago

There seems to be nothing remaining to do on this issue.