Support time data before 1970

[Erudition]

Hey there Justin!

The beginning of 1970 is just an arbitrarily chosen moment so that UTC/Unix time can have something to be relative to. In the context of many possible uses of time-moments, it's not really that long ago.

But for some reason, it looks like your code deliberately drops data from before 1970. Why is that? Am I correct in assuming that this means all times interpreted via this database are not reliable if they are pre-1970? Can we remove this limitation?

Thanks!

[Slumber86]

I opened this repository to open the same issue, because we now have a bug in production: time before 1970 are wrong if the day is in DST. Also IANA time zone start from 1900, so it is an arbitrary limitation

[justinmimbs]

Hi there!

Yes, as the README states, this package only contains data for time zone transitions between 1970 and 2037.

One reason for this is that the IANA tzdb only includes entries for regions where civil clocks have agreed since 1970. While the tzdb does contain pre-1970 data for the reference cities, it does not split out regions where timekeeping rules diverged before 1970. In other words, there are locations for which the tzdb doesn't contain accurate pre-1970 transitions.

Another reason is that I think it's rare to use timestamps that old, so most applications wouldn't benefit from the extra data. If you have an application using historical timestamps, then I would be curious to hear about your use case.

As an aside, I don't think it's rare to use historical dates; if dates-only suffice for your application, then you could use a date library.

Thanks,

Justin

[Slumber86]

Hi justin, we have a backend that expose birthdates as ISO timestamps at midnight (I know, is wrong, but we cannot change that) Also is exposing DST correctly, so when we have a pre-1970 born people, his birthdate shift in the day before. The alternative can be to truncate the ISO at T and using only the date part.

[justinmimbs]

Hi Agostino,

Yes, you could treat the birthdays as if they are in UTC time, both when you parse and when you format for display. If your ISO strings have an offset other than UTC (i.e. they don't end with Z or +00:00), then you'll want to replace the offset with a Z.

If you wanted to use a different type for representing dates without times, then check out justinmimbs/date; you could take the string before T as you said and parse that with Date.fromIsoString. It supports formatting dates in custom languages, and it formats in English by default.

Hope this helps,

Justin

[Slumber86]

The date is not in UTC format, sadly. I still think that is better to include pre 1970 data, because inaccurate date is better than no data at all.. At the moment as workaround Included a modified data-set in my application, rather than the elm package. I'm evaluating if strip the time from data or request changes to the back-end.

[justinmimbs]

No problem if the date string isn't in UTC, as it should easy to transform it before parsing:

String.left 10 "1960-01-01T12:00-01:00" ++ "T00:00Z"

Did you modify the Elm source by hand, or did you rebuild the source with a different MIN_YEAR? I would suggest the latter and not the former.

But in this case I would recommend either treating the birthdays as UTC times at midnight or representing them with a date-only type. Trying to rely on historical time zone data for this is needlessly fragile.

Justin

[Slumber86]

I rebuilt the surce with a different MIN_YEAR, of course

[Erudition]

I agree that it is hard to immediately think of uses for timestamps (yes, time, not just date) pre-1970, but I do not agree that I should enforce that lack of imagination on my users - someone will inevitably still enter an old timestamp, but now it's far less likely to be correctly interpreted. Of course clocks had all sorts of agreement issues pre-1970, but pretending those times don't exist doesn't lead to better results. As I'm sure you know, in Elm we try to make invalid states unrepresentable ("make impossible states impossible"), so if there's something you wish the user didn't do, you make it unrepresentable with the type system. Unfortunately, you can't always do this (Elm still has no Positive/Nonnegative types), so in this case it's best practice to still support these cases (that you wish you didn't have to) just as much as the rest of the library - rather than letting it fall into undefined behavior. (Or even "untested" or "unsupported" behavior).

[justinmimbs]

Assuming we have to choose some minimum time to support, then when else should it be? To me, 1970 seems a reasonable limit for this package.

I think I'd be better informed by hearing about real use cases that require handling pre-1970 POSIX times than by imagining hypothetical ones.

Thanks for bringing up the concern. I'll close this for now.

Justin

[Slumber86]

@justinmimbs 1970 is reasonable, but for pre-1970 maybe is better to give a some kind of convention for invalid date rather than actual weird behiviour

[justinmimbs]

@Slumber86, This package only provides data for Time.Zone values; what happens when you use those values is defined in Time from elm/time.

The offset-lookup behavior for pre-1970 times isn't undefined. If your Posix time is before the earliest transition found in a Zone, then the default offset is used. The default offset for each Zone in this library is its offset at the beginning of 1970. So the behavior you'll see pre-1970 is that you'll stop seeing transitions, as you'll be stuck on the offset at the beginning of 1970 (for northern hemisphere locations observing DST, this is usually their standard time, and for southern hemisphere locations, this is usually their DST). I hope that helps!