Closed gilmoreorless closed 2 years ago
The smaller size of the data files in 2019b is expected, see the "Changes in code" section in the announcement: http://mm.icann.org/pipermail/tz-announce/2019-July/000056.html
There used to be extra data that didn't contain any new information, they were needed only to workaround bugs in various older software.
But there is indeed some information missing in moment-timezone
0.5.26.
A few winter/summer dates formatted in Australia/Sydney
for various years, version 0.5.25:
1890-01-01T20:04:52+10:04
1890-07-01T20:04:52+10:04
2000-01-01T21:00:00+11:00
2000-07-01T20:00:00+10:00
2037-01-01T21:00:00+11:00
2037-07-01T20:00:00+10:00
2038-01-01T21:00:00+11:00
2038-07-01T20:00:00+10:00
The same dates formatted with version 0.5.26:
1890-01-01T20:00:00+10:00
1890-07-01T20:00:00+10:00
2000-01-01T21:00:00+11:00
2000-07-01T20:00:00+10:00
2037-01-01T21:00:00+11:00
2037-07-01T20:00:00+10:00
2038-01-01T21:00:00+11:00
2038-07-01T21:00:00+11:00
Relevant content of the canonical tzdata file:
# Zone NAME STDOFF RULES FORMAT [UNTIL]
Zone Australia/Sydney 10:04:52 - LMT 1895 Feb
# Rule NAME FROM TO TYPE IN ON AT SAVE LETTER/S
Rule AN 2008 max - Apr Sun>=1 2:00s 0 S
Rule AN 2008 max - Oct Sun>=1 2:00s 1:00 D
The smaller size of the data files in 2019b is expected, see the "Changes in code" section in the announcement: http://mm.icann.org/pipermail/tz-announce/2019-July/000056.html
As it also says in those notes, that only applies if you manually compile the data files with zic -b slim
, which the current build process doesn't. In fact, I notice that the build process just uses zic
and zdump
binaries provided by the local machine, so the "changes in code" section of 2019b isn't entirely relevant. In most cases this won't matter, but there are times when the data and the binaries can get out of sync (e.g. zic
on my macOS 10.14.15 machine doesn't contain the -b
option yet). Having a release process dependent on environment-provided binaries opens the path for inconsistent release files, depending on who's build the release at the time.
Regardless, large scale changes to data ranges due to a compilation process should be mentioned in release notes and versioned as a breaking change. All this makes me think the changes are unintentional.
Why aren't the maintainers looking into this? I'm really worried about the sudden size drop too. If I download the the data archives directly from https://www.iana.org/time-zones then I can compare them and see that the newer release 2019b is bigger by about ~5KB in size. That makes sense as the newer release is expected to contain more information than the previous. And if I compare these two: https://github.com/moment/moment-timezone/blob/develop/data/packed/2019a.json (904 KB) https://github.com/moment/moment-timezone/blob/develop/data/packed/2019b.json (176 KB) There is a whopping 728KB difference! The unpacked versions are also very different (2.03 MB for 2019b compared to 11.9 MB for 2019a).
I'm going to pass updating this package for now.
Edit: gilmoreorless has explained the actual problem and what is going on internally. I just want to bring more attention to the issue.
This looks to be the inverse problem of #697, which noted a big increase in size with release 0.5.23.
Quoting @mj1856's response:
With this release, #308 is fixed. That bug was truncating data before certain dates, due to differences in zdump output on various systems. So prior to this release, the "full" versions of the data files weren't actually quite full.
This reinforces my belief that the change in release 0.5.26 is due to differing zdump
versions, depending on who's built the release.
@HNDeploys @ellenaua Sorry to be a pain, but the broken release was over a month ago and there's been no response (or indeed any activity in the repository) since then. (I'm pinging you directly purely because you're listed on the release commit.)
Will check it today. Sorry for the delay
I may have a related issue. In 0.5.25
in my Node REPL:
> m.tz('2043-09-12T04:00:00.000Z', 'America/New_York').format()
'2043-09-12T00:00:00-04:00'
In 0.5.26
:
> m.tz('2043-09-12T04:00:00.000Z', 'America/New_York').format()
'2043-09-11T23:00:00-05:00'
I'm observing the same behavior starting in 0.5.26:
import moment from 'moment-timezone'
function logDateTime(dateTime: string) {
console.log(moment.tz(dateTime, 'America/Chicago').format())
}
logDateTime('2037-09-08T12:34')
logDateTime('2038-09-08T12:34')
With moment-timezone 0.5.25 (tzdata 2019a)
2037-09-08T12:34:00-05:00
2038-09-08T12:34:00-05:00
With moment-timezone 0.5.26 (tzdata 2019b)
2037-09-08T12:34:00-05:00
2038-09-08T12:34:00-06:00
I originally noticed and reported this for the downstream project js-joda that uses time zone data from moment-timezone.
I would also like to point out that java.time is outputting the time zones correctly for 2038 and beyond.
Java 8u231 updated using TZUpdater with tzdata 2019c:
package com.joshden;
import java.time.LocalDateTime;
import java.time.ZoneId;
import java.time.ZonedDateTime;
import java.time.zone.ZoneRulesProvider;
public class TimezoneStuff {
private static ZoneId zoneId = ZoneId.of("America/Chicago");
private static void logDateTime(String dateTime) {
LocalDateTime localDate = LocalDateTime.parse(dateTime);
ZonedDateTime zonedDateTime = ZonedDateTime.of(localDate, TimezoneStuff.zoneId);
System.out.println(zonedDateTime.toString());
}
public static void main(String[] args) {
TimezoneStuff.logDateTime("2037-09-08T12:34");
TimezoneStuff.logDateTime("2038-09-08T12:34");
System.out.println(ZoneRulesProvider.getVersions("America/Chicago").keySet());
}
}
And this is the output:
2037-09-08T12:34-05:00[America/Chicago]
2038-09-08T12:34-05:00[America/Chicago]
[2019c]
Honestly I'm worried about the lack of attention to this... it's causing troubles for several people (see #798)
Yes, this is preventing me from upgrading to more recent versions since my application deals with dates well into the future. The off by one hour issue causes my tests to fail.
Hey, sorry guys for breaking up your flow. Apparently we didn't know either, but it was caused by Mac/Linux differences for different maintainers. Check https://github.com/moment/moment-timezone/issues/999
Ideally we should move to rule-based data, so this crap doesn't happen. Can you please share your use case (in #999) about why you need timezoned data past 2047, just curious :)
Closing in favor of #999.
BTW 0.5.37 was released on linux so the range is back up :)
My moment-timezone webpack plugin has a failing build when using version 0.5.26. After some investigation it seems the range of years in the compiled data has shrunk considerably from 0.5.25.
Compare the packed data files for 2019a and 2019b. The file size has reduced by around 80%, and a quick switch between the two files shows that the count of abbreviations and offset has significantly changed for many zones.
Picking an example at random:
To see the impact of this, I wrote a quick script to log out the extremes of a particular zone, based on the packed data files:
The result of running this script in the root of the
moment-timezone
repo is as follows:I don't think this is intended behaviour, and I suspect it might have something to do with differences in
zdump
on different machines (as the release commit had to change thezdump
usage).