lib/time: update tzdata before release

[tklauser]

The timezone database in lib/time should be updated shortly before the 1.10 release (to whatever tzdata release is current then). There was https://golang.org/cl/74230 attempting to do this just now, but it was too early. @ALTree suggested to open an issue about this, so we don't forget.

The latest available version is shown at https://www.iana.org/time-zones.

[gopherbot]

Change https://golang.org/cl/74230 mentions this issue: lib/time: update tzdata to 2017c

[bradfitz]

2017c is still the latest. Will regarget to Go 1.11.

[ksshannon]

2018a came out a few days ago:

https://www.iana.org/time-zones

I'm assuming it's too late in the cycle, but just thought I'd post.

[ALTree]

That's weird, they didn't even announce it in the mailing list: http://mm.icann.org/pipermail/tz-announce/

Anyway I've looked at the CHANGELOG and there are a few changes in the timezones data, here's the short version:

Release 2018a

  - São Tomé and Príncipe switched from +00 to +01.
  - Brazil's DST will now start on November's first Sunday.
  - Ireland's standard time is now in the summer, not the winter.

We could try to update it and see if anything breaks (usually it's the tests). If everything is fine, maybe it could be shipped with go1.10

[ALTree]

At the moment our update script is broken because the 2018a release is not yet published at https://data.iana.org/time-zones/releases/, which is the place we visit to download the data. Considering this (and the fact that the new release wasn't even announced in the tz-announce mailing list) we should probably wait.

[ksshannon]

Agreed, I've downloaded the archives and make fails, so I think it may have issues.

[ksshannon]

There was an issue with the archive, 2018b will be released in a few days:

http://mm.icann.org/pipermail/tz/2018-January/025814.html

[ksshannon]

Another issue with 2018b, but 2018c finally made it to the announce list:

http://mm.icann.org/pipermail/tz-announce/2018-January/000048.html

[bradfitz]

Doesn't seem worth doing this late in the cycle. @ianlancetaylor?

[ianlancetaylor]

I have no opinion. I note that there is some disagreement going on about Irish time; I don't know how or whether that was resolved.

[gopherbot]

Change https://golang.org/cl/89375 mentions this issue: lib/time: follow redirects in curl

[ksshannon]

The Irish issue was reverted. It sounded a little messy. It may just be a good idea to wait, but I have no strong opinion either. They did change the location of the archives, which the referenced CL fixes when the tree opens up.

[gopherbot]

Change https://golang.org/cl/117855 mentions this issue: lib/time: update vendored tzdata to release 2018e

[gopherbot]

Change https://golang.org/cl/151299 mentions this issue: lib/time: update tzdata to 2018g

[ksshannon]

Just an update, there have been two more releases since 2018g, 2018i is available as of 2018-12-30:

https://www.iana.org/time-zones

[bradfitz]

This isn't critical for Go 1.12 but would also be fine to include in Go 1.12 too.

Except when I try to update it, I get awk crashes, even with the tzdata version we already have:

awk -v DATAFORM=`expr main.zi : '\(.*\).zi'` -f ziguard.awk \
          africa antarctica asia australasia europe northamerica southamerica etcetera systemv factory backward  >main.zi.out
*** Error in `awk': malloc(): memory corruption: 0x000055983caeb9c8 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x70bfb)[0x7f3863057bfb]
/lib/x86_64-linux-gnu/libc.so.6(+0x76fc6)[0x7f386305dfc6]
/lib/x86_64-linux-gnu/libc.so.6(+0x79089)[0x7f3863060089]
/lib/x86_64-linux-gnu/libc.so.6(__libc_malloc+0x54)[0x7f3863061f64]
awk(+0x119c7)[0x55983c0bb9c7]
awk(+0x730d)[0x55983c0b130d]
awk(+0xfbe0)[0x55983c0b9be0]
awk(+0x82be)[0x55983c0b22be]
awk(+0x2b5f)[0x55983c0acb5f]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1)[0x7f38630072e1]
awk(+0x2b9a)[0x55983c0acb9a]
======= Memory map: ========
55983c0aa000-55983c0c6000 r-xp 00000000 08:01 401125                     /usr/bin/mawk
55983c2c5000-55983c2c6000 r--p 0001b000 08:01 401125                     /usr/bin/mawk
55983c2c6000-55983c2c8000 rw-p 0001c000 08:01 401125                     /usr/bin/mawk
55983c2c8000-55983c2cc000 rw-p 00000000 00:00 0 
55983cacf000-55983caf0000 rw-p 00000000 00:00 0                          [heap]
7f385c000000-7f385c021000 rw-p 00000000 00:00 0 
7f385c021000-7f3860000000 ---p 00000000 00:00 0 
7f3862dd0000-7f3862de6000 r-xp 00000000 08:01 525224                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7f3862de6000-7f3862fe5000 ---p 00016000 08:01 525224                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7f3862fe5000-7f3862fe6000 r--p 00015000 08:01 525224                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7f3862fe6000-7f3862fe7000 rw-p 00016000 08:01 525224                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7f3862fe7000-7f386317c000 r-xp 00000000 08:01 580558                     /lib/x86_64-linux-gnu/libc-2.24.so
7f386317c000-7f386337c000 ---p 00195000 08:01 580558                     /lib/x86_64-linux-gnu/libc-2.24.so
7f386337c000-7f3863380000 r--p 00195000 08:01 580558                     /lib/x86_64-linux-gnu/libc-2.24.so
7f3863380000-7f3863382000 rw-p 00199000 08:01 580558                     /lib/x86_64-linux-gnu/libc-2.24.so
7f3863382000-7f3863386000 rw-p 00000000 00:00 0 
7f3863386000-7f3863489000 r-xp 00000000 08:01 580575                     /lib/x86_64-linux-gnu/libm-2.24.so
7f3863489000-7f3863688000 ---p 00103000 08:01 580575                     /lib/x86_64-linux-gnu/libm-2.24.so
7f3863688000-7f3863689000 r--p 00102000 08:01 580575                     /lib/x86_64-linux-gnu/libm-2.24.so
7f3863689000-7f386368a000 rw-p 00103000 08:01 580575                     /lib/x86_64-linux-gnu/libm-2.24.so
7f386368a000-7f38636ad000 r-xp 00000000 08:01 580495                     /lib/x86_64-linux-gnu/ld-2.24.so
7f386389c000-7f386389e000 rw-p 00000000 00:00 0 
7f38638a9000-7f38638ad000 rw-p 00000000 00:00 0 
7f38638ad000-7f38638ae000 r--p 00023000 08:01 580495                     /lib/x86_64-linux-gnu/ld-2.24.so
7f38638ae000-7f38638af000 rw-p 00024000 08:01 580495                     /lib/x86_64-linux-gnu/ld-2.24.so
7f38638af000-7f38638b0000 rw-p 00000000 00:00 0 
7ffe2057d000-7ffe2059e000 rw-p 00000000 00:00 0                          [stack]
7ffe205ee000-7ffe205f0000 r--p 00000000 00:00 0                          [vvar]
7ffe205f0000-7ffe205f2000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
Aborted

So I'll let others do this.

/cc @ianlancetaylor

[gopherbot]

Change https://golang.org/cl/156637 mentions this issue: lib/time: update tzdata to 2018i

[gopherbot]

Change https://golang.org/cl/184577 mentions this issue: lib/time: update tz data to 2019b

[gopherbot]

Change https://golang.org/cl/208797 mentions this issue: lib/time: update tz data to 2019c

[gopherbot]

Change https://golang.org/cl/230360 mentions this issue: lib/time, time/tzdata: update tz data to 2020a

[gopherbot]

Change https://golang.org/cl/261363 mentions this issue: lib/time, time/tzdata: update tz data to 2020b

[gopherbot]

Change https://golang.org/cl/264681 mentions this issue: lib/time, time/tzdata: update tz data to 2020d

[gopherbot]

Change https://golang.org/cl/279794 mentions this issue: lib/time, time/tzdata: update tzdata to 2020e

[ksshannon]

It appears 2020f is on the way:

https://mm.icann.org/pipermail/tz/2020-December/029646.html

[gopherbot]

Change https://golang.org/cl/280455 mentions this issue: lib/time, time/tzdata: update tzdata to 2020f

[gopherbot]

Change https://golang.org/cl/285719 mentions this issue: lib/time, time/tzdata: update tzdata to 2021a

[heschi]

2021a is still the latest, so nothing to do at the moment.

[cagedmantis]

A quick look confirmed that 2021a is still the latest version.

[dmitshur]

2021a is still the latest, so we'll be using it for 1.17. Moving to next milestone.

[ksshannon]

2021b may be out soon, and it appears that we may need to add PACKRATDATA=backzone to upate.bash. This will keep the pre-1970/Unix epoch information that was recently moved in commit 1facc7d. From what I can tell, many zones that were the same post-epoch were merged, and the pre-epoch data (which is not specifically in scope for tzdb) was removed from the standard build. A simple reproducer, building both ways and testing against each shows a change in behavior(from a fresh alpine container, and tested on darwin/amd64 too):

$ git clone https://github.com/eggert/tz
$ cd tz
$ make PACKRATDATA=backzone CFLAGS=-DSTD_INSPIRED AWK=awk TZDIR=packratinfo posix_only
$ make clean
$ make CFLAGS=-DSTD_INSPIRED AWK=awk TZDIR=zoneinfo posix_only

$ cat << EOF > zone.go
package main

import (
    "fmt"
    "log"
    "time"
)

func main() {
    t := time.Date(1963, 6, 1, 12, 0, 0, 0, time.UTC)
    z, err := time.LoadLocation("Europe/Oslo")
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(z)
    fmt.Println(t.In(z))
    z, err = time.LoadLocation("Europe/Berlin")
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(z)
    fmt.Println(t.In(z))
}
EOF

# Go 1.17.1 with $GOROOT/lib/time/zoneinfo.zip explicitly set
$ ZONEINFO=/usr/lib/go/lib/time/zoneinfo.zip go run zone.go
Europe/Oslo
1963-06-01 14:00:00 +0200 CEST
Europe/Berlin
1963-06-01 13:00:00 +0100 CET

$ ZONEINFO=/opt/tz/zoneinfo go run zone.go
Europe/Oslo
1963-06-01 13:00:00 +0100 CET
Europe/Berlin
1963-06-01 13:00:00 +0100 CET

$ ZONEINFO=/opt/tz/packratinfo go run zone.go
Europe/Oslo
1963-06-01 14:00:00 +0200 CEST
Europe/Berlin
1963-06-01 13:00:00 +0100 CET

So there is a difference, between the normal 2021b and the PACKRAT=backzone build, the latter matching Go's current behavior. I personally don't really have a preference, but setting PACKRAT=backzone may be less disruptive.

Original discussion on the change can be found here:

https://mm.icann.org/pipermail/tz/2021-May/thread.html

In the thread that changes to [tz] [PROPOSED] Merge timezones that are alike since 1970

And spirited discussion, including talk of a fork or having two releases (2021a1 and 2021b) can be found in this months archive:

https://mm.icann.org/pipermail/tz/2021-September/thread.html

[gopherbot]

Change https://golang.org/cl/352153 mentions this issue: lib/time: update tzcode to 2021b

[ksshannon]

Many of the backzone changes were reverted for 2021b.

[leonklingele]

Wouldn't it make sense to use the embed package to bundle the generated zoneinfo.zip file? The go-generate-strconv-quote magic is so… oldschool and wastes around 1.5MB of additional diff for the src/time/tzdata/zipdata.go file on each update.

[ianlancetaylor]

@leonklingele See #43350.

[gopherbot]

Change https://golang.org/cl/362874 mentions this issue: lib/time, time/tzdata: update to 2021e

[dmitshur]

@tklauser Do you think it's safe to expect that 2021e will still be latest by the time 1.18 final is released (est. Feb 2022)? If there's a chance of a newer timezone release before then, we should move this issue back to 1.18 milestone and check again in 2022.

[tklauser]

Let‘s move it back to the 1.18 milestone and see whether there will be another release.

[heschi]

2021e is still the latest. Moving to 1.19.

[gopherbot]

Change https://go.dev/cl/409374 mentions this issue: lib/time, time/tzdata: update to 2022a

[dmitshur]

Mailed CL 409374 to update to 2022a, which was released this March. Once it's submitted we can apply okay-after-beta1 here in case there are further TZ releases to consider before the final Go 1.19 release.

[dmitshur]

According to https://www.iana.org/time-zones, 2022a is still the latest today. The final 1.19 release is this week and that's likely the version we'll use, so moving this to the next milestone.

[gopherbot]

Change https://go.dev/cl/422875 mentions this issue: lib/time, time/tzdata: update to 2022b

[raykov]

hey everyone, the new version of tzdata was recently released: 2022c

 Changes to future timestamps

 Chile's 2022 DST start is delayed from September 4 to September 11.
 (Thanks to Juan Correa.)

 Iran plans to stop observing DST permanently, after it falls back
 on 2022-09-21.  (Thanks to Ali Mirjamali.)

We have a discrepancy right now because month ago Chile decided to move DST one week further :(

[dmitshur]

Go tip is currently at 2022b, which includes the mentioned Chile DST start change, and 2022c is described to work around an awk bug but otherwise not contain further timezone changes.

@raykov Please file a new issue and describe the discrepancy you're seeing in more detail (please answer the "what did you expect to see" and "what did you see instead" questions in the template), and what Go version(s) you're using. Thanks.

[gopherbot]

Change https://go.dev/cl/453055 mentions this issue: lib/time, time/tzdata: update to 2022f

[gopherbot]

Change https://go.dev/cl/454815 mentions this issue: lib/time, time/tzdata: update to 2022g

[gopherbot]

Change https://go.dev/cl/455356 mentions this issue: lib/time: update to 2022g/2022g

[gopherbot]

Change https://go.dev/cl/455357 mentions this issue: time/tzdata: generate zip constant during cmd/dist

[qmuntal]

We should also update time/zoneinfo_abbrs_windows.go when tzdata is updated so #58113 does not happen again.

Should I keep #58113 open to track the update or should it be tracked in this issue?

[dmitshur]

@qmuntal Do you know the frequency of windowsZones.xml (the source for zoneinfo_abbrs_windows.go) changes? If it's tied to the same tzdata release schedule at https://www.iana.org/time-zones then it makes sense to unify the two issues, but if it's a different schedule then maybe keeping the issues separate is fine.