Dates/types test fails on v1.10

[d-netto]

julia> Base.runtests(["Dates/types"])
Running parallel tests with:
  nworkers() = 1
  nthreads() = 1
  Sys.CPU_THREADS = 4
  Sys.total_memory() = 64.000 GiB
  Sys.free_memory() = 9.855 GiB

Test    (Worker) | Time (s) | GC (s) | GC % | Alloc (MB) | RSS (MB)
Dates/types  (1) |        started at 2024-09-20T16:17:15.276
Dates/types  (1) |         failed at 2024-09-20T16:17:16.087
Test Failed at /Users/dnetto/RAI/julia-master/usr/share/julia/stdlib/v1.10/Dates/test/types.jl:265
  Expression: time == Dates.Time(10, 26, 14)
   Evaluated: Time(11, 26, 14) == Time(10, 26, 14)

Test Summary: | Pass  Fail  Total  Time
  Overall     |  232     1    233  1.4s
    Dates/types |  232     1    233  1.1s
    FAILURE

The global RNG seed was 0x5fd0ba6d8d1d31594a951734c5156f97.

Error in testset Dates/types:
Test Failed at /Users/dnetto/RAI/julia-master/usr/share/julia/stdlib/v1.10/Dates/test/types.jl:265
  Expression: time == Dates.Time(10, 26, 14)
   Evaluated: Time(11, 26, 14) == Time(10, 26, 14)

ERROR: LoadError: Test run finished with errors
in expression starting at /Users/dnetto/RAI/julia-master/usr/share/julia/test/runtests.jl:95
ERROR: A test has failed. Please submit a bug report (https://github.com/JuliaLang/julia/issues)
including error messages above and the output of versioninfo():
Julia Version 1.10.5
Commit 6f3fdf7b36* (2024-08-27 14:19 UTC)
Build Info:

    Note: This is an unofficial build, please report bugs to the project
    responsible for this build and not to the Julia project unless you can
    reproduce the issue using official builds available at https://julialang.org/downloads

Platform Info:
  OS: macOS (arm64-apple-darwin23.4.0)
  CPU: 12 × Apple M2 Max
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, apple-m1)
Threads: 1 default, 0 interactive, 1 GC (on 8 virtual cores)
...

[giordano]

Is that reproducible for you? It isn't for me:

julia> Base.runtests("Dates/types"; seed=0x5fd0ba6d8d1d31594a951734c5156f97)
Running parallel tests with:
  nworkers() = 1
  nthreads() = 1
  Sys.CPU_THREADS = 2
  Sys.total_memory() = 16.000 GiB
  Sys.free_memory() = 1.231 GiB

Test    (Worker) | Time (s) | GC (s) | GC % | Alloc (MB) | RSS (MB)
Dates/types  (1) |        started at 2024-09-20T23:30:25.192
Dates/types  (1) |     0.67 |   0.01 |  1.3 |      66.48 |   306.53

Test Summary: | Pass  Total  Time
  Overall     |  233    233  1.2s
    SUCCESS

julia> versioninfo()
Julia Version 1.10.5
Commit 6f3fdf7b362 (2024-08-27 14:19 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: macOS (arm64-apple-darwin22.4.0)
  CPU: 8 × Apple M1
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, apple-m1)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

You'll need to provide more information to address the issue.

[d-netto]

Yes, reproduces reliably on my M2.

Works fine on a Linux x86_64 machine though:

julia> Base.runtests(["Dates/types"])
Running parallel tests with:
  nworkers() = 1
  nthreads() = 1
  Sys.CPU_THREADS = 18
  Sys.total_memory() = 68.568 GiB
  Sys.free_memory() = 66.711 GiB

Test    (Worker) | Time (s) | GC (s) | GC % | Alloc (MB) | RSS (MB)
Dates/types  (1) |        started at 2024-09-20T23:33:44.658
Dates/types  (1) |     1.41 |   0.02 |  1.2 |      66.35 |   345.91

Test Summary: | Pass  Total  Time
  Overall     |  233    233  2.6s
    SUCCESS

julia> versioninfo()
Julia Version 1.10.5
Commit 6f3fdf7b36 (2024-08-27 14:19 UTC)
Build Info:

    Note: This is an unofficial build, please report bugs to the project
    responsible for this build and not to the Julia project unless you can
    reproduce the issue using official builds available at https://julialang.org/downloads

Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 36 × Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, skylake-avx512)
Threads: 1 default, 0 interactive, 1 GC (on 36 virtual cores)

[giordano]

So you're saying that for you

julia> Libc.strptime("%Y-%M-%dT%H:%M:%SZ", "2018-11-16T10:26:14Z")
Base.Libc.TmStruct(14, 26, 10, 16, 0, 118, 2, 15, 0, 0, 0, 0, 1208009808, 1)

julia> Time(Libc.strptime("%Y-%M-%dT%H:%M:%SZ", "2018-11-16T10:26:14Z"))
10:26:14

returns 11:26:14? What timezone do you have on your machine?

[d-netto]

returns 11:26:14?

Yes:

julia> using Dates

julia> Libc.strptime("%Y-%M-%dT%H:%M:%SZ", "2018-11-16T10:26:14Z")
Base.Libc.TmStruct(14, 26, 11, 16, 0, 118, 2, 15, 1, 0, -7200, -1, 1325674060, 1)

julia> Time(Libc.strptime("%Y-%M-%dT%H:%M:%SZ", "2018-11-16T10:26:14Z"))
11:26:14

What timezone do you have on your machine?

GMT-3.

[d-netto]

Even the C version seems strange on my M2?

#define _XOPEN_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>

int
main(void)
{
    struct tm tm;
    char buf[255];

    memset(&tm, 0, sizeof(tm));
    strptime("2018-11-16T10:26:14Z", "%Y-%M-%dT%H:%M:%SZ", &tm);
    // Pretty print the tm struct
    strftime(buf, sizeof(buf), "%Y-%M-%dT%H:%M:%SZ", &tm);
    puts(buf);
    exit(EXIT_SUCCESS);
}

Note how it got the month wrong:

2018-26-16T10:26:14Z

[giordano]

This should be a reproducer of what strptime does internally on macOS:

julia> tm = Libc.TmStruct()
Base.Libc.TmStruct(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)

julia> r = @ccall strptime("2018-11-16T10:26:14Z"::Cstring, "%Y-%M-%dT%H:%M:%SZ"::Cstring, tm::Ref{Libc.TmStruct})::Cstring
Cstring(0x000000010eef22ac)

julia> @ccall mktime(tm::Ref{Libc.TmStruct})::Int
1516098374

julia> tm
Base.Libc.TmStruct(14, 26, 10, 16, 0, 118, 2, 15, 0, 0, 0, 0, 1208009808, 1)

Even the C version seems strange on my M2?

Sounds like a problem on your system then, although it'd be good to understand what exactly, to make this test more robust in weird situations.

[giordano]

Uhm, maybe using %M for both month and minutes is wrong...

[giordano]

What do you get with

julia> Libc.strptime("%Y-%m-%dT%H:%M:%SZ", "2018-11-16T10:26:14Z")
Base.Libc.TmStruct(14, 26, 10, 16, 10, 118, 5, 319, 0, 0, 0, 0, 1208009808, 1)

julia> Time(Libc.strptime("%Y-%m-%dT%H:%M:%SZ", "2018-11-16T10:26:14Z"))
10:26:14

? Note the %m for the month (as it should be).

[d-netto]

Still 11:26:14...

julia> using Dates

julia> Libc.strptime("%Y-%m-%dT%H:%M:%SZ", "2018-11-16T10:26:14Z")
Base.Libc.TmStruct(14, 26, 11, 16, 10, 118, 5, 319, 1, 0, -7200, -1, 588499020, 1)

julia> Time(Libc.strptime("%Y-%m-%dT%H:%M:%SZ", "2018-11-16T10:26:14Z"))
11:26:14

[giordano]

How about your C reproducer, with the correct format string?

[d-netto]

How about your C reproducer, with the correct format string?

Yes, this one works:

#define _XOPEN_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>

int
main(void)
{
    struct tm tm;
    char buf[255];

    memset(&tm, 0, sizeof(tm));
    strptime("2018-11-16T10:26:14Z", "%Y-%m-%dT%H:%M:%SZ", &tm);
    // Pretty print the tm struct
    strftime(buf, sizeof(buf), "%Y-%m-%dT%H:%M:%SZ", &tm);
    puts(buf);
    exit(EXIT_SUCCESS);
}

2018-11-16T10:26:14Z

[giordano]

Can you try

julia> tm = Libc.TmStruct()
Base.Libc.TmStruct(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)

julia> r = @ccall strptime("2018-11-16T10:26:14Z"::Cstring, "%Y-%m-%dT%H:%M:%SZ"::Cstring, tm::Ref{Libc.TmStruct})::Cstring
Cstring(0x000000010e45d2ec)

julia> tm
Base.Libc.TmStruct(14, 26, 10, 16, 10, 118, 5, 319, 0, 0, 0, 0, 0, 0)

julia> @ccall mktime(tm::Ref{Libc.TmStruct})::Int
1542363974

julia> tm
Base.Libc.TmStruct(14, 26, 10, 16, 10, 118, 5, 319, 0, 0, 0, 0, 1208009808, 1)

?

[vtjnash]

Is this the test bug where the code is trying to check that the users timezone on Jan 1 of some random year is the same timezone as the current month? I thought we fixed that test, but maybe we only talked about it

[giordano]

The test was meant to check the construction of a Time object from a TmStruct instance: #31524 (this is where the wrong format string comes from, BTW), no mention to timezones in that issue.

[giordano]

Can you try

Alright, I can reproduce the issue myself if I manually set the timezone to UTC-3:

julia> tm = Libc.TmStruct()
Base.Libc.TmStruct(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)

julia> r = @ccall strptime("2018-11-16T10:26:14Z"::Cstring, "%Y-%m-%dT%H:%M:%SZ"::Cstring, tm::Ref{Libc.TmStruct})::Cstring
Cstring(0x000000010f0b906c)

julia> tm
Base.Libc.TmStruct(14, 26, 10, 16, 10, 118, 5, 319, 0, 0, 0, 0, 0, 0)

julia> @ccall mktime(tm::Ref{Libc.TmStruct})::Int
1542374774

julia> tm
Base.Libc.TmStruct(14, 26, 11, 16, 10, 118, 5, 319, 1, 0, -7200, -1, 1435613772, 1)

It's the mktime call, which happens only on macOS: https://github.com/JuliaLang/julia/blob/911e02558d0c145a192facd28808b68e157aa5af/base/libc.jl#L261-L267 which is screwing up the hour. You couldn't reproduce in C because you weren't calling mktime, and you couldn't reproduce on Linux because mktime isn't called there.

[giordano]

julia> withenv("TZ" => "UTC") do
           Libc.strptime("%Y-%M-%dT%H:%M:%SZ", "2018-11-16T10:26:14Z")
       end
Base.Libc.TmStruct(14, 26, 10, 16, 0, 118, 2, 15, 0, 0, 0, 0, 880856648, 1)

fixes the issue for me. A similar trick is used in https://github.com/JuliaLang/julia/blob/911e02558d0c145a192facd28808b68e157aa5af/stdlib/Dates/test/conversions.jl#L61-L68 to avoid making tests on local timezone on macOS. I'll update #55829.

[d-netto]

Can you try

Seems like you found the culprit already, but in any case:

julia> using Dates

julia> tm = Libc.TmStruct()
Base.Libc.TmStruct(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)

julia> r = @ccall strptime("2018-11-16T10:26:14Z"::Cstring, "%Y-%m-%dT%H:%M:%SZ"::Cstring, tm::Ref{Libc.TmStruct})::Cstring
Cstring(0x000000010fa059fc)

julia> tm
Base.Libc.TmStruct(14, 26, 10, 16, 10, 118, 5, 319, 0, 0, 0, 0, 0, 0)

julia> @ccall mktime(tm::Ref{Libc.TmStruct})::Int
1542374774

julia> tm
Base.Libc.TmStruct(14, 26, 11, 16, 10, 118, 5, 319, 1, 0, -7200, -1, 1166172236, 1)