JuliaTime / TimeZones.jl

IANA time zone database access for the Julia programming language
Other
86 stars 51 forks source link

Lazily initialize the time zone cache #457

Closed KristofferC closed 1 month ago

KristofferC commented 3 months ago

This avoids having to compile a bunch of code in __init__ and improves the load time of the package (on Julia 1.11) from:

julia> @time_imports import TimeZones
...
              ┌ 2.6 ms TimeZones.TZData.__init__() 92.83% compilation time
               ├ 237.9 ms TimeZones.__init__() 94.11% compilation time (89% recompilation)
    276.3 ms  TimeZones 81.90% compilation time (88% recompilation)
julia> @time_imports import TimeZones
...
     17.5 ms  TimeZones 16.03% compilation time

The change in performance of the functions is (in my opinion) acceptable.

After:

julia> using BenchmarkTools, TimeZones

julia> @btime istimezone("Europe/Warsaw")
  48.068 ns (1 allocation: 16 bytes)
true

Before:

julia> @btime istimezone("Europe/Warsaw")
  45.356 ns (1 allocation: 16 bytes)
true
omus commented 1 month ago

Rebased this PR now that #456 has been merged

omus commented 1 month ago

Posting some benchmarks from my machine so I can do some refactoring without losing the performance gains.

Before this PR with Julia 1.11.0-beta1 on an M1 MacBook

julia> @time_imports import TimeZones
...
               ┌ 4.4 ms TimeZones.TZData.__init__() 53.17% compilation time
               ├ 318.6 ms TimeZones.__init__() 85.10% compilation time (93% recompilation)
    690.8 ms  TimeZones 59.42% compilation time (85% recompilation)

julia> using BenchmarkTools, TimeZones

julia> @btime istimezone("Europe/Warsaw");
  77.314 ns (1 allocation: 16 bytes)

With this PR with Julia 1.11.0-beta1 on an M1 MacBook

julia> @time_imports import TimeZones
...
               ┌ 3.7 ms TimeZones.TZData.__init__() 56.80% compilation time
               ├ 0.0 ms TimeZones.__init__()
    107.0 ms  TimeZones 83.43% compilation time (53% recompilation)

julia> using BenchmarkTools, TimeZones

julia> @btime istimezone("Europe/Warsaw");
  80.966 ns (1 allocation: 16 bytes)
omus commented 1 month ago
Iterating on the `istimezone` internals ### Iteration 1 ```julia function istimezone(str::AbstractString, mask::Class=Class(:DEFAULT)) # Start by performing quick FIXED class test if mask & Class(:FIXED) != Class(:NONE) && occursin(FIXED_TIME_ZONE_REGEX, str) return true end # Checks against pre-compiled time zones tz, class = get(get_tz_cache(), str, (nothing, Class(:NONE))) return tz !== nothing && mask & class != Class(:NONE) end ``` ```julia julia> @time_imports import TimeZones ... ┌ 2.7 ms TimeZones.TZData.__init__() 77.28% compilation time ├ 0.0 ms TimeZones.__init__() 106.6 ms TimeZones 84.49% compilation time (53% recompilation) julia> using BenchmarkTools, TimeZones julia> @btime istimezone("Europe/Warsaw"); 95.032 ns (2 allocations: 64 bytes) ``` ### Iteration 2 ```julia function istimezone(str::AbstractString, mask::Class=Class(:DEFAULT)) # Start by performing quick FIXED class test if mask & Class(:FIXED) != Class(:NONE) && occursin(FIXED_TIME_ZONE_REGEX, str) return true end # Checks against pre-compiled time zones tz_class = get(get_tz_cache(), str, nothing) tz_class === nothing && return false tz, class = tz_class return mask & class != Class(:NONE) end ``` ```julia julia> @time_imports import TimeZones ... ┌ 2.5 ms TimeZones.TZData.__init__() 78.26% compilation time ├ 0.0 ms TimeZones.__init__() 103.5 ms TimeZones 83.75% compilation time (52% recompilation) julia> using BenchmarkTools, TimeZones julia> @btime istimezone("Europe/Warsaw"); 75.789 ns (1 allocation: 16 bytes) ``` ### Iteration 3 ```julia function istimezone(str::AbstractString, mask::Class=Class(:DEFAULT)) # Start by performing quick FIXED class test if mask & Class(:FIXED) != Class(:NONE) && occursin(FIXED_TIME_ZONE_REGEX, str) return true end # Checks against pre-compiled time zones _, class = get(get_tz_cache(), str, (UTC_ZERO, Class(:NONE))) return mask & class != Class(:NONE) end ``` ```julia julia> @time_imports import TimeZones ... ┌ 3.1 ms TimeZones.TZData.__init__() 66.09% compilation time ├ 0.0 ms TimeZones.__init__() 103.3 ms TimeZones 83.89% compilation time (52% recompilation) julia> using BenchmarkTools, TimeZones julia> @btime istimezone("Europe/Warsaw"); 80.492 ns (1 allocation: 16 bytes) ``` ### Iteration 4 ```julia function istimezone(str::AbstractString, mask::Class=Class(:DEFAULT)) # Start by performing quick FIXED class test if mask & Class(:FIXED) != Class(:NONE) && occursin(FIXED_TIME_ZONE_REGEX, str) return true end # Checks against pre-compiled time zones class = get(get_tz_cache(), str, (UTC_ZERO, Class(:NONE)))[2] return mask & class != Class(:NONE) end ``` ```julia julia> @time_imports import TimeZones ... ┌ 2.8 ms TimeZones.TZData.__init__() 79.89% compilation time ├ 0.0 ms TimeZones.__init__() 102.1 ms TimeZones 83.99% compilation time (54% recompilation) julia> using BenchmarkTools, TimeZones julia> @btime istimezone("Europe/Warsaw"); 75.610 ns (1 allocation: 16 bytes) ```
omus commented 1 month ago

Commit 5701f3a3c67c7dd3da9579c22414e9e7236a89b6

julia> @time_imports import TimeZones
...
               ┌ 3.1 ms TimeZones.TZData.__init__() 77.56% compilation time
               ├ 0.0 ms TimeZones.__init__()
    106.5 ms  TimeZones 84.26% compilation time (54% recompilation)

julia> using BenchmarkTools, TimeZones

julia> @btime istimezone("Europe/Warsaw");
  78.689 ns (1 allocation: 16 bytes)

Commit b5948fc5e593d2a52087d5b835e1dcd05876f31d

julia> @time_imports import TimeZones
...
      0.6 ms  Mocking
               ┌ 2.5 ms TimeZones.TZData.__init__() 81.56% compilation time
               ├ 0.0 ms TimeZones.__init__()
    108.0 ms  TimeZones 79.75% compilation time (53% recompilation)

julia> using BenchmarkTools, TimeZones

julia> @btime istimezone("Europe/Warsaw");
  75.703 ns (1 allocation: 16 bytes)

Looks like I broke some tests in the process

codecov-commenter commented 1 month ago

Codecov Report

Attention: Patch coverage is 91.66667% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 92.67%. Comparing base (2bc8f50) to head (d900457). Report is 3 commits behind head on master.

Files Patch % Lines
src/build.jl 0.00% 1 Missing :warning:
src/types/timezone.jl 95.00% 1 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #457 +/- ## ========================================== - Coverage 92.79% 92.67% -0.12% ========================================== Files 39 39 Lines 1818 1830 +12 ========================================== + Hits 1687 1696 +9 - Misses 131 134 +3 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

omus commented 1 month ago

Close to the finish line on this one. I'd like to recheck the performance with the fixed tests yet.