JuliaTime / TimeZones.jl

IANA time zone database access for the Julia programming language
Other
86 stars 51 forks source link

Optimize walk performance #465

Closed omus closed 1 month ago

omus commented 1 month ago

In reviewing _reload_tz_cache I noticed some minor improvements that could be done to the directory walking code. An example which just looks at the walk performance:

using TimeZones

# Emulating original walk
function reload_walk(root_dir)
    results = String[]
    check = Tuple{String,String}[(root_dir, "")]
    for (dir, partial) in check
        for filename in readdir(dir)
            startswith(filename, ".") && continue

            path = joinpath(dir, filename)
            name = isempty(partial) ? filename : join([partial, filename], "/")

            if isdir(path)
                push!(check, (path, name))
            else
                push!(results, name)
            end
        end
    end
    return results
end

function walk_tz_dir_demo(dir)
    results = String[]
    TimeZones.walk_tz_dir(dir) do name, path
        push!(results, name)
    end
    return results
end

Benchmarks on Julia 1.10.3 with a M1 MacBook:

julia> @btime collect(walkdir(TimeZones._COMPILED_DIR[]));
  4.009 ms (3332 allocations: 405.84 KiB)

julia> @btime reload_walk(TimeZones._COMPILED_DIR[]);
  2.067 ms (6419 allocations: 438.91 KiB)

julia> @btime walk_tz_dir_demo(TimeZones._COMPILED_DIR[]);
  1.962 ms (3089 allocations: 275.15 KiB)

The time difference is minor put a good drop in allocations

Overall benchmarks using master 1b8057c

julia> @time_imports using TimeZones
      0.5 ms  Scratch
      4.2 ms  InlineStrings
      0.4 ms  TZJData
      0.4 ms  Compat
      0.2 ms  Compat → CompatLinearAlgebraExt
      0.3 ms  ExprTools
      0.6 ms  Mocking
               ┌ 0.6 ms TimeZones.TZData.__init__()
               ├ 0.0 ms TimeZones.__init__()
     34.4 ms  TimeZones 37.87% compilation time

julia> @btime TimeZone("UTC");
  18.578 ns (0 allocations: 0 bytes)

julia> @btime TimeZone("America/Winnipeg");
  29.523 ns (2 allocations: 96 bytes)

julia> @btime istimezone("Europe/Warsaw");
  75.111 ns (1 allocation: 48 bytes)

julia> @btime TimeZones._reload_tz_cache(TimeZones._COMPILED_DIR[]);
  21.029 ms (313574 allocations: 12.52 MiB)

Overall using PR 570e1c8

julia> @time_imports using TimeZones
      0.4 ms  Scratch
      3.8 ms  InlineStrings
      0.6 ms  TZJData
      0.6 ms  Compat
      0.3 ms  Compat → CompatLinearAlgebraExt
      0.4 ms  ExprTools
      0.5 ms  Mocking
               ┌ 0.6 ms TimeZones.TZData.__init__()
               ├ 0.0 ms TimeZones.__init__()
     32.0 ms  TimeZones 36.85% compilation time

julia> @btime TimeZone("UTC");
  18.578 ns (0 allocations: 0 bytes)

julia> @btime TimeZone("America/Winnipeg");
  29.523 ns (2 allocations: 96 bytes)

julia> @btime istimezone("Europe/Warsaw");
  75.438 ns (1 allocation: 48 bytes)

julia> @btime TimeZones._reload_tz_cache(TimeZones._COMPILED_DIR[]);
  20.912 ms (310793 allocations: 12.38 MiB)

julia> @btime TimeZones.walk_tz_dir((n, p) -> n, TimeZones._COMPILED_DIR[])
  1.950 ms (3084 allocations: 267.62 KiB)