JuliaIO / JLD2.jl

HDF5-compatible file format in pure Julia
Other
534 stars 82 forks source link

Add @nospecializeinfer around worst offenders #527

Closed JonasIsensee closed 5 months ago

JonasIsensee commented 6 months ago

This improves time to first execution in particular when handling many types. Resolves the ancient issue #2 (this is where the test code below is from)

JLD2 v0.4.41

   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.10.0 (2023-12-25)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> include("jldtest.jl")
Precompiling JLD2
  1 dependency successfully precompiled in 18 seconds. 10 already precompiled.
  1.474348 seconds (2.41 M allocations: 157.608 MiB, 13.24% gc time, 99.76% compilation time)
  0.213206 seconds (358.63 k allocations: 23.462 MiB, 3.82% gc time, 99.71% compilation time)
 43.048239 seconds (100.61 M allocations: 6.450 GiB, 3.16% gc time, 99.84% compilation time)

julia> include("jldtest.jl")
  0.000466 seconds (1.35 k allocations: 54.711 KiB)
  0.000492 seconds (1.26 k allocations: 51.461 KiB)
  0.006597 seconds (27.15 k allocations: 1.185 MiB)

This PR

   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.10.0 (2023-12-25)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> include("tupletest.jl")
  0.223693 seconds (122.60 k allocations: 8.262 MiB, 34.51% gc time, 97.49% compilation time)
  0.178638 seconds (182.30 k allocations: 11.534 MiB, 94.68% compilation time)
  0.543427 seconds (692.44 k allocations: 46.006 MiB, 2.65% gc time, 88.67% compilation time)

julia> include("tupletest.jl")
  0.000982 seconds (2.88 k allocations: 127.492 KiB)
  0.007700 seconds (21.34 k allocations: 930.008 KiB)
  0.012911 seconds (47.05 k allocations: 2.141 MiB)

fixes #2

Loading has also been improved significantly. The file from above: JLD2 v0.4.41:

julia> @time load("valtypes.jld2");
 17.044142 seconds (9.38 M allocations: 647.871 MiB, 1.51% gc time, 99.19% compilation time)

julia> @time load("valtypes.jld2");
  0.045572 seconds (84.62 k allocations: 10.131 MiB, 23.01% gc time)

this PR:

julia> @time load("valtypes.jld2");
  0.748558 seconds (690.43 k allocations: 51.296 MiB, 15.69% gc time, 94.32% compilation time)

julia> @time load("valtypes.jld2");
  0.036137 seconds (88.80 k allocations: 10.255 MiB)
codecov[bot] commented 6 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (f8a9dd3) 86.90% compared to head (9614369) 86.85%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #527 +/- ## ========================================== - Coverage 86.90% 86.85% -0.06% ========================================== Files 31 31 Lines 4276 4289 +13 ========================================== + Hits 3716 3725 +9 - Misses 560 564 +4 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.