JuliaDynamics / Agents.jl

Agent-based modeling framework in Julia
https://juliadynamics.github.io/Agents.jl/stable/
MIT License
729 stars 117 forks source link

Precompile some model examples reducing TTFX #829

Closed Tortar closed 1 year ago

Tortar commented 1 year ago

This addresses a comment in https://github.com/JuliaDynamics/Agents.jl/issues/818 .

I think it's a very good improvement in TTFX with little side effects as I will illustrate:

Before pr:

@time using Agents # -> 3.466778

After pr

@time using Agents # -> 3.798331

so there is a little regression, however running this:

using Agents
models = [Models.flocking, Models.schelling, Models.zombies, Models.sir]

for m in models
    t = @time model, agent_step!, model_step! = m(); step!(model, agent_step!, model_step!, 10)
end

before:

  0.501026 seconds (1.27 M allocations: 87.205 MiB, 6.06% gc time, 99.95% compilation time)
  0.353901 seconds (1.15 M allocations: 76.176 MiB, 2.58% gc time, 99.96% compilation time)
  7.463354 seconds (13.40 M allocations: 879.895 MiB, 4.50% gc time, 99.23% compilation time: 4% of which was recompilation)
  1.325875 seconds (2.77 M allocations: 183.958 MiB, 3.19% gc time, 99.75% compilation time)

after:

  0.012826 seconds (2.43 k allocations: 129.844 KiB, 98.79% compilation time)
  0.011626 seconds (436 allocations: 33.000 KiB, 99.18% compilation time)
  0.200568 seconds (610.16 k allocations: 44.531 MiB, 10.97% gc time, 61.06% compilation time)
  0.007742 seconds (19.04 k allocations: 2.683 MiB, 61.97% compilation time)

So I think a win overall

Precompiling the models in the Models folder makes the examples models almost without latency which is good for users using the library for the first time

codecov-commenter commented 1 year ago

Codecov Report

Merging #829 (7aeea19) into main (6902229) will increase coverage by 0.68%. The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #829      +/-   ##
==========================================
+ Coverage   70.01%   70.70%   +0.68%     
==========================================
  Files          42       43       +1     
  Lines        2708     2717       +9     
==========================================
+ Hits         1896     1921      +25     
+ Misses        812      796      -16     
Impacted Files Coverage Δ
src/Agents.jl 100.00% <ø> (ø)
src/precompile.jl 100.00% <100.00%> (ø)

... and 2 files with indirect coverage changes

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

Tortar commented 1 year ago

mmmmhhhh I'm actually not sure that this is as good as it was intended to be since running https://julialang.github.io/PrecompileTools.jl/stable/#Seeing-what-got-precompiled we get e.g.

MethodInstance for Agents.add_agent!(::Agents.UnremovableABM{Agents.ContinuousSpace{2, true, Float64, typeof(Agents.no_vel_update)}, Main.Agents.Models.Bird, Agents.Schedulers.Randomly, Nothing, Random.TaskLocalRNG}, ::Tuple{Float64, Float64}, ::Float64, ::Float64, ::Float64, ::Float64, ::Float64, ::Float64)

which is actually not reusable in other contexts, indeed for not precompiled examples (e.g. predator_prey) the time is very similar with or without precompilation.

The advantage of fast tutorial examples seems still something to consider, but don't know if worth the (even if little) regression in importing

Datseris commented 1 year ago

Yeah it's because practically all functions in Agents.jl dispatch on the Agent type, which is almost certainly different in each application. So pre-compiling is very unlikely to lead to any difference in user developed models...

Datseris commented 1 year ago

But thanks a lot for trying this out!