JuliaDynamics / ABMFrameworksComparison

Benchmarks and comparisons of leading ABM frameworks
Other
10 stars 8 forks source link

Include model instantiation in benchmarks #12

Closed Tortar closed 1 year ago

Tortar commented 1 year ago

I saw that the model instantiation is considered a part of the setup of the benchmarks, but for more accurate comparisons we should instead include it in the benchmark itself. I could easily do this without harm for Agents.jl and Mesa, but can't say the same for the other frameworks :Q Don't know if there was some difficulty when this repo was created in the standardization of the comparisons between frameworks doing the benchmarks this way. At the same time this is a really minor issue probably since the setup probably doesn't cost a lot in all frameworks, will do some experiment including it in Mesa and Agents,jl to see by how much comparisons change if included.

Probably it's not worth the effort.

Tortar commented 1 year ago

Verified on the schelling and flocking models: the times are rather stable so I think that the time taken by the model instantiation is negligible.