JuliaDynamics / ABMFrameworksComparison

Benchmarks and comparisons of leading ABM frameworks
Other
10 stars 8 forks source link

Boundary condition in Flockers? #72

Closed Corvince closed 1 year ago

Corvince commented 1 year ago

Sorry, its me again. I am now looking at the comparison between agents.jl and mesa for the flockers model.

Quite some time for mesa is spent on handling the torus adjustments, what is more formally called periodic boundary condition in agents.jl. And I think here something is off with the Agents.jl implementation. I haven't run the agents.jl version locally but looking at animation on the docs page (https://juliadynamics.github.io/Agents.jl/stable/examples/flock/) I observed the following:

Agents should cross boundaries and appear on the other side. However, this seems to only work for single agents. Flock of agents bounce of the wall, probably due to how headings and coherence is modeled. But this shouldn't happen.

Datseris commented 1 year ago

cc @Tortar yeah @Corvince is right, the flocks seem to "bounce off" the edges. I wonder if this has anything to do with the sequence of the schedulling, or the sequence with which we calculate the vectors for coherence etc.

In Agents.jl handling periodic boundary conditions have little performance impact versus the non-periodic ones. One needs to do a bounds check in closed, and one needs to perform the modulo operation in periodic. I doubt this difference would equate the performance disparity between mesa and agents.jl, but of course one would have to check.

But I thought that the ABM rules we used were identical with Mesa and Agents.jl...?

Corvince commented 1 year ago

Thanks for confirming. Yes I wouldn't assume this would make the performance difference significantly smaller (no way to tell), but I think it does have an impact. Not necessarily moving beyond the border, but I think the heading and neighbors calculations are a bit more involved. At least that's my impression from the mesa code. But I haven't been involved in writing any continuous space code, so I never completely got my head around these issues

Tortar commented 1 year ago

thanks @Corvince for the bug report!!

the problem is that the heading is not calculated correctly, the one there is now is a non-periodic space version. Will do a PR in a minute (this should be done also in the Agents repo), here you can see a new output where the behaviour is correct

https://github.com/JuliaDynamics/ABM_Framework_Comparisons/assets/68152031/cb1a9490-5cca-47ed-addc-17da348a735e

Tortar commented 1 year ago

Actually I found another incoherency in the Mesa implementation which I included in the PR...in the end Mesa is slower than before, actually 200x slower not 100x

Tortar commented 1 year ago

Hi @Corvince, I have to say that your contributions to this repo have been invaluable, finding bugs and problems I didn't see in all three models you analyzed, thank you really much for that. For this reason, I'd like to ask you if you could take a look at the last remaining model: the WolfSheep one. Always if you have the time to do so obviously, thank you very much for the past contributions anyway.

Datseris commented 1 year ago

I agree. I only wish other developers from other ABM frameworks have participated and helped more here like @Corvince ...

Corvince commented 1 year ago

Woah, thanks for the nice words!

I have had quite some fun so far. I haven't looked at Wolf-Sheep yet, because it's not as slow as the others in mesa and I was primarily looking for potential bottlenecks. And I haven't fully looked into flockers yet. I think I will do that first before I take a look at wolf sheep, but with some time I can do both.

Btw, are there any updates regarding further models to include in the comparison?

Datseris commented 1 year ago

Btw, are there any updates regarding further models to include in the comparison?

You can always open a new issue with a description for a model and suggest new models to compare to, and the merit there would be in comparing them. They need to be different enough from existing comparisons, so they can highlight fundamentally different aspects.

There are currently the issues:

that need more discussion to become more concrete.

In the end of the day the only thing we need is a DECLARATION.md file, like the existing ones, that declares the model. And of course we would need the developers to add the model. Me and @Tortar should not be spending more time implementing models of other frameworks - we have already done enough for this comparison. We will only be implementing the Julia versions of the models.

Tortar commented 1 year ago

I think a model which highlight new features both implemented in Agents.jl and Mesa should be picked, not to waste too much time at doing comparisons, maybe adding an even larger version for all models as asked in #46 .

Struggling to find something suitable though (added the graph-based models in Mesa-examples in #70, maybe we should implement one of them in Agents.jl)

Tortar commented 1 year ago

Whenever you find the time @Corvince I will be glad to review what you come up with (and probably change something because there is a fault ;D)