search

projectmesa / mesa

Mesa is an open-source Python library for agent-based modeling, ideal for simulating complex systems and exploring emergent behaviors.
https://mesa.readthedocs.io
Other
2.37k stars 863 forks source link

Make mesa faster #882

Closed ghost closed 4 days ago

ghost commented 4 years ago

What's the problem this feature will solve? Sadly, Python belongs to the slowest programming languages, but it's friendly code makes it popular and extremely usable. Agent-based simulations can involve very large number of agents (e.g. millions), where concurrency control, parallelism and run-time speed of simulations could be crucial.

Describe the solution you'd like Using:

could help a lot in simulation speed ups.

rht commented 4 years ago

This is the first time PyPy is mentioned in this repo. Disclaimer: I am also one of the maintainers of abcEconomics.

It should be easy to test if Mesa runs on PyPy with Travis (see e.g. https://github.com/AB-CE/abce/blob/master/.travis.yml#L24).

rht commented 4 years ago

In abcEconomics, there is an option to run the execution of each agents in a separate CPU (https://github.com/AB-CE/abce/blob/master/abcEconomics/scheduler/multiprocess.py). If there are 20 agents and 4 CPUs, each CPU could execute 5 agents sequentially, but the 4 CPUs are doing it in parallel independently of each other. See any pool.map in the file I linked before. Care is needed to ensure that the objects are able to be pickled so they can run in each CPUs.

Corvince commented 4 years ago

There is also a discussion about mesas performance in #798.

To recap my 2 cents on this issue: Mesa is mostly a framework and by itself it doesn't "do" much that could be improved by your suggestions. It really depends on the models it is used for (and if the actions are computationally expensive numba might be a very good and easy to use recommendation).

Re parallelism: Depending on the model running agents concurrently is either trivial or (almost) impossible. But usually you will run multiple simulations any way and running those in parallel is far easier, either manually or by using BatchrunnerMP (although it doesn't work at the moment, but it is being worked on).

rht commented 1 year ago

It's worth trying Cinder. Currently discussed in https://news.ycombinator.com/item?id=36612835.

adamamer20 commented 8 months ago

addressing this issue, I started to develop an extension of mesa which makes use of vectorized operations on pandas dataframes to achieve faster runtimes. you can provide a feedback or features you'd like to be supported/introduced here

EwoutH commented 4 days ago

I'm going to close this issue as completed, now that we have mesa-frames officially under the Mesa umbrella:

If there are any specific issues or ideas for Mesa performance, feel free to open a new issue or discussion!