search

arup-group / pam

Generate and modify transport demand scenarios via a Python API.
MIT License
55 stars 18 forks source link

Airspeed Velocity benchmarking #260

Open brynpickering opened 9 months ago

brynpickering commented 9 months ago

What can be improved?

We currently have some basic time and memory benchmarking in our CI (see e.g. #258 #243) that rely on running one time/memory hungry example. I just saw that pandas and numpy use airspeed velocity for their benchmarking (see here for benchmark results that led to the long runtimes fixed in #258), which might provide us with a more feature-complete suite of time/memory consumption results for different parts of PAM.

Here is an example of benchmarking results in a PR CI action and a here is a discussion on how to implement it in a CI.

Version

0.2.5-dev

brynpickering commented 9 months ago

Digging into it a bit more, it might be a little outdated (pandas/numpy started working with it ~2015) and only pandas seems to have a hook to keep their benchmarking website up and running (asv-runner). However, the CI action might be an easy win to have a wider spread of time/memory checks.