pypsa-meets-earth / pypsa-earth

PyPSA-Earth: A flexible Python-based open optimisation model to study energy system futures around the world.
https://pypsa-earth.readthedocs.io/en/latest/
225 stars 176 forks source link

Create speed measurement infrastructure #996

Open pz-max opened 6 months ago

pz-max commented 6 months ago

Describe the feature you'd like to see

Sometimes we make code changes and don't recognize its speed impacts.

It would be good to track for each rule (and ideally each function within the rule) the performance comparing PR branch to OS main. I saw some Python packages doing that in the past (just remembered they used Azure infra) and reading some UNIX & C wisdoms measuring that stuff is generally a good idea.

ekatef commented 6 months ago

Describe the feature you'd like to see

Sometimes we make code changes and don't recognize its speed impacts.

It would be good to track for each rule (and ideally each function within the rule) the performance comparing PR branch to OS main. I saw some Python packages doing that in the past (just remembered they used Azure infra) and reading some UNIX & C wisdoms measuring that stuff is generally a good idea.

Absolutely agree. It would be not a good idea to compromise the performance, given the goal to keep at least continental scope.

As a clarification question: do you mean profiling methods integrated into the development workflow? Or do you also have in mind something more deep, like integration wit computational/platform infrastructure?