search

gem / oq-engine

OpenQuake Engine: a software for Seismic Hazard and Risk Analysis
https://github.com/gem/oq-engine/#openquake-engine
GNU Affero General Public License v3.0
380 stars 273 forks source link

Reduced the memory consumption and improved the performance in postclassical #9689

Closed micheles closed 4 months ago

micheles commented 4 months ago

This is only marginally better for New Zealand:

# before
| calc_99956999, maxmem=75.4 GB | time_sec | memory_mb | counts |
|-------------------------------+----------+-----------+--------|
| total postclassical           | 7_860    | 117.1     | 250    |
| combine pmaps                 | 5_033    | 0.0       | 3_991  |
| reading rates                 | 2_272    | 116.4     | 250    |
| compute stats                 | 429.0    | 0.0       | 3_991  |
| ClassicalCalculator.run       | 101.6    | 1_731     | 1      |

# after
| calc_99957000, maxmem=73.3 GB | time_sec | memory_mb | counts |
|-------------------------------+----------+-----------+--------|
| total postclassical           | 6_335    | 110.8     | 250    |
| combine pmaps                 | 5_177    | 0.0       | 3_991  |
| reading rates                 | 540.0    | 110.0     | 250    |
| compute stats                 | 475.9    | 0.0       | 3_991  |
| ClassicalCalculator.run       | 89.3     | 1_712     | 1      |

However it can be a lot better in other cases, since "reading rates" is 4.2x faster.