search

OceanParcels / parcels

Main code for Parcels (Probably A Really Computationally Efficient Lagrangian Simulator)
http://www.oceanparcels.org
MIT License
286 stars 122 forks source link

Performance of IO-Loop #473

Closed willirath closed 4 years ago

willirath commented 5 years ago

Parcels spends a lot of time in https://github.com/OceanParcels/parcels/blob/master/parcels/particlefile.py#L161_L170.

For 12-hourly output and a 5-minute time step, the Agulhas example spends 95% of the time in this loop. (See this gist for details.)

To me, the root of the problem seems to be the fact that this loop supports the most general case (particles being killed / created at runtime) in an explicit way.

willirath commented 5 years ago

(cc: @ohlmann and @phandmann This is the IO-bottleneck I was referring to.)

willirath commented 5 years ago

(cc: @handmannp)

delandmeterp commented 5 years ago

This should be fixed in the near future. We haven't attacked that bottleneck since two students are working on it for a project. Outcomes should come within two weeks.

willirath commented 5 years ago

That's great to hear. Then as a next step, we can talk about OpenMP parallelization of the particle loop. :)

erikvansebille commented 4 years ago

This has been fixed now in #486 and #497