GLAMOS / dataflow

Code and tools for data exchange between GLAMOS and externals partners.
http://www.glamos.ch
Other
1 stars 3 forks source link

Performance issues during reading mass-balance data caused by reading elevation bands #26

Open yweidmann opened 6 years ago

yweidmann commented 6 years ago

During reading the data of mass-balance readings, all connected elevation-bands are read as well and added to the according mass-balance object. Some glaciers (e.g. Clariden) have a lot of mass-balance readings and elevation-bands.

The performance of the process has to be improved. Possible approaches:

yweidmann commented 6 years ago

The table mass_balance.elevation_distribution didn't had an index on the foreign key Adding the index INDEX "IXFK_elevation_distribution_mass_balance" ON mass_balance.elevation_distribution (fk_mass_balance ASC); increased the performance of 1/3. But still not enough.

Further analysis in the Python code and query planning on the database needed. Currently only 25 transaction per second during the query for the UnitTest.