AlexLipp / thames-sewage

Realtime mapping of sewage release events in Thames basin
GNU General Public License v3.0
11 stars 0 forks source link

Speed up the i/o of the drainage network #2

Closed AlexLipp closed 10 months ago

AlexLipp commented 1 year ago

Create a load_from_node_arrays function in autocatchments so the network stack doesn't need to be built every run.

AlexLipp commented 1 year ago

Such a function is now implemented in autocatchments and results in a 2x speed but result is undesirable as:

Instead, more desirable speed ups can be gained by:

AlexLipp commented 1 year ago

PR #8 has partially increased speed up (3-4x) by use of pickling but it can be improved more by using a lower overheads object than the RMG

AlexLipp commented 1 year ago

Easy speed-ups could be gained by:

  1. Downsampling the input DEM and preprocessing it in advance (i.e. sink-filling)
  2. Cutting the study area down to be more tightly along the Thames Basin

More challenging speed-ups could be gained by cythonizing the channel-profiler