search

r-barnes / richdem

High-performance Terrain and Hydrology Analysis
GNU General Public License v3.0
272 stars 70 forks source link

Multithreading possible for Depression-Filling? #35

Open jfbourdon opened 4 years ago

jfbourdon commented 4 years ago

The documentation of RichDEM states that it "uses parallel processing and state of the art algorithms to quickly process even very large DEMs." I'm amazed at how fast it is even with large high resolution DEMs compared to the tools I used before. However, for the tool Depression-Filling for example, it didn't use parallel processing (the progress bar also states 1 thread used). Is there a parameter that I need to set prior to running Depression-Filling or is it that the task cannot be multithreaded (or is not multithreaded yet)? Note that I currently use RichDEM as a Python package on Windows.

Don't get my wrong, RichDEM as it is now is very fast, I'm just on the impression that it can be even faster.

feefladder commented 2 years ago

The depression-filling algorithms use a (single) priority queue, so that is really hard to multi-thread. I think this is also difficult to make parallel in the near future

jfbourdon commented 2 years ago

It makes sense, thanks.

I suppose the raster could be split internally into tiles which would be processed by each core available (4 cores = 4 tiles), then merged back together before having a final (single-core) pass on the whole raster. My reasoning here is that most depressions would already be filled and only depressions crossings tiles would have to be filled (a second time). I don't know if it would be more efficient though, It might depends on how much filling has to be done.

feefladder commented 2 years ago

Oops I was talking bs. Actually in #22 someone tried to do that. and there is documentation here. you could maybe give it a shot? (not in Python though)