wschwanghart / topotoolbox

A MATLAB software for the analysis of digital elevation models -
https://topotoolbox.wordpress.com/
153 stars 89 forks source link

Waitbar in polygon2GRIDobj closes waitbar in external functions #22

Closed kerryleith closed 1 year ago

kerryleith commented 2 years ago

I'm working on a polygon - grid - polygon workflow. To save memory, I'm running polygon2GRIDobj on extracted segments of my DEM (if the MS field value is a double, the grid size blows up in memory).

For some reason the waitbar in polygon2GRIDobj closes the waitbar I create (2019b). Can we add a waitbar true / false parameter as in quantcarve (addParameter(p,'waitbar',true);)

f1 = waitbar(0, 'Separating landslide source and deposit polygons'); for i = 1: size(MS, 1) waitbar(i/size(MS, 1), f1); %cut the landslide area out of the DEM DEMcrop = crop(DEM, MS(i).BoundingBox(1:2, 1), MS(i).BoundingBox(1:2, 2)); %grid the landslide polygon P = polygon2GRIDobj(DEMcrop,MS(i)); %etc end

Thanks!

schmittrjp commented 2 years ago

Hi @kerryleith did you ever find a solution for this? I noticed that the memory problem is purely related to the linear index of the pour point, which is then assigned to all upstream cells in the drainage raster, E.g., I have 27 pour points, and the largest linear index of any of those is 248283526. Then, the regionprops function (called within polygon2GRIDobj) tries to create a 27x248283526 (49.9GB) array and fails.

One approach is to remap the values of the drainage basin raster to a smaller number, e.g., some unique ID of the pourpoints. For that, I firstly create the drainage raster ("LRiverbasins" in my case). My pourpoints are stored in a mapstruct with an .ID field from 1...27 and a .LinInd field that stores each pourpoint's linear index.

Then you can do

[~,locb]=ismember(Lriverbasins.Z(:),[Pourpoints.LinInd]); Lriverbasins_r=Lriverbasins; Lriverbasins_r.Z(locb>0)=[RiverBasinOutlets(locb(locb>0)).ID];

That does not exactly solve your waitbar problem but might be a way around the segmentation of your DEM.

@wschwanghart I wonder if it would be possible to change the drainagebasin function in a way that the cells draining to a pourpoint could be assigned a user-defined values rather than the linear index. E.g., drainagebasins(FD,IX,desired_basin_ID_value)

kerryleith commented 2 years ago

Hi @schmittrjp, yes, and no. I saw that regionprops was causing the problem, and figured it was likely out of our control, so just modified a version of polygon2GRIDobj to disable the waitbar. I figured it's consistent with other TT functions, and was easy enough for @wschwanghart to include later. That said, it's a painfully slow way of solving the workflow, your suggestion could be a good way to go. Thanks!

schmittrjp commented 2 years ago

Interesting. Let me know if the workaround works for you. Something must have changed as I did not have that problem before.

wschwanghart commented 1 year ago

@kerryleith: I have now updated zonalstats and polygon2GRIDobj to enable control over waitbar. By default, both functions display waitbars. If you want to switch off the waitbar in polygon2GRIDobj, note that there is a slight change in syntax in how you provide a field to be mapped. This was necessary to ensure backward compatibility. Sorry that it really took a while to implement it.

kerryleith commented 1 year ago

Thanks @wschwanghart! Does exactly what it says on the packet.