search

landam / grass-gis-git-migration-test

0 stars 0 forks source link

r.fillnulls: per hole filling speed-up #88

Open landam opened 5 years ago

landam commented 5 years ago

Reported by sbl on 19 Apr 2013 10:20 UTC When running on a large grid (60,000 x 50,000 cells) with lots of NoData areas (52,000 holes) r.fillnulls is very slow. I figured out, that there was significant potential for speedup in the way r.fillnulls applies region cropping (the way it moves the region from hole to hole).

Therefore I would like to propose a speedup provided by the attached patch.

Main difference is, that the raster based region cropping (which ran a map calculator expression on (nearly) the entire input raster) within the loop over the holes is replaced by a vector based approach (similar to the one used in r.connectivity.distance). Here, before r.fillnulls loops over the holes, the latter are converted to vector (r.to.vect). Within the loop the focal hole is extracted based on category (v.extract) and the region is adjusted to this single polygon. By doing so, the r.mapcalc function for raster based region cropping could be replaced by the (significantly faster v.extract). Finally, if I am not mistaken the two runs of g.region could be joined to one (though this should not have major impact on the processing speed).

GRASS GIS version and provenance

svn-trunk

Migrated-From: https://trac.osgeo.org/grass/ticket/1938

landam commented 5 years ago

Attachment from sbl on 19 Apr 2013 10:21 UTC

https://trac.osgeo.org/grass/attachment/ticket/1938/r.fillnulls.patch

landam commented 5 years ago

Attachment from sbl on 1 Jul 2014 22:28 UTC Some more speed-up https://trac.osgeo.org/grass/attachment/ticket/1938/r.filnulls2.diff

landam commented 5 years ago

Comment by marisn on 6 Jul 2013 17:17 UTC A slightly modified version has been applied in https://trac.osgeo.org/grass/changeset/57030

Thanks, Stefan!

Next speed up for r.fillnulls would be if someone would implement parallel filling of holes.

landam commented 5 years ago

Comment by sbl on 1 Jul 2014 22:48 UTC Please find attached another proposal for speed-up of rst-interpolation by hole.

In the diff, there are two sections which can probably be removed in addition:

1) The new r.clump version should account for NoData, so the following r.mapcalc operation can be removed (could not test it since I work with an older version).

2) I was unsure if r.patch has a limit regarding the number of open maps, if not than block-wise application of r.patch can be removed too...

Maybe better to use bilinear or bicubic interpolation as default as it is significant faster (and simpler) than rst?

landam commented 5 years ago

Comment by neteler on 5 May 2016 14:08 UTC Milestone renamed

landam commented 5 years ago

Comment by neteler on 28 Dec 2016 15:04 UTC Ticket retargeted after milestone closed

landam commented 5 years ago

Modified by @landam on 5 May 2017 20:41 UTC

landam commented 5 years ago

Comment by @landam on 1 Sep 2017 20:28 UTC All enhancement tickets should be assigned to 7.4 milestone.

landam commented 5 years ago

Comment by neteler on 26 Jan 2018 11:40 UTC Ticket retargeted after milestone closed

landam commented 5 years ago

Comment by @landam on 28 Feb 2018 20:42 UTC See also https://grass.osgeo.org/grass74/manuals/r.fill.stats.html

landam commented 5 years ago

Modified by neteler on 12 Jun 2018 20:48 UTC

landam commented 5 years ago

Comment by neteler on 24 Sep 2018 20:36 UTC For a while r.fillnulls optionally uses r.resamp.bspline (https://trac.osgeo.org/grass/ticket/1088), anything left here?

landam commented 5 years ago

Comment by @landam on 25 Sep 2018 16:51 UTC All enhancement tickets should be assigned to 7.6 milestone.

landam commented 5 years ago

Comment by @landam on 25 Jan 2019 21:08 UTC Ticket retargeted after milestone closed