GDAL simplification is slow

Permafacture commented 9 years ago

For 21,000 OGRGeometry linestrings, building the simplifiers takes ~55 seconds and spitting out a filtered vertices takes 112 seconds! The culprit seems to be converting the numpy arrays to strings rather than actual OGRGeometry instantiation.

    69541563 function calls in 112.199 seconds

Ordered by: cumulative time

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
1    0.035    0.035  112.199  112.199 <string>:1(<module>)
21110    0.027    0.000  112.147    0.005 <ipython-input-39-0ef6ac46c01c>:297(from_threshold)
21082    0.104    0.000  110.920    0.005 <ipython-input-39-0ef6ac46c01c>:256(polymask)
373919    0.887    0.000  109.358    0.000 <ipython-input-39-0ef6ac46c01c>:226(pt2str)
21082    0.684    0.000  108.886    0.005 <ipython-input-39-0ef6ac46c01c>:249(poly2wkt)
373919    0.261    0.000  108.089    0.000 numeric.py:1581(array_str)
373919    1.592    0.000  107.828    0.000 arrayprint.py:343(array2string)
373919    5.114    0.000  106.080    0.000 arrayprint.py:233(_array2string)
1121757    2.038    0.000   69.328    0.000 arrayprint.py:533(__init__)
1121757   32.550    0.000   67.290    0.000 arrayprint.py:547(fillFormat)
373919    1.224    0.000   44.817    0.000 arrayprint.py:700(__init__)
373919    2.202    0.000   20.464    0.000 arrayprint.py:465(_formatArray)
747838    9.085    0.000   16.707    0.000 arrayprint.py:598(__call__)
3365271   14.293    0.000   14.293    0.000 {method 'reduce' of 'numpy.ufunc' objects}
3739190    6.252    0.000   14.162    0.000 numeric.py:2320(seterr)
1121757    1.217    0.000   10.797    0.000 fromnumeric.py:1762(any)
1869595    1.429    0.000    8.885    0.000 numeric.py:2705(__enter__)
1869595    1.598    0.000    8.305    0.000 numeric.py:2710(__exit__)
1121757    0.942    0.000    7.566    0.000 {method 'any' of 'numpy.ndarray' objects}
373919    3.341    0.000    6.865    0.000 arrayprint.py:639(__init__)
1121757    0.870    0.000    6.624    0.000 _methods.py:31(_any)
3739190    4.551    0.000    4.887    0.000 numeric.py:2416(geterr)
1495676    3.050    0.000    3.730    0.000 arrayprint.py:632(_digits)
1121757    2.206    0.000    2.206    0.000 {method 'compress' of 'numpy.ndarray' objects}
1121757    0.769    0.000    2.014    0.000 numeric.py:462(asanyarray)
3739190    1.949    0.000    1.949    0.000 {numpy.core.umath.seterrobj}
1495676    1.684    0.000    1.684    0.000 {numpy.core.multiarray.array}
1869595    1.274    0.000    1.663    0.000 numeric.py:2701(__init__)
21110    0.559    0.000    1.577    0.000 geometries.py:74(__init__)
7478380    1.411    0.000    1.411    0.000 {numpy.core.umath.geterrobj}
373919    0.326    0.000    1.336    0.000 fromnumeric.py:1281(ravel)

Calling str(pt) for pt in pts is not right.

Permafacture commented 9 years ago

now, 21,000 OGRGeometry polygons builds in 55 seconds and returns new OGRGeometry objects in 2.5 seconds.

Permafacture commented 9 years ago

and 21,000 arrays of points in 400 ms.

Permafacture / Py-Visvalingam-Whyatt

GDAL simplification is slow #3