spatial-data-lab / knime-geospatial-extension

This repository is built for KNIME-CGA Geospatial Project, and the goal is to build Python-based nodes for geospatial analysis in KNIME Analytic Platform.
MIT License
25 stars 10 forks source link

Add option to GeoPackage Reader to ignore invalid rows #260

Open koettert opened 1 year ago

koettert commented 1 year ago

This problem was reported in the KNIME forum while reading this file. The node should offer an option to enable skipping of invalid lines which is what QGIS is doing. This is the full stacktrace:

  File "C:\KNIME\KNIME_AP\KNIME\plugins\org.knime.python3.nodes_4.7.3.v202305081333\src\main\python\_node_backend_launcher.py", line 457, in execute
    outputs = self._node.execute(exec_context, *inputs)
  File "C:\KNIME\KNIME_AP\KNIME\plugins\sdl.harvard.geospatial_1.1.1.202306211527\src\main\python\src\nodes\io.py", line 250, in execute
    gdf = gp.GeoDataFrame.from_features(src)
  File "C:\KNIME\KNIME_AP\KNIME\plugins\sdl.harvard.geospatial.channel.bin.win32.x86_64_1.1.1.202306211527\env\lib\site-packages\geopandas\geodataframe.py", line 640, in from_features
    "geometry": shape(feature["geometry"]) if feature["geometry"] else None
  File "C:\KNIME\KNIME_AP\KNIME\plugins\sdl.harvard.geospatial.channel.bin.win32.x86_64_1.1.1.202306211527\env\lib\site-packages\shapely\geometry\geo.py", line 106, in shape
    return MultiLineString(ob["coordinates"])
  File "C:\KNIME\KNIME_AP\KNIME\plugins\sdl.harvard.geospatial.channel.bin.win32.x86_64_1.1.1.202306211527\env\lib\site-packages\shapely\geometry\multilinestring.py", line 50, in __new__
    line = linestring.LineString(lines[i])
  File "C:\KNIME\KNIME_AP\KNIME\plugins\sdl.harvard.geospatial.channel.bin.win32.x86_64_1.1.1.202306211527\env\lib\site-packages\shapely\geometry\linestring.py", line 73, in __new__
    geom = shapely.linestrings(coordinates)
  File "C:\KNIME\KNIME_AP\KNIME\plugins\sdl.harvard.geospatial.channel.bin.win32.x86_64_1.1.1.202306211527\env\lib\site-packages\shapely\decorators.py", line 77, in wrapped
    return func(*args, **kwargs)
  File "C:\KNIME\KNIME_AP\KNIME\plugins\sdl.harvard.geospatial.channel.bin.win32.x86_64_1.1.1.202306211527\env\lib\site-packages\shapely\creation.py", line 120, in linestrings
    return lib.linestrings(coords, out=out, **kwargs)
shapely.errors.GEOSException: IllegalArgumentException: point array must contain 0 or >1 elements

2023-07-11 18:40:19,463 : DEBUG : KNIME-Worker-7-GeoPackage Reader 3:1552 :  : Node : GeoPackage Reader : 3:1552 : reset
2023-07-11 18:40:19,463 : ERROR : KNIME-Worker-7-GeoPackage Reader 3:1552 :  : Node : GeoPackage Reader : 3:1552 : Execute failed: IllegalArgumentException: point array must contain 0 or >1 elements

org.knime.python3.nodes.PythonNodeRuntimeException: IllegalArgumentException: point array must contain 0 or >1 elements

    at org.knime.python3.nodes.CloseablePythonNodeProxy$FailureState.throwIfFailure(CloseablePythonNodeProxy.java:475)
    at org.knime.python3.nodes.CloseablePythonNodeProxy.execute(CloseablePythonNodeProxy.java:315)
    at org.knime.python3.nodes.DelegatingNodeModel.lambda$4(DelegatingNodeModel.java:145)
    at org.knime.python3.nodes.DelegatingNodeModel.runWithProxy(DelegatingNodeModel.java:204)
    at org.knime.python3.nodes.DelegatingNodeModel.execute(DelegatingNodeModel.java:143)
    at org.knime.core.node.NodeModel.executeModel(NodeModel.java:549)
    at org.knime.core.node.Node.invokeFullyNodeModelExecute(Node.java:1267)
    at org.knime.core.node.Node.execute(Node.java:1041)
    at org.knime.core.node.workflow.NativeNodeContainer.performExecuteNode(NativeNodeContainer.java:595)
    at org.knime.core.node.exec.LocalNodeExecutionJob.mainExecute(LocalNodeExecutionJob.java:98)
    at org.knime.core.node.workflow.NodeExecutionJob.internalRun(NodeExecutionJob.java:201)
    at org.knime.core.node.workflow.NodeExecutionJob.run(NodeExecutionJob.java:117)
    at org.knime.core.util.ThreadUtils$RunnableWithContextImpl.runWithContext(ThreadUtils.java:367)
    at org.knime.core.util.ThreadUtils$RunnableWithContext.run(ThreadUtils.java:221)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
    at org.knime.core.util.ThreadPool$MyFuture.run(ThreadPool.java:123)
    at org.knime.core.util.ThreadPool$Worker.run(ThreadPool.java:246)
koettert commented 1 year ago

Relates to https://github.com/geopandas/geopandas/issues/2791 and might already work with the latest version of Geopandas: https://github.com/geopandas/geopandas/issues/2791#issuecomment-1638806042

koettert commented 11 months ago

Still not fixed in Geopandas so we have to postpone this issue to next milestone and see if it is fixed by then.