search

Riverscapes / riverscapes-tools

Open-source Python 3.0 tools for the Riverscapes organization
https://tools.riverscapes.net/
GNU General Public License v3.0
11 stars 11 forks source link

VBET: Level path fail on HUC 10190002 #635

Closed MattReimer closed 1 year ago

MattReimer commented 1 year ago

Looks like we've got a new error with level path calculations:

this is from level path 23001900010467 on HUC 10190002 AND 23001900030195 on HUC10190003

Here's an excerpt from the log file

2022-11-15 07:01:15:: [info] [VBET] Processing Level Path: 23001900010467 164/28161
2022-11-15 07:01:15:: [info] [MakeDir] Folder not found. Creating: /usr/local/data/output/temp/levelpath_23001900010467
2022-11-15 07:01:15:: [info] [VectorBase] Dataset not found. Creating: /usr/local/data/output/temp/levelpath_23001900010467/channel_polygons.gpkg
2022-11-15 07:01:15:: [debug] [VectorBase] Dataset created: /usr/local/data/output/temp/levelpath_23001900010467/channel_polygons.gpkg
2022-11-15 07:01:15:: [debug] [VectorBase] No layer named "level_path_23001900010467" found
2022-11-15 07:01:15::         PROGRESS: 1 / 73  (1%)  Copying features     (Ellapsed: 0.0 seconds)
2022-11-15 07:01:55:: [warning] [GDALError] ERR# 1 -- A geometry of type MULTIPOLYGON is inserted into layer level_path_23001900010467 of geometry type POLYGON, which is not normally allowed by the GeoPackage specification, but the driver will however do it. To create a conformant GeoPackage, if using ogr2ogr, the -nlt option can be used to override the layer geometry type. This warning will no longer be emitted for this combination of layer and feature geometry type.
2022-11-15 07:01:55:: [info] [Copying features] Completed 73 operations.  Total Time: 0.0 seconds     
2022-11-15 07:01:55:: [info] [VectorBase] 400m distance converts to 0.0045326932 using UTM EPSG 26913
2022-11-15 07:01:55:: [debug] [VBET] channel_envelope_geom area: <bound method Geometry.Area of <osgeo.ogr.Geometry; proxy of <Swig Object of type 'OGRGeometryShadow *' at 0x7fc60a297ea0> >>
2022-11-15 07:01:55:: [info] [VectorBase] Dataset not found. Creating: /usr/local/data/output/temp/levelpath_23001900010467/envelope_polygon.gpkg
2022-11-15 07:01:55:: [debug] [VectorBase] Dataset created: /usr/local/data/output/temp/levelpath_23001900010467/envelope_polygon.gpkg
2022-11-15 07:01:55:: [debug] [VectorBase] No layer named "level_path_23001900010467" found
2022-11-15 07:01:55:: [info] [Raster Warp] Raster Warp input raster /usr/local/data/output/intermediates/dinfflowdir_ang.tif
2022-11-15 07:01:55:: [info] [Raster Warp] Raster Warp output raster /usr/local/data/output/temp/levelpath_23001900010467/dinfflowdir_ang_23001900010467.tif
2022-11-15 07:01:55:: [info] [Raster Warp] Output spatial reference EPSG: 4326
2022-11-15 07:01:55:: [info] [Raster Warp] Performing GDAL warp to temporary VRT file.
2022-11-15 07:01:55:: [info] [Raster Warp] Clipping to polygons using /usr/local/data/output/temp/levelpath_23001900010467/envelope_polygon.gpkg/level_path_23001900010467
2022-11-15 07:01:55:: [info] [Raster Warp] Using GDAL translate to convert VRT to compressed raster format.
2022-11-15 07:01:55:: [info] [Raster Warp] Process completed successfully.
2022-11-15 07:01:55:: [info] [Raster Warp] Raster Warp input raster /usr/local/data/output/intermediates/pitfill.tif
2022-11-15 07:01:55:: [info] [Raster Warp] Raster Warp output raster /usr/local/data/output/temp/levelpath_23001900010467/pitfill_dem_23001900010467.tif
2022-11-15 07:01:55:: [info] [Raster Warp] Output spatial reference EPSG: 4326
2022-11-15 07:01:55:: [info] [Raster Warp] Performing GDAL warp to temporary VRT file.
2022-11-15 07:01:55:: [info] [Raster Warp] Clipping to polygons using /usr/local/data/output/temp/levelpath_23001900010467/envelope_polygon.gpkg/level_path_23001900010467
2022-11-15 07:01:55:: [info] [Raster Warp] Using GDAL translate to convert VRT to compressed raster format.
2022-11-15 07:01:55:: [info] [Raster Warp] Process completed successfully.
2022-11-15 07:01:55::         PROGRESS: 0 / 100  (0%)  Rasterizing      (Ellapsed: 0.0 seconds)
2022-11-15 07:01:55:: [debug] [VBETRasterize] Temporary file: /tmp/tmp0m676e4k.tiff
2022-11-15 07:01:55:: [info] [Rasterizing ] Completed 100 operations.  Total Time: 0.0 seconds     
2022-11-15 07:01:55::         PROGRESS: 0 / 2,096  (0%)  Applying nodata mask     (Ellapsed: 0.0 seconds)
2022-11-15 07:02:08:: [info] [Applying nodata mask] Completed 2,096 operations.  Total Time: 1.0 seconds     
2022-11-15 07:02:08:: [info] [mask_rasters_nodata] Complete
2022-11-15 07:02:08:: [info] [Subprocess] Running command: mpiexec -n 2 dinfdistdown -ang /usr/local/data/output/temp/levelpath_23001900010467/dinfflowdir_ang_23001900010467.tif -fel /usr/local/data/output/temp/levelpath_23001900010467/pitfill_dem_23001900010467.tif -src /usr/local/data/output/temp/levelpath_23001900010467/rasterized_channel_23001900010467.tif -dd /usr/local/data/output/temp/rasters/local_hand_23001900010467.tif -m ave v
2022-11-15 07:02:08:: [info] [Subprocess] DinfDistDown -v version 5.3.8
2022-11-15 07:02:08:: [info] [Subprocess] Input file /usr/local/data/output/temp/levelpath_23001900010467/dinfflowdir_ang_23001900010467.tif has geographic coordinate system.
2022-11-15 07:02:08:: [info] [Subprocess] Input file /usr/local/data/output/temp/levelpath_23001900010467/pitfill_dem_23001900010467.tif has geographic coordinate system.
2022-11-15 07:02:08:: [info] [Subprocess] Input file /usr/local/data/output/temp/levelpath_23001900010467/rasterized_channel_23001900010467.tif has geographic coordinate system.
2022-11-15 07:02:08:: [info] [Subprocess] Processors: 2
2022-11-15 07:02:08:: [info] [Subprocess] Read time: 0.522571
2022-11-15 07:02:08:: [info] [Subprocess] Compute time: 10.482924
2022-11-15 07:02:08:: [info] [Subprocess] Write time: 0.448200
2022-11-15 07:02:08:: [info] [Subprocess] Total time: 11.453696
2022-11-15 07:02:08:: [error] [Subprocess] This run may take on the order of 1 minutes to complete.
2022-11-15 07:02:08:: [error] [Subprocess] This estimate is very approximate. 
2022-11-15 07:02:08:: [error] [Subprocess] Run time is highly uncertain as it depends on the complexity of the input data 
2022-11-15 07:02:08:: [error] [Subprocess] and speed and memory of the computer. This estimate is based on our testing on 
2022-11-15 07:02:08:: [error] [Subprocess] a dual quad core Dell Xeon E5405 2.0GHz PC with 16GB RAM.
2022-11-15 07:02:08:: [info] [Subprocess] Command completed in 11.0 seconds
2022-11-15 07:02:08::         PROGRESS: 0 / 9,404  (0%)  Calculating evidence layer     (Ellapsed: 0.0 seconds)
2022-11-15 07:02:18::         PROGRESS: 0 / 2,096  (0%)  Applying nodata mask     (Ellapsed: 0.0 seconds)
2022-11-15 07:02:20:: [info] [Applying nodata mask] Completed 2,096 operations.  Total Time: 1.0 seconds     
2022-11-15 07:02:20:: [info] [mask_rasters_nodata] Complete
2022-11-15 07:02:20:: [info] [threshold] Thresholding at 0.68
2022-11-15 07:02:20::         PROGRESS: 0 / 2,096  (0%)  Thresholding at 0.68     (Ellapsed: 0.0 seconds)
2022-11-15 07:02:22:: 100 - done.
2022-11-15 07:02:26:: 0...10...20...30...40...50...60...70...80...90...[info] [Thresholding at 0.68] Completed 2,096 operations.  Total Time: 1.0 seconds     
2022-11-15 07:02:26:: [debug] [threshold] Timer: 1.0 seconds
2022-11-15 07:02:26:: [info] [VBET Generate Polygon] Sieve Filter vbet
2022-11-15 07:02:26:: [info] [VBET Generate Polygon] Generate regions
2022-11-15 07:02:26:: [info] [VBET Generate Polygon] Cleaning Raster edges
2022-11-15 07:02:26:: [debug] [VBET Generate Polygon] Timer: 5.0 seconds
2022-11-15 07:02:26:: [info] [VBET] Add VBET Raster to Output
2022-11-15 07:02:26::         PROGRESS: 0 / 2,096  (0%)  Applying nodata mask     (Ellapsed: 0.0 seconds)
2022-11-15 07:02:27:: [info] [Applying nodata mask] Completed 2,096 operations.  Total Time: 1.0 seconds     
2022-11-15 07:02:27:: [info] [mask_rasters_nodata] Complete
2022-11-15 07:02:27:: [info] [threshold] Thresholding at 0.9
2022-11-15 07:02:27::         PROGRESS: 0 / 2,096  (0%)  Thresholding at 0.9     (Ellapsed: 0.0 seconds)
2022-11-15 07:02:29:: 100 - done.
2022-11-15 07:02:36:: 0...10...20...30...40...50...60...70...80...90...[info] [Thresholding at 0.9] Completed 2,096 operations.  Total Time: 1.0 seconds     
2022-11-15 07:02:36:: [debug] [threshold] Timer: 1.0 seconds
2022-11-15 07:02:36:: [info] [VBET Generate Polygon] Sieve Filter vbet
2022-11-15 07:02:36:: [info] [VBET Generate Polygon] Generate regions
2022-11-15 07:02:36:: [info] [VBET Generate Polygon] Cleaning Raster edges
2022-11-15 07:02:36:: [debug] [VBET Generate Polygon] Timer: 4.0 seconds
2022-11-15 07:02:36:: [info] [VBET] Generating Centerline from cost path
2022-11-15 07:02:36:: [debug] [Generate Centerline Surface] Timer: 1.0 seconds
2022-11-15 07:02:36:: [info] [VBET] Find least cost path for centerline
2022-11-15 07:02:36:: [error] [VBET] end points must all be within the costs array
2022-11-15 07:02:36:: Traceback (most recent call last):
2022-11-15 07:02:36::   File "/usr/local/src/riverscapes-tools/packages/vbet/vbet/vbet.py", line 838, in main
2022-11-15 07:02:36::     vbet_centerlines(args.flowline_network, args.dem, args.slope, args.hillshade, args.catchments, args.channel_area, args.vaa_table, args.output_dir, args.scenario_code, args.huc, level_paths, args.pitfill, args.dinfflowdir_ang, args.dinfflowdir_slp, args.twi_raster, meta=meta, reach_codes=reach_codes, mask=args.mask, debug=args.debug)
2022-11-15 07:02:36::   File "/usr/local/src/riverscapes-tools/packages/vbet/vbet/vbet.py", line 512, in vbet_centerlines
2022-11-15 07:02:36::     least_cost_path(cost_path_raster, centerline_raster, coords[0], coords[1])
2022-11-15 07:02:36::   File "/usr/local/src/riverscapes-tools/packages/vbet/scripts/cost_path.py", line 63, in least_cost_path
2022-11-15 07:02:36::     pathArray = createPath(CostSurfacefn, costSurfaceArray, startCoord, stopCoord)  # creates path array
2022-11-15 07:02:36::   File "/usr/local/src/riverscapes-tools/packages/vbet/scripts/cost_path.py", line 52, in createPath
2022-11-15 07:02:36::     indices, _weight = route_through_array(costSurfaceArray, (startIndexY, startIndexX), (stopIndexY, stopIndexX), geometric=True, fully_connected=True)
2022-11-15 07:02:36::   File "/usr/local/venv/lib/python3.8/site-packages/skimage/graph/mcp.py", line 88, in route_through_array
2022-11-15 07:02:36::     costs, traceback_array = m.find_costs([start], [end])
2022-11-15 07:02:36::   File "skimage/graph/_mcp.pyx", line 483, in skimage.graph._mcp.MCP.find_costs
2022-11-15 07:02:36:: ValueError: end points must all be within the costs array
2022-11-15 07:02:36:: 100 - done.
2022-11-15 07:02:37:: <<RS CONTEXT PROCESS ENDED WITH AN ERROR>>
MattReimer commented 1 year ago

Both hucs completed from last night's run. I think this is done.