search

network-wrangler / projectcard

Project card standard for transportation network project properties
https://network-wrangler.github.io/projectcard/main/
Apache License 2.0
0 stars 0 forks source link

🚀 FEAT: Add true geometry for new links #15

Open i-am-sijia opened 3 months ago

i-am-sijia commented 3 months ago

As a user, when adding new roads, l would like to give them true geometries, instead of having them as stick links.

The current user workflow to add new roads

Met Council:

  1. Open the CUBE .net file, add new stick links and nodes in CUBE, save out the .log file
  2. Run Lasso to create Add New Roadway project cards from the .log file. A typical new roadway project card (e.g., an Met Council example in application) looks like: 
    project: example new road
tags: ''
dependencies: ''
changes:
- category: Add New Roadway
links:
- A: 111
B: 222
model_link_id: 333
name: fake new road
roadway: tertiary
drive_access: 1
walk_access: 1
bike_access: 1
rail_only: 0
bus_only: 0
lanes: 1
nodes:
- model_node_id: 111
drive_access: 1
walk_access: 1
bike_access: 1
rail_only: 0
X: -93.04409
Y: 44.948720
- model_node_id: 222
drive_access: 1
walk_access: 1
bike_access: 1
rail_only: 0
X: -93.05000
Y: 44.950000
  3. Sometimes users choose to skip the coding in CUBE, instead they create the new roadway project card manually following the above format.
  4. Run Network Wrangler to apply the new roadway project card. Network Wrangler creates the new links and nodes with attributes specified in the project card, and stick link geometries between nodes.

Ideas for solution

Option Pros Cons
Option A: project card Uses the existing Add New Roadway project card schema More manual because it requires the user to edit the Add New Roadway project card. Also requires additional Lasso and Network Wrangler code changes.
Option B: wrangler card with input true shapes for the new links More flexible, good for batch editing. Does not require additional Lasso or Network Wrangler code. User can input geometry using shapefile, geojson, etc. Not a schema, harder to standardize
Option C: wrangler card with two input shapefiles Same as Option B, except that user will input two network shapefiles, one from the CUBE export, one from the CUBE export after editing true shapes in GIS. Both shapefiles are of the same network with links more than just the true shape edited ones, to ensure connectivity Not a schema, harder to standardize

Option A: Project Card

In the Add New Roadway project card, add geometry as a property under links, as a list of tuples (x,y coordinates).

project: example new road
tags: ''
dependencies: ''
changes:
- category: Add New Roadway
  links:
  - A: 111
    B: 222
    model_link_id: 333
    name: fake new road
    roadway: tertiary
    drive_access: 1
    walk_access: 1
    bike_access: 1
    rail_only: 0
    bus_only: 0
    lanes: 1
    geometry:
      - (-93.04409, 44.948720)
      - (-93.04609, 44.948820)
      - (-93.04809, 44.949020)
      - (-93.05000, 44.950000)
  nodes:
  - model_node_id: 111
    drive_access: 1
    walk_access: 1
    bike_access: 1
    rail_only: 0
    X: -93.04409
    Y: 44.948720
  - model_node_id: 222
    drive_access: 1
    walk_access: 1
    bike_access: 1
    rail_only: 0
    X: -93.05000
    Y: 44.950000

Option B: Wrangler Card

Use a roadway change wrangler card to overwrite stick geometries

---
project: example
tags:
dependencies:
category: Calculated Roadway
---

import geopandas as gpd
from NetworkWrangler import RoadwayNetwork
from NetworkWrangler.utils import create_unique_shape_id

# read input geometry file supplied by user, can be shapefiles, geojsons, the geometry to replace the base network with
new_geometry_gdf = gpd.read_file('xxxxx.geojson')

# convert the new_geometry to the same CRS as the base network
new_geometry_gdf = new_geometry_gdf.to_crs(self.shapes_df.crs)

# create unique shape hash for new geometry
new_geometry_gdf[RoadwayNetwork.UNIQUE_SHAPE_KEY] = new_geometry_gdf[
    "geometry"
].apply(lambda x: create_unique_shape_id(x))

# drop new geometry if 'model_link_id' is not in the base network
new_geometry_gdf = new_geometry_gdf[new_geometry_gdf['model_link_id'].isin(self.links_df['model_link_id'])]

# overwrite the base network with the new geometry, if it exists
self.links_df = self.links_df.set_index('model_link_id')
new_geometry_gdf = new_geometry_gdf.set_index('model_link_id')
# replace the geometry, and unique shape hash
self.links_df['geometry'] = new_geometry_gdf['geometry']
self.links_df[RoadwayNetwork.UNIQUE_SHAPE_KEY] = new_geometry_gdf[RoadwayNetwork.UNIQUE_SHAPE_KEY]

# reset the index
self.links_df = self.links_df.reset_index()
new_geometry_gdf = new_geometry_gdf.reset_index()

# add the new shape records to shapes
self.shapes_df = self.shapes_df.append(
    new_geometry_gdf[
        [RoadwayNetwork.UNIQUE_SHAPE_KEY, 'geometry']
    ]
)

Other considerations

i-am-sijia commented 3 months ago

Initial thoughts on true geometry. We can discuss at tomorrow's meeting. @DavidOry @e-lo @RachelWikenMC @yueshuaing

RachelWikenMC commented 3 months ago

option c - would it be possible to make that work without requiring the full network for both shapefiles? The full network with all the walk links is VERY large and is difficult to work with. I always clip it down to a county or just drive links, etc, before doing any editing. Requiring the edits to be on entire network in a shapefile format will be very slow / impossible for editing

RachelWikenMC commented 3 months ago

Our conversation about this at the last meeting was choppy because of my tech issues. I'm fine with the wrangler file method, even though it requires a few steps of editing, I don't think we will use this feature for every edit. But it will be key for large complicated interchange projects. In option B would I be running one wrangler file for each link id that needs new geometry? Or could I feed in a .json file with a number of new links?
Could the new geometry be in a different format - like .shp or does it have to be .json? I'm not using .json for my work currently so would have to figure out that conversion.

i-am-sijia commented 3 months ago

option c - would it be possible to make that work without requiring the full network for both shapefiles? The full network with all the walk links is VERY large and is difficult to work with. I always clip it down to a county or just drive links, etc, before doing any editing. Requiring the edits to be on entire network in a shapefile format will be very slow / impossible for editing

Sure. It can work with subset of network, as long as the two input shapefiles have the same links. The only thing it does is to compare the geometries from the two input shapefiles to find the true shapes user edited. a subset network is only better since it's faster to read in and compare.

i-am-sijia commented 3 months ago

In option B would I be running one wrangler file for each link id that needs new geometry? Or could I feed in a .json file with a number of new links?

You can feed in a number of links in one file. I think preferably a .shp, .geojson, instead of .json.

Option C is better than Option B because Option B's .shp only has the new shapes which is more prone to have them disconnected from the rest of the network. In Option C user first exports the network around the true shape into .shp, and then makes edits on the exported .shp for the true shapes, which helps maintain the connectivity.

i-am-sijia commented 2 months ago

Option C template:

---
project: example true geometry
tags:
dependencies:
category: Calculated Roadway
---

# this is an example wrangler card to implement Option C dicussed in this issue:
# https://github.com/network-wrangler/projectcard/issues/15
# users will supply two shapefiles, both includes model_link_id and geometry
# the only difference they are trying to capture is the geometry for some links

import geopandas as gpd
from network_wrangler import RoadwayNetwork
from network_wrangler.utils import create_unique_shape_id

########
# INPUTS - USER TO UPDATE
########

# read input geometry file supplied by user, can be shapefiles, geojsons

## UPDATE HERE: shapefile 1
## this is the link .shp before user changes any geometry
links_gdf = gpd.read_file("BaseLinks_head2.shp")

## UPDATE HERE: shapefile 2
## this is the link .shp after user changes any geometry
links_geometry_gdf = gpd.read_file("BaseLinks_head2_geometry_changes.shp")

## UPDATE HERE: in case the link id in the input .shp is not called model_link_id
UNIQUE_INPUT_LINK_ID_KEY = "link_id"

## set index
links_gdf = links_gdf.set_index(UNIQUE_INPUT_LINK_ID_KEY).sort_index()
links_geometry_gdf = links_geometry_gdf.set_index(UNIQUE_INPUT_LINK_ID_KEY).sort_index()

## checking consistency
## check the two input shapefiles are of the same length
assert len(links_gdf)==len(links_geometry_gdf), "the two input files have different length"
## check the two input shapefiles have the same index, ignore sequence
assert links_gdf.index.equals(links_geometry_gdf.index), "the two input files have different links"

#########
# PROCESS - NO NEED TO UPDATE
#########

# convert the two input files to the same CRS as the base network
links_gdf = links_gdf.to_crs("epsg:4269")
links_geometry_gdf = links_geometry_gdf.to_crs("epsg:4269")

# find links with different geometry
new_geometry_gdf = links_geometry_gdf[(~links_geometry_gdf.geom_equals(links_gdf))&(links_geometry_gdf.geometry.notnull())]

# create unique shape hash for new geometry
new_geometry_gdf[RoadwayNetwork.UNIQUE_SHAPE_KEY] = new_geometry_gdf[
    "geometry"
].apply(lambda x: create_unique_shape_id(x))

# reset the index and rename it as "model_link_id"
new_geometry_gdf = new_geometry_gdf.reset_index().rename(columns={UNIQUE_INPUT_LINK_ID_KEY: "model_link_id"})

# overwrite the base network with the new geometry, if it exists
self.links_df = self.links_df.set_index('model_link_id')
new_geometry_gdf = new_geometry_gdf.set_index('model_link_id')
# replace the geometry, and unique shape hash
self.links_df['geometry'].update(new_geometry_gdf['geometry'])
self.links_df[RoadwayNetwork.UNIQUE_SHAPE_KEY].update(new_geometry_gdf[RoadwayNetwork.UNIQUE_SHAPE_KEY])

# reset and drop the index
self.links_df = self.links_df.reset_index()
assert "model_link_id" in self.links_df.columns
new_geometry_gdf = new_geometry_gdf.reset_index()

# add the new shape records to shapes
self.shapes_df = self.shapes_df.append(
    new_geometry_gdf[
        [RoadwayNetwork.UNIQUE_SHAPE_KEY, 'geometry']
    ]
)
# drop duplicate unique shape id, keep last 
self.shapes_df = self.shapes_df.drop_duplicates(subset=[RoadwayNetwork.UNIQUE_SHAPE_KEY], keep='last')