remove ArcGIS requirement

[smnorris]

Process the road data using open tools.

The ArcGIS integrate tool is very handy and works well for this purpose. However, it requires an ArcGIS licence (and thus must be run on Windows). Investigate possible alternatives.

[smnorris]

Note that the script uses the Erase tool - it also requires an Advanced ArcGIS license.

To replicate the existing analysis, all that is necessary is snapping lower priority lines to reference lines and then removing the shared paths.

This can likely be done in PostGIS by:

• increasing line vertex density in all sources (ST_Segmentize)
• iterate through all lower priority road sources,
  • ST_Snap lower priority road vertex to higher priority road vertexes, snapping only to nearest neighbour
  • remove snapped roads from lower priority source using ST_Difference
  • insert lower priority source into output/reference layer

To improve upon existing analysis, we could:

• before performing subsequent snapping/matching, snap candidate layer nodes (ends, intersections) to reference layer nodes within tolerance. If more than 1 reference node is within our matching tolerance distance for a given candidate layer node, match to closest already unmatched reference node
• break candidate roads at intersections with reference roads
• compare and match entire lines using methods such as Hausdorf Distance (https://en.wikipedia.org/wiki/Hausdorff_distance)

For other possibilities, see:

• https://github.com/ssinger/roadmatcher (and http://wiki.openstreetmap.org/wiki/RoadMatcher)
• https://gis.stackexchange.com/questions/60072/snapping-start-and-end-nodes-of-lines-to-other-lines-in-postgis
• http://blog.mathieu-leplatre.info/use-postgis-topologies-to-clean-up-road-networks.html
• https://medium.com/the-downlinq/spacenet-road-detection-and-routing-challenge-part-ii-apls-implementation-92acd86f4094
• https://medium.com/strava-engineering/activity-grouping-the-heart-of-a-social-network-for-athletes-865751f7dca

[smnorris]

ArcGIS integrate functionality more or less replicated with PostGIS as noted above. The results are by no means identical but are reasonably similar and could be further improved.

• increasing line vertex density (ST_Segmentize) Done for reference roads - this works for the reference roads but experimenting with increasing the vertex density in the roads to be inserted produced much lower quality results from ST_Snap

• iterate through all lower priority road sources, snap lower priority road vertex to higher priority roads vertexes, snapping only to nearest neighbour Done but unfortunately snapping has to be done to all nearby features, the road networks are not clean enough to snap to just the nearest neighbour - often a feature in a lower priority source will span several features in the reference roads

• insert difference between lower priority roads and reference roads into output Done

Additionally, lower priority roads are now cleaned up slightly: dangling nodes are snapped to same source features within 7m and the resulting features are re-noded before snapping to higher priority roads is done.

[smnorris]

I'm happy enough with PostGIS output to close this for now and move improvement ideas to a different issue.