roadintegrator

Development and maintenance of this project has been moved to bcgov/CE_integratedroads. Please file any issues/PRs at the new location.

Quckly merge various BC road data sources into a single layer.

Sources

Method

First, sources are preprocessed:

• centerlines of polygon road sources are approximated
• FTEN roads are cleaned slightly, snapping endpoints within 7m to other same-source roads

Next, all roads are loaded to the output table in order of decreasing priority. Portions of lower priority roads within 7m of a higher priority road are deleted. Where the endpoint of a remaining lower priority road is within 7m of a higher prioirity road, the endpoint of the lower priority road is snapped to the closest point on the higher priority road.

Limitations and Caveats

The authoritative source for built roads in British Columbia is the Digital Road Atlas. The process used in these scripts IS NOT A COMPRENSIVE CONFLATION/MERGE of the input road layers, it is a quick approximation. The intent of the processing is to retain all input features not covered by a higher priority road - due to the nature of duplication in BC road data, the output will always be an over-representation of roads.

Several specific issues will lead to over-representation of roads:

• the same road present in different source layers will only be de-duplicated when features are less than 7m apart, see Duplications below)
• roads are present in the tenure layers that have not been built
• roads may have been decomissioned, overgrown or become otherwise impassible
• duplicate same-source roads are not accounted for

Additional notes:

• the various road data sources are not 100% comprehensive, there may be roads present in the landscape that are not included in the analysis and output product
• because processing is tiled by BCGS 20k tile, any portion of road falling outside of these tiles will not be included (tile edges do not exactly match the surveyed BC border)
• attributes from all sources associated with a given road feature are populated when available - attributes for the source feature will be correct but because additional overlapping roads will not always be matched correctly, use additional attribute values with caution (the matching is based only on greatest overlap)

Requirements

• bash/make/wget/zip/unzip/parallel
• PostgreSQL >= 14
• PostGIS >= 3.3
• GDAL >= 3.6
• Python >= 3.9
• bcdata >= 0.9.1

Tested with versions as noted in Dockerfile

Setup

Clone the repository, navigate to the project folder:

    git clone https://github.com/smnorris/roadintegrator.git
    cd roadintegrator

If you do not have above noted requirements installed on your system, installation via conda or Docker is recommended. An environment.yml is provided for users familiar with conda, instructions for Docker usage are below.

Docker

Install Docker for your OS:

• MacOS
• Windows

Build and start the containers:

    docker-compose build
    docker-compose up -d

As long as you do not remove the container roadintegrator-db, it will retain all the data you put in it. If you have shut down Docker or the container, start it up again with this command:

    docker-compose up -d

Usage

Scripts are run via make. To run the full job:

    make

If using Docker:

    docker-compose run --rm app make

Note that connecting to the dockerized database from your local OS is possible via the port specified in docker-compose.yml:

    psql postgresql://postgres:postgres@localhost:8001/roadintegrator

Duplications

As mentioned above, this analysis is very much a rough approximation. It works well in areas where roads are not duplicated between sources or where source road networks are near-coincident.

These diagrams illustrate a problematic sample area, showing three similar input road layers and the resulting output.

three input layers

resulting output

Output summary

Summary of length by data source

Alternative approaches

Road network conflation is a common task, many additional approaches and tools are available. This list provides a starting point for additional reading:

• RoadMatcher JUMP/OpenJump plugin source, wiki
• PostGIS topology
• Average Path Length Similarity
• Tiled similarity scoring
• Hootenanny - a conflation tool
• Graph based merging