opennewzealand / linz2osm

Some tools for helping move LINZ data into OpenStreetMap
http://wiki.openstreetmap.org/wiki/LINZ
GNU General Public License v3.0
22 stars 2 forks source link

highlight features already in OSM that conflict with workslice features #91

Closed HamishB closed 12 years ago

HamishB commented 12 years ago

Hi,

now that the basic checkout system is up and running pretty well, perhaps it is time to delve into the trickier task of creating tools to help with merging issues.

The manual script I had been using for this in the 2010-2011 upload effort: http://wiki.openstreetmap.org/wiki/Script_for_finding_nearby_nodes required you to use Xapi to download existing features matching your tagging and bounding box, preparing your new workslice, then running the comparator tool. The comparator tool exported feature URLs which could both be put into the browser and, quite usefully, directly into the JOSM download data dialog. With this you could quickly identify problems and prep the merging canvas.

A lot of that could be automated in the web app, maybe in an openlayers mash-up too, the trickiest part is the sloppy tag matching.

As seen on the above wiki page & talk page, to keep the number of targets manageable I was clumping together clustered hits within a certain radius into a single URL output, using GRASS, QGIS, some grep'ing, and/or all three. Perhaps not the most efficient or robust approach, but the basic method proved to be quite effective for quickly identifying problem areas. I imagine with access to a local OSM db or doing the search + clumping stuff in the source data db could make it a bit cleaner.

I had been going through the osm wiki and looking for any nearby key and value variants, then exploring the TagWatch site for even more possibilities and typos. The result was a Xapi query with many a=|b=|=c|=d mixes, e.g. for water tanks ISTR there were about a dozen different manmade=, =reservoir, landuse= tags too look out for, as well as way-areas and single-node versions to test for. That's the hardest part to automate.

Another issue we've discussed is that the Xapi servers have historically had about a 6 month half-life before they fell over and someone else took over with another one. Unfortunately the current ones with the biggest hardware behind them apparently don't support complex queries, just dumb bbox ones. (?) So it would be good to keep the URL as an easy to change variable by the webmaster, and if it isn't working out perhaps look at hosting our own Australasian OSM mirror

https://github.com/iandees/xapi-servlet http://wiki.openstreetmap.org/wiki/OSM3S/install http://wiki.openstreetmap.org/wiki/OSM3S#XAPI_Compatibility_Layer

# Overpass server with Xapi compatibility layer
XAPI_URL="http://www.overpass-api.de/api/xapi"
# needed for Overpass server:
XAPI_EXTRA="[@meta]"
CITY="Nottingham"
BBOX="-1.25,52.888,-1.06,53.02"
# visualize:
#http://www.openstreetmap.org/?box=yes&bbox=$BBOX

wget --progress=dot:mega -O "$CITY.osm" \
     "${XAPI_URL}?*[bbox=$BBOX]$XAPI_EXTRA"

here's an example, grabbing some LINZ V16 data for the Chathams:

BBOX="-177,-44.5,-175.5,-43.5"
XAPI_URL="http://www.overpass-api.de/api/xapi"
wget -O chatham_hedges_and_fences.osm \
 "$XAPI_URL?way[bbox=$BBOX][LINZ:source_version=V16][barrier=hedge|fence]"

but it doesn't work :( for some reason it doesn't like the OR in [barrier=hedge|fence].

the sample on the help page works ok though:

wget -O test.osm \
  "$XAPI_URL?node[bbox=7.1,51.2,7.2,51.3][highway=bus_stop|traffic_signals]"

?? ( https://github.com/drolbr/Overpass-API/issues/3 )

thoughts?

Hamish B

HamishB commented 12 years ago

fyi, nightly osm planet file extracts for NZ available here: http://www.osmaustralia.org/osmnzextract.php

stephend commented 12 years ago

This requires the 'tags for overpass QL' field to be set in the admin interface for the relevant layer. I've done layers starting with A-F, except where there's no tagging at all, today: will do the rest on Monday.

stephend commented 12 years ago

OK, I've made queries for most of the layers. Feel free to play around with them and/or update them if you change the tags.