search

opennewzealand / linz2osm

Some tools for helping move LINZ data into OpenStreetMap
http://wiki.openstreetmap.org/wiki/LINZ
GNU General Public License v3.0
22 stars 2 forks source link

missing data in scattered_scrub_poly layer #119

Open HamishB opened 10 years ago

HamishB commented 10 years ago

Hi,

not sure if this tracker is appropriate for this since it might be a data problem not a code one, but here goes anyway.

The scattered_scrub_poly layer is just about complete, only two features remain. Can't find where they are. There are a couple off the bottom of the trapezoid at the southern end of Stewart Island, but they can't be selected. Not sure if that's because they were already taken or if there's bug in the selector.

http://linz2osm.openstreetmap.org.nz/workslices/create/layer_in_dataset/213/

Also I noticed a chunk missing in the data in the central South Island. See workslices 2119, 2120, and 2122; screenshot attached. ![Uploading central_si_scat_scrub_hole.png . . .]() There's possibly another one north of Hanmer Springs, see workslice 2140.

thanks, Hamish

HamishB commented 10 years ago

try that screenshot again with another browser... central_si_scat_scrub_hole

stephend commented 10 years ago

The two missing features are 20327, a bit inland of Cheviot and 22577, slightly south-east of Masterton. It's going to be a bug in the code that fetches which features you get for a given workslice, possibly one that was already fixed in the code (but the two features are already "covered" by existing workslices).

Longer term, I want to rethink how we display and create checkouts when data has been added or missed within an existing workslice (this will become a feature, not a bug, once we start updating datasets).

In the meantime, there is a possible workaround to hackily check these features out: select a workslice box that covers the feature(s) and also covers some area that's not checked out. The un-checked-out area doesn't have to contain any features itself (could be open ocean) but needs to be within the extent of the layer. Once checkout out, it'll appear on the map as your slice of open ocean, but the downloaded file will include your feature(s).

HamishB commented 10 years ago

ok, got those last two in changesets 2155 and 2156 using the open ocean technique. scattered scrub is now fully uploaded.

The Cheviot feature would have been missed in a checkout from the last few days, so not a historical bug.

while we're on it, I'd note a weird checkout bug for exotic_poly, if you look deep in the Catlins at, and to the east of, Mokoreta there looks like two blocks which are not checked out. But they were checked out and completed. I think the problem there is related to the off-the-bottom since the workslice went down to Slope Point. Look at the bounding box representation for workslice 2061, it's just a dot in the ocean:

http://linz2osm.openstreetmap.org.nz/workslices/2061/show/

thanks, Hamish

stephend commented 10 years ago

I'll have to have a look at that. But I'm probably just going to end up allowing overlapping checkouts, however they end up getting displayed.

HamishB commented 10 years ago

fwiw besides for these few minor corner cases I'd say the existing system works remarkably well. (2 lost features out of 30000 isn't so bad...)

stephend commented 10 years ago

Oh, and I think the missing chunks must be missing in the source data. The one you picture exactly corresponds to a map sheet (BW30). There's several other places where data is either completely missing or becomes suddenly less dense at map-sheet boundaries.

stephend commented 10 years ago

fwiw besides for these few minor corner cases I'd say the existing system works remarkably well. (2 lost features out of 30000 isn't so bad...)

Thanks, I wish I had some more time to work on it.

HamishB commented 10 years ago

(2 lost features out of 30000 isn't so bad...)

... at least for linz2osm purposes. same for areas of missing source data.

can't find a link right now but wrt things changing between map sheets I'm reminded of a quite famous story dating from ~1960s, maybe even visible in (pre-satellite altimetry) maps today if you look closely, for the first US govt big ocean map one operator was responsible for contouring down to ~60S, then another took over for the Antarctic region. And these two people had different ideas about how much detail to include... and the ocean magically became rougher across the line for the next 40+ years.... :) I digress but probably a lot of that sort of thing about.....