Add more kind:* POV from Natural Earth for low zooms and places layer

[nvkelso]

Originally posted by @nvkelso in https://github.com/tilezen/vector-datasource/issues/1552#issuecomment-463516835

There's also work to add this in from the Natural Earth 1:50M and 1:110M scale sets (we've only added from 1:10M set in v1.7)... and from populated_places and states_provinces as some country and region capitals aren't recognized, and some regions aren't recognized.

This issue tracks integrating that data.

[nvkelso]

Sister issue to OSM data in #1810.

[nvkelso]

Nearterm, I'll prototype a sample data change in the 1:10m populated_placeas and region boundary lines, and same samples in the 1:110m and 1:50m scale sets to unblock Tilezen work. Then we'll do one other asset build once the final NE data is ready.

[nvkelso]

Prototype data:

• ne_v5.0.0-pre3-boundaries-pov.zip

Notes:

• All themes included here have the additional POV columns with the same naming schema.
• Some themes, like 1:50m and 1:110m state and province boundary lines include the schema but none of the line features are disputed so those columns are entirely blank. But there are requests to add that data for India and China so let's plumb it in here to future proof.
• The 1:10m state and province boundary lines does include sample data for India, China.
• The populated places in Taiwan include the placetype swapping (eg country capital goes to region capital, and region capitals go to generic localities)
• The 1:50m and 1:110m international boundaries files have been updated with sample data.
• More work remains to flesh out the column values completely, but these are good sample to unblock development.

Flagging @zerebubuth for integration work.

[nvkelso]

Noting we should rename kind unrecognized to unrecognized_country so we can also support unrecognized_region.

Under our SemVer we can rename a recently added kind as a Patch release, but let's just bundle it in this minor update.

[nvkelso]

I've done more data work and need to post a new preview ZIP.

[nvkelso]

Here's v5.0.0-pre-4 with more detail filled out (not just samples) for the other themes, and some fixes in the 1:10m themes:

• ne_v5.0.0-pre4-boundaries-pov.zip

Still needs more QA, but is close.

[nvkelso]

We're missing data / modeling around Cyprus, followup here: https://github.com/tilezen/vector-datasource/issues/1898.

Cyprus low zoom from NE turns out to be styling problem:

[nvkelso]

For Crimea we should also support FCLASS_UA (since Ukraine is primary claimant with Russia there). I'll update Natural Earth and provide new file here.

But kind:gb here should say unrecognized_country instead of just unrecognized.

[nvkelso]

I'm not sure why most of the India - Pakistan boundary is coming thru as kind:cn of line_of_control as the data I see in my source copy of NE says it should just be basic country boundary – that POV shouldn't be exported in tiles for that boundary as no one disputes it:

[nvkelso]

UPDATE: This is a data problem with the NE 50m set. New data forthcoming.

Similarly this disputed boundary is considered as country by Russia, China, and Pakistan, and is unrecognized_country by India in source Natural Earth data but those are all missing in this tile build - the red line between Kashmir and China:

[nvkelso]

Missing gaps between China and India, Bhutan at low zooms (fine and mid-zoom from NE 1:10m see 4/31.58/82.53). Data seems okay in my original of NE v5, wondering if the asset file didn't use latest ZIP?

[nvkelso]

I also need to plumb thru unrecognized_* versus the simpler unrecognized in NE and Tilezen logic needs to update.

[zerebubuth]

This is what I see in the ne_50m_admin_0_boundary_lines_land.shp shapefile from the ne_v5.0.0-pre4-boundaries-pov.zip NE pre-release:

Possibly it's just a typo? I don't see the same FCLASS_CN in the 1:10m version of the file.

[nvkelso]

Looks like POV for NE admin-1 region lines isn't fully wired up?

Here we see India's POV, but the Xinxang region line is still showing up inside of India's claim area:

[nvkelso]

Same with China / India border in the east:

This isn't a problem in Taiwan as those lines only show up zoom 8+ and would require further wiring up in https://github.com/tilezen/vector-datasource/issues/1900.

[nvkelso]

This needs a new NE data asset bundle and fixes for POV on region lines.

We should also investigage: https://github.com/tilezen/vector-datasource/issues/1840#issuecomment-497161486.

UPDATE: I suspect we're not seeing Taiwan not because the ETL needs work but because Taiwan region boundaries min_zoom is 8+ which never shows in mid-zoom from Natural Earth. But the ones mentioned above have been fixed in new data ZIP.

[nvkelso]

Here's Natural Earth v5.0.0-pre5:

• ne_v5.0.0-pre5-boundaries-pov.zip

[nvkelso]

@zerebubuth Can you look at https://github.com/tilezen/vector-datasource/issues/1840#issuecomment-497545789 again? I still don't see the POV wired up on NE sourced region lines.

[nvkelso]

There's also this oddness around zoom 2 where all the Kashmir lines go missing for some POV, here's India's (they're in zoom 3). But looking at the data it's because those are POV "assertion" lines that need a lower min_zoom when they are part of a POV, even though normally we wouldn't want to see them that early. The easiest fix now is to revise their min_zoom.

India:

China:

Pakistan:

Default:

[nvkelso]

I've verified https://github.com/tilezen/vector-datasource/pull/1904, which resolves all outstanding code change requirements.