Additional shield_text cleanup

[sensescape]

Followup to https://github.com/tilezen/vector-datasource/issues/1062... this will be running tally.

• [x] Argentina
  • [x] Argentina roads are generally prefixed with 2 characters and then a third [A-Z] character can prefix the number part of the ref, and should be included in the generated shield_text (but is currently not):
  • https://www.openstreetmap.org/way/85235685
    • ref: RNA002
    • shield_text: bug: 002, expected A002
  • [x] Argentina provincial routes often aren't in relations with proper network value (expected to be AR:provincial), but they do have a RP prefix in he ref.
  • https://www.openstreetmap.org/way/132808888
    • ref: RP21
    • shield_text: bug: not exported, expected 21
    • network: bug: not exported, expected AR:provincial
• [x] Australia
  • Some roads have network, but almost none of those are prefixed with a country code (they should be).
  • Sometimes the network has addr:country=AU (and even addr:state=QLD), which could be used like we have operator country assignments now.
  • [ ] Australia AU:N-highway national highway (slowly being replaced with A-road and M-road)
  • [ ] Australia AU:A-road
  • [ ] Australia AU:M-road
    • https://www.openstreetmap.org/relation/198331
  • [ ] Australia AU:B-road
  • [ ] Australia AU:C-road
  • [ ] Australia AU:N-route for national route
  • [ ] Australia AU:S-route
    • https://www.openstreetmap.org/relation/7936070#map=11/-25.5780/152.3691
  • [ ] Australia AU:Metro-road
  • [ ] Australia AU:T-drive
    • https://www.openstreetmap.org/relation/7936092#map=12/-27.9501/153.2236
    • https://www.openstreetmap.org/way/17796959#map=17/-26.36878/152.70198

• [x] Brazil

  • [ ] Network values of BR should sort first for Brazil, then BR:Trans-Amazonian, than the various BR:* state networks.
  • [ ] Brazil BR:Trans-Amazonian Trans-Amazonian Highway
  • if( shield_text == 230) && (feature.network == 'BR'), and should be "sorted" before general BR networks (see second relation), and BR:* state networks.
    • https://www.openstreetmap.org/way/258644490
    • https://www.openstreetmap.org/relation/3948880
    • https://www.openstreetmap.org/relation/4605583
  • [ ] Various state highways missing network but have ref and a country specific source (limit to highways and major roads?):

  • https://www.openstreetmap.org/way/472168229 ref=PE-050 source=IBGE

    code	name	type
    BR:DF	Distrito Federal	federal district
    BR:AC	Acre	state
    BR:AL	Alagoas	state
    BR:AP	Amapá	state
    BR:AM	Amazonas	state
    BR:BA	Bahia	state
    BR:CE	Ceará	state
    BR:ES	Espírito Santo	state
    BR:GO	Goiás	state
    BR:MA	Maranhão	state
    BR:MT	Mato Grosso	state
    BR:MS	Mato Grosso do Sul	state
    BR:MG	Minas Gerais	state
    BR:PA	Pará	state
    BR:PB	Paraíba	state
    BR:PR	Paraná	state
    BR:PE	Pernambuco	state
    BR:PI	Piauí	state
    BR:RJ	Rio de Janeiro	state
    BR:RN	Rio Grande do Norte	state
    BR:RS	Rio Grande do Sul	state
    BR:RO	Rondônia	state
    BR:RR	Roraima	state
    BR:SC	Santa Catarina	state
    BR:SP	São Paulo	state
    BR:SE	Sergipe	state
    BR:TO	Tocantins	state

• [x] Canada
  • [ ] CA:transcanada highway network should sort before CA:yellowhead, before other CA:* https://www.openstreetmap.org/way/95904761 https://www.openstreetmap.org/relation/1121378 https://www.openstreetmap.org/relation/1307245
• [x] China
  • The majority of roads are tagged with network, though a significant minority of G and S roads aren't tagged correctly.
  • Generally working but there are 2 types of "expressways" that need disambiguation.
  • [x] CN-expressways network should sort before CN-expressways-regional, before AsianHighway, before other CN-roads. (AsianHighway is like Europe E-roads at end of the lists, same is true for Russia for instance.)
    • https://www.openstreetmap.org/way/127553001
    • https://www.openstreetmap.org/relation/5869573
    • https://www.openstreetmap.org/relation/913366
    • ref: G42
    • shield_text: bug: 5, expected G42
    • network: bug: AsianHighway, expected CN-expressways
  • [x] CN-expressways-regional is a subset of CN-expressways network tagged features when the ref begins with an S
    • https://www.openstreetmap.org/way/168706293
    • https://www.openstreetmap.org/relation/194414
    • ref: S20
    • shield_text: good as-is: S20
    • network: bug: CN-expressways, expected CN-expressways-regional
• [x] France
  • The majority of important national roads are tagged with network, though a majority of D or department level roads aren't tagged correctly at all with networks.
  • [ ] E-road network shield_text should strip leading 0.
    • https://www.openstreetmap.org/way/16108247
    • https://www.openstreetmap.org/relation/1159709
    • https://www.openstreetmap.org/relation/2133077
    • shield_text: bug: E05, expected E5
    • network: good/expected: e-road
  • [ ] Make synthetic FR:D-roads network when no relation is present
    • NOTE: Turkey has similar D * and D* roads that collide without a spatial check.
    • https://www.openstreetmap.org/way/35593697
    • shield_text: bug: not exported, expected D983
    • network: bug: not present, expected FR:D-road
  • [ ] Replace more specific FR:##:D-road with FR:D-road network
    • NOTE: Where ## is a numerical department code. Only a handful of departments have more than 10 instances:FR:01:D-road, FR:52:D-road, FR:51:D-road, FR:55:D-road, FR:08:D-road so we could do a replace, but a general pattern match would be better.
    • https://www.openstreetmap.org/way/137943124
    • https://www.openstreetmap.org/relation/6353081
    • shield_text: good / expected D17
    • network: bug: FR:01:D-road, expected FR:D-road
  • [ ] Update operator-based fallback network logic to dedup &/or not trigger unless no other network, and sort order should be FR:A-road, FR:N-road, FR:D-road, FR, e-road.
    • NOTE: Sometimes
    • https://www.openstreetmap.org/way/147156883
    • https://www.openstreetmap.org/relation/72825
    • https://www.openstreetmap.org/relation/3904095
    • https://www.openstreetmap.org/relation/158074
    • operator: DIRNO
    • shield_text: bug: E03, A84
    • network: bug: e-road, expected FR:A-road
    • all_networks: bug: e-road, FR, e-road, FR:A-road, expected FR:A-road, e-road, e-road (no FR at all)
• [x] Germany
  • Most federal autobahns are setup, but middle and local level are not.
  • [ ] BAB network roads should receive a DE:BAB network value instead to enforce country code prefixes.
    • https://www.openstreetmap.org/way/52480330
    • https://www.openstreetmap.org/relation/20904 (BAB)
    • https://www.openstreetmap.org/relation/57817 (e-road)
    • https://www.openstreetmap.org/relation/1485158 (junk)
    • shield_text: good as-is: A1
    • network: bug: BAB, expected DE:BAB
  • [ ] Extend operator based network assignment for Via Solutions Südwest to map to network DE:BAB.
    • https://www.openstreetmap.org/way/207881430
    • https://www.openstreetmap.org/relation/2148632
    • https://www.openstreetmap.org/relation/2148374
    • https://www.openstreetmap.org/relation/945817
    • operator = Via Solutions Südwest
    • shield_text: good as-is: A5
    • network: bug: DE, expected DE:BAB
  • [ ] Extend operator based network assignment for autobahnplus A8 GmbH to map to network DE:BAB.
    • NOTE: This one is odd since one of the relations has network=BAB, that may point at operator wrongly taking priority over extant network values.
    • https://www.openstreetmap.org/way/280688897
    • https://www.openstreetmap.org/relation/2166961
    • https://www.openstreetmap.org/relation/401492
    • https://www.openstreetmap.org/relation/2148631
    • operator = autobahnplus A8 GmbH
    • shield_text: good as-is: A8
    • network: bug: NULL, expected DE:BAB
  • [ ] Bundesstraßen* (federal road) network value roads should receive a DE:BS network value instead of generic DE based on operator fallback.
    • https://www.openstreetmap.org/way/5136775
    • https://www.openstreetmap.org/relation/3403428
    • https://www.openstreetmap.org/relation/106962
    • https://www.openstreetmap.org/relation/3403427
    • https://www.openstreetmap.org/relation/2645003 (a relation on top of a relation)
    • operator = Bundesrepublik Deutschland
    • shield_text: good as-is: B6
    • network: bug: NULL, expected DE:BS (or at least DE with old logic)
  • [ ] Bundesstraßen* (federal road) without a network should receive a DE:BS network value instead based in the B* ref pattern match, and should strip the space in B #
    • https://www.openstreetmap.org/way/246556060
    • https://www.openstreetmap.org/relation/29891 (but missing a network tag)
    • shield_text: bug: B 9 should be: B9
    • network: bug: NULL, expected DE:BS
  • [ ] Landesstraßen* (state/region road) network value roads should receive a DE:Landesstraßen network value instead.
    • Most common values are Landesstraßen NRW, Landesstraßen Niedersachsen, Landesstraßen 2. Ordnung im Saarland, Landesstraßen Brandenburg, Landesstraßen 1. Ordnung im Saarland, but we should use match on general pattern.
    • https://www.openstreetmap.org/way/28859327
    • https://www.openstreetmap.org/relation/3838880
    • shield_text: good as-is: L84
    • network: bug: Landesstraßen NRW, expected DE:LS
  • [ ] Landesstraßen (state/region) roads without a network should receive a DE:LS network value based on L* ref pattern matching (limited to Germany bbox), more basic example.
    • https://www.openstreetmap.org/way/114110755
    • shield_text: good as-is: L39
    • network: bug: null, expected DE:LS
  • [ ] Kreisstraßen (county) roads with a network should receive a DE:KS network value based on network value subsitution.
    • https://www.openstreetmap.org/way/24319153
    • https://www.openstreetmap.org/relation/5185505
    • shield_text: good as-is: K203
    • network: bug: Kreisstraßen Hildesheim, expected DE:KS
  • [ ] Kreisstraßen (county) roads without a network should receive a DE:KS network value based on K* ref pattern matching (limited to Germany bbox, watch out for Malaysia overlap)
    • https://www.openstreetmap.org/way/25839765
    • shield_text: bug NULL. should be:K40
    • network: bug: NULL, expected DE:KS
• [x] Japan
  • NOTE: Bit of a hot mess.
  • [ ] JP:national network roads often missing but can be added looking at way name.
    • https://www.openstreetmap.org/way/242802845
    • name: 国道163号 (国道 means national route with the postifx 号 indicating that the number is a route number, but sometimes there's additional postfixes)
    • shield_text: good as-is: 163
    • network: bug: null, expected JP:national
  • [ ] jp:prefectural network should be replaced with JP:prefectural
    • https://www.openstreetmap.org/way/415221603
    • https://www.openstreetmap.org/way/415221603
    • shield_text: good as-is: 305
    • network: bug: jp:prefectural, expected JP:prefectural
• [x] Mexico
  • [ ] Often the ref will prefix with MEX (note space), no network
    • https://www.openstreetmap.org/way/285597347
    • ref: good as-is: MEX 15D
    • shield_text: bug: null, should be: 15D
    • network: bug: null, expected MX:MEX
  • [ ] Often the ref will prefix with MEX (note space), no network
    • https://www.openstreetmap.org/way/184722720
    • ref: good as-is: MEX610
    • shield_text: bug: null, should be: 610
    • network: bug: null, expected MX:MEX
  • [ ] Network values of MEX need to be prefixed with country code: MX:MEX (substitution)
    • https://www.openstreetmap.org/way/317748112
    • shield_text: good as-is: 120
    • network: bug: MEX, expected MX:MEX
• [x] South Korea
  • A void on the map that need network relation examples added in OSM data directly. There are a couple KR:national relations (perhaps those should be KR:expressway though?), but there should be many...
    • [x] KR:expressway for https://en.wikipedia.org/wiki/Expressways_in_South_Korea added in capital
    • [x] KR:national https://en.wikipedia.org/wiki/National_highways_of_South_Korea added in capital
    • [x] KR:local https://en.wikipedia.org/wiki/Local_highways_of_South_Korea added in Jeju
  • [ ] AH network should substitute out for AsianHighway:
    • https://www.openstreetmap.org/way/547188348
    • https://www.openstreetmap.org/relation/3836355
    • shield_text: good-as-is: 1
    • network: bug: AH, expected AsianHighway
• [x] United Kingdom
  • Some networks mostly based on operator, but lots of gaps...
  • [ ] GB network based on Highways England operator should substitute out for GB:M-road when ref starts with M:
    • https://www.openstreetmap.org/way/3256123
    • https://www.openstreetmap.org/relation/106164
    • shield_text: good-as-is: M25
    • network: bug: GB, expected GB:M-road
  • [ ] GB network based on Highways England operator should substitute out for GB:A-road when ref starts with A:
    • https://www.openstreetmap.org/way/152213939
    • shield_text: good-as-is: A3
    • network: bug: GB, expected GB:A-road
  • [ ] network should be set for GB:A-road-green ??? when ref starts with A in the bounding box of the United Kingodm, but there seem to be 2 flavours of A-road, namely GREEN when highway=motorway ???:
    • https://www.openstreetmap.org/way/298766837
    • shield_text: good-as-is: A30
    • network: bug: null, expected GB:A-road-green
  • [ ] network should be set for GB:A-road-white when ref starts with A in the bounding box of the United Kingodm, but there seem to be 2 flavours of A-road, namely WHITE when not highway=motorway ???:
    • https://www.openstreetmap.org/way/171868782
    • shield_text: good-as-is: A312
    • network: bug: null, expected GB:A-road-white
    • NOTE: Generally speaking these shields should be revealed later on the map than other shields
• [x] Greece
  • Generally really good!
  • [x] GR:national networks need to strip off the leading EO on the shield_text.
    • https://www.openstreetmap.org/way/299496935
    • https://www.openstreetmap.org/relation/5863927
    • shield_text: bug: ΕΟ8 should be: 8
    • network: good-as-is GR:national
  • [x] GR networks (based on operator) should sub out GR:motorway (another operator overriding network problem?)
    • https://www.openstreetmap.org/way/85601631
    • https://www.openstreetmap.org/relation/363465
    • https://www.openstreetmap.org/relation/1084054
    • shield_text: good-as-is: A6
    • network: bug GR, should be: GR:motorway

[nvkelso]

These ones are the "good" networks that need to be normalized to better values:

• [x] Canada set to network CA:transcanada when nat_name = Trans-Canada Highway
• [x] Canada, New Brunswick set to network CA:NB2 when shield_text is >= 100 but < 200.
• [x] Canada, New Brunswick set to network CA:NB3 when shield_text is >= 200.
• [x] Malaysia replace my:federal with MY:federal
• [x] Norway replace no:Fylkesvei and no:Fylkesvei with NO:Fylkesvei
• [x] Peru replace pe:national with PE:national PE:PE
• [x] South Africa replace za:national with ZA:national
• [x] South Africa replace za:regional with ZA:regional ZA:region_code
• [x] Thailand replace TH-roads with TH:road
• [x] Thailand replace TH-network with TH:network
• [x] Ukraine replace ua:international with UA:international
• [x] Ukraine replace ua:national with UA:national
• [x] Ukraine replace ua:regional with UA:regional
• [x] Ukraine replace ua:territorial with UA:territorial
• [x] Turkey replace TR-roads with TR:road
• [x] Turkey replace TR-road with TR:road (think this is the same as above)
• [x] Turkey replace TR-motorway with TR:motorway
• [x] Turkey set TR:DY for refs that begin with D .
• [x] Vietnam replace VN-roads with VN:roads

I'll followup with "funky" per country refs that need to get networks...

[nvkelso]

These are out of the "funky" client side scene file hacks:

• [x] ARGENTINA and ref begins with RP then network: AR:RP
• [x] BRAZIL when ref begins with BR and the network (via operator?) is just BR make network is BR:BR
• [x] FRANCE ensure A, N, D and other roads keep their leading char but remove the space before the number for shield text.
• [x] FRANCE needs some standardization for department networks so they are 2 levels deep only? And/or report all the department codes here so the texture defintion client side can dual list them.
• [x] INDIA remove NH text from shield text for IN:NH networks
• [x] INDIA remove SH text from shield text for IN:SH networks
• [x] INDIA set network to IN:NH when ref begins with NH
• [x] INDIA set network to IN:SH when ref begins with SH
• [ ] ITALY B-road have an SS ref varient to look after. The scene file sets network to IT:B-road when network is IT:* and not IT:A-road or ref begins with SS? I think the SS should probably receive a different network that's aliased client side, what do you think? Similar for SR and SP.
• [x] MALAYSIA expressways need a MY:expressway network or similar. Scene file is confusing if all these are valid ref tests: E, B, J, K, P.
• [x] ROMANIA roads with refs starting with DN should return RO:DX network (why not DN?)
• [x] PERU "national" roads with refs starting with PE should return PE:PE
• [x] PERU "provincial" roads with refs starting with thier region code should return PE:{region_code}. Partial list: AR, AY, CU, HV, IC, MO, TA, PU, LM. Shield text shouldn't include that prefix.
• [x] SPAIN autopistas with refs staring RP should return ES:AP network
• [x] SPAIN national routes with refs staring N or N- should return ES:N network. Note some of these get shield text now, some don't.
• [x] SPAIN national routes with refs staring NA should return ES:AA network. (yellow)
• [x] SPAIN "provincial" roads with refs starting with thier region code should return ES:{region_code}. Partial list: AS, BU, C-, EI, M-, PA, TF and AG, AS, BI, C-, CM, Ma, MA, Me, EI, EX, FV, GC, GI, LO, LR, LZ, M-, PA, RM, TF, V-. Shield text shouldn't include that prefix.
• [x] SINGAPORE look for refs with AYE, BKE, ECP, PIE, KJE, KPE, MCE, SLE, TPE and set to SG:Expressway network?
• [x] SOUTH AFRICA refs M and MO should get network ZA:M Not sure difference between M and MO.
• [x] TURKEY refs D roads should get network TR:DY
• [x] TURKEY Otoyol roads with refs O roads should get network TR:O-roads or drop the 0- based on OSM tag prevenance?
• [x] VIETNAM refs QL to VN:QL network (eg: us route)
• [x] VIETNAM refs CT to VN:CT network (eg: us interstate)
• [x] VIETNAM network of VI-roads should we normalize that? I think it should be client alias to VN:QL artwork.
• [x] VIETNAM highway name:en of Ho Chi Minh Highway should get VN:QL network instead of it's VI-roads network? maybe just change the data after confirming?

[nvkelso]

• [x] Switzerland CH:motorway versus CH:national versus CH:nationalstrasse
• [x] Poland sort order for expressway versus motorway

[nvkelso]

Australia M, B, C roads should keep their prefix (eg M1 instead of 1) in the shield_text calculation. See Melbourne.

[nvkelso]

Natural Earth e-road shield_text needs to include the E (see zoom 7).

[nvkelso]

New Zealand State Highway isn't coming thru as NZ:SH network:

• https://www.openstreetmap.org/edit?way=119547951#map=17/-36.93116/174.85064

[nvkelso]

Not all Canadian shields are drawing correctly: https://github.com/tangrams/tangram/issues/662

Not all Canadian shields have default network of CA.

[nvkelso]

Chile Panamerican when network is CL:national and shield_text is 5 than network should be CL:panamerican instead.

[nvkelso]

Indonesia National should be ID:national (alternatively ID:motorway) but is just ID. Test for motorway and assign?

[nvkelso]

Not all Norway shields have default network of NO (very common). Similar to to https://github.com/tilezen/vector-datasource/issues/1491#issuecomment-414106052.

Also odd that the e-road network's shield_text values here include the space which is removed in other countries.

See: http://localhost:8000/#12/60.1398/10.3021

[nvkelso]

Thailand seems broken (just TH with no more detail).

[nvkelso]

For Mexico, to disambiguate between the region of Mexico and the federal routes let's use MX:MX instead of MX:MEX (reserving the 3-char for the capital region).

[nvkelso]

India national highways are sometimes not receiving a shield_text value: https://www.openstreetmap.org/edit?way=180205706#map=17/20.92767/72.99029.

[nvkelso]

Blue Italian networks generally aren't being set:

Examples:

• https://www.openstreetmap.org/edit?way=32759362#map=17/45.06297/8.27979
• https://www.openstreetmap.org/edit?way=220398452#map=17/46.05134/11.48824

Goals:

• IT:SR Regional road
• IT:SS Strade Statali (same artwork as IT:SR) - State highways
• IT:SP strada provinciale (same artwork as IT:SR) - provincial road

[nvkelso]

Netherlands roads could be set looking at the ref prefix, and we have artwork:

• NL:A-road national motorway route, ‘A’ prefix
• NL:N-road provincial route, ‘N’ prefix
• NL:S-road city route, ‘S’ prefix

[nvkelso]

Austria is mostly working for AT:A-road relations, but there are many roads not in relations starting with A ref that could be assigned that network value. Similarly B refs could be given AT:B-road.

[nvkelso]

Belgium could use similar cleanup as Netherlands.

[nvkelso]

For the most part this all looks good. I took the last few QA comments and moved them into https://github.com/tilezen/vector-datasource/issues/1561 for the next milestone.

We should fix the Australia ref prefixes in this release, that's in https://github.com/tilezen/vector-datasource/issues/1562.