[PROD] Missing 'districts' compared to admin1 shapefile

[geohacker]

Here's a list of all districts that are missing or could be matched because the names are different / has unicode issues (related to #1456)

code,name
N.A,Ashmore and Cartier Islands
N.A,Bassas da India
N.A,Bird Island
N.A,Glorioso Island
N.A,Liancourt Rocks
N.A,Paracel Islands
N.A,Scarborough Reef
N.A,Spratly Islands
N.A,Tromelin Island
AG002,Redonda
BS013,Grand Cay
BS031,Spanish Wells
f,Saint Martin (France)
f,Saint Martin (France)
N.A,United Kingdom of Great Britain and Northern Ireland
MC001,Monaco
MH002,Ralik Chain
MT017,Isla
MT055,Mtarfa
MT049,Valletta
MU002,Coco Island
MV004,Central Province
MV005,North Central Province
MV006,North Province
MV007,Upper North Province
MV002,Upper South Province
PW003,Kayangel
PW013,Sonsorol
TK001,Tokelau
TV001,Funafuti
TV002,Nanumanga
TV003,Nanumea
TV004,Niutao
TV005,Nui
TV006,Nukufetau
TV007,Nukulaelae
UM001,Baker Island
UM002,Howland Island
UM003,Jarvis Island
UM004,Johnston Atoll
VA001,Vatican City
XZ001,Deçan / Decane
XZ002,Dragash / Dragas
XZ003,Ferizaj / Urosevac
XZ004,Fushë Kosovë / Kosovo Polje
XZ005,Gjakovë / Djakovica
XZ006,Gjilan / Gnjilane
XZ007,Gllogoc / Glogovac
XZ008,Graçanicë / Gracanica
XZ009,Hani i Elezit / Elez Han
XZ010,Istog / Istok
XZ011,Junik / Junik
XZ012,Kaçanik / Kacanik
XZ013,Kamenicë / Kamenica
XZ014,Klinë / Klina
XZ015,Kllokoti / Klokot
XZ016,Leposaviq / Leposavic
XZ017,Lipjan / Lipljan
XZ018,Malishevë / Malisevo
XZ019,Mamushë / Mamusa
XZ020,Mitrovicë / Mitrovica
XZ038,Mitrovicë Veriore / Severna Mitrovica
XZ021,Novobërdë / Novo Brdo
XZ022,Obiliq / Obilic
XZ023,Partesh / Partes
XZ024,Pejë / Pec
XZ025,Podujevë / Podujevo
XZ026,Prishtinë / Pristina
XZ027,Prizren / Prizren
XZ028,Rahovec / Orahovac
XZ029,Ranillug / Ranilug
XZ030,Shtërpcë /Strpce
XZ031,Shtime / Stimlje
XZ032,Skënderaj / Srbica
XZ033,Suharekë / Suva Reka
XZ034,Viti / Vitina
XZ035,Vushtri / Vucitrn
XZ036,Zubin Potok / Zubin Potok
XZ037,Zveçan /Zvecane
NP01,Sudurpashchim

This is also related to #1464 #938

[jhenshall]

We've had an email from a user regarding two of the missing districts for Palau not being available when needed for field reports. Just noting here for reference but will already be covered by this ticket. Districts are: PW003,Kayangel PW013,Sonsorol

[jhenshall]

The reasons for the missing districts seems to vary with some being in disputed territories (e.g. Kosovo) and other that are simply missing from the GO database. For this release, we are prioritising adding the missing districts from independent nations. We will then review all districts as part of the wider geo-data review. Districts being added now:

ADMIN01COD	ADMIN01NAM	ADMIN00NAM
AG002	Redonda	Antigua and Barbuda
BS013	Grand Cay	Bahamas
BS031	Spanish Wells	Bahamas
MH002	Ralik Chain	Marshall Islands
MV004	Central Province	Maldives
MV005	North Central Province	Maldives
MV006	North Province	Maldives
MV007	Upper North Province	Maldives
MV002	Upper South Province	Maldives
PW003	Kayangel	Palau
PW013	Sonsorol	Palau
TV001	Funafuti	Tuvalu
TV002	Nanumanga	Tuvalu
TV003	Nanumea	Tuvalu
TV004	Niutao	Tuvalu
TV005	Nui	Tuvalu
TV006	Nukufetau	Tuvalu
TV007	Nukulaelae	Tuvalu

[geohacker]

These have been imported to staging. Thanks @jhenshall and @GregoryHorvath!

[jhenshall]

@geohacker @GregoryHorvath - I've just noticed that the newly added districts here and for Nepal #1422 do not have a country_name value in the API. I missed this before, as the front end functionality is fine and must work on ISO2 code? Would be a nice to have on staging, but definitely when adding to prod. 🙂

[geohacker]

@jhenshall Nice catch. Yes, that's because the shapefile that we used to import Nepal districts didn't have anything in the COUNTRY column. The import script will grab the name if that column is not null https://github.com/IFRCGo/go-api/blob/develop/api/management/commands/import-admin1-data.py#L79

On staging, I recommend you going ahead in the admin and adding these as fixing for just these would likely be a bunch of work programmatically.

If you'd like this to be fixed in production, can you fix this in the shapefile and share with us? We can add it when we release!

[jhenshall]

Thanks @geohacker. I've filled in the missing countries on staging and updated the country column in the shapefile to be used for prod.

[jhenshall]

@geohacker - I've added the updated shapefiles with COUNTRY column populated to the same filepaths on dropbox (moved the old ones to archived folder). Hopefully this means it will now carry across the country_name attribute on prod.

[batpad]

@geohacker can you please close this if everything has been resolved on prod or create a new issue / clarify remaining bits?

[geohacker]

This isn't entirely resolved and should go into 4.4 for @jhenshall to decide how we approach the missing districts.

[jhenshall]

Can we make country_iso nullable in the districts table? This is actually a symptom of this issue that the districts table stores country name and iso separate to country table. I assume the broader fix is not possible this time round, so need to have nullable iso for disputed territories. Example screenshot of how one of the disputed territories should look:

@GregoryHorvath @geohacker @batpad

[jhenshall]

Remaining districts added in March 2021 - all districts from ICRC shapefile now in GO! 🎉