Duplicate Entries in Cities List

[ManojKiranA]

I've created a php/laravel script to import the Countries, States, Cities and while importing I have found the following duplicates in cities json file

Duplicates Entries

{ "AO_Angola_BGU_Benguela Province_Catumbela": [ { "id": 597, "name": "Catumbela", "state_id": 218, "state_code": "BGU", "state_name": "Benguela Province", "country_id": 7, "country_code": "AO", "country_name": "Angola", "latitude": "-12.43002000", "longitude": "13.54677000", "wikiDataId": "Q1051360" }, { "id": 153652, "name": "Catumbela", "state_id": 218, "state_code": "BGU", "state_name": "Benguela Province", "country_id": 7, "country_code": "AO", "country_name": "Angola", "latitude": "-12.39860000", "longitude": "13.54940000", "wikiDataId": "Q3840" } ], "CN_China_TJ_Tianjin_Zhaoguli": [ { "id": 149214, "name": "Zhaoguli", "state_id": 2276, "state_code": "TJ", "state_name": "Tianjin", "country_id": 45, "country_code": "CN", "country_name": "China", "latitude": "39.16673000", "longitude": "117.24191000", "wikiDataId": "Q13667923" }, { "id": 149236, "name": "Zhaoguli", "state_id": 2276, "state_code": "TJ", "state_name": "Tianjin", "country_id": 45, "country_code": "CN", "country_name": "China", "latitude": "39.16673000", "longitude": "117.24191000", "wikiDataId": "Q13667923" } ], "IN_India_GJ_Gujarat_Amod": [ { "id": 57683, "name": "Amod", "state_id": 4030, "state_code": "GJ", "state_name": "Gujarat", "country_id": 101, "country_code": "IN", "country_name": "India", "latitude": "21.99317000", "longitude": "72.87047000", "wikiDataId": "Q2724262" }, { "id": 147574, "name": "Amod", "state_id": 4030, "state_code": "GJ", "state_name": "Gujarat", "country_id": 101, "country_code": "IN", "country_name": "India", "latitude": "21.99100000", "longitude": "72.87100000", "wikiDataId": "Q2724262" } ], "IN_India_GJ_Gujarat_Nadiad": [ { "id": 133106, "name": "Nadiad", "state_id": 4030, "state_code": "GJ", "state_name": "Gujarat", "country_id": 101, "country_code": "IN", "country_name": "India", "latitude": "22.69385000", "longitude": "72.86157000", "wikiDataId": "Q219039" }, { "id": 147636, "name": "Nadiad", "state_id": 4030, "state_code": "GJ", "state_name": "Gujarat", "country_id": 101, "country_code": "IN", "country_name": "India", "latitude": "22.70000000", "longitude": "72.87000000", "wikiDataId": "Q219039" } ], "IN_India_MH_Maharashtra_Beed": [ { "id": 57912, "name": "Beed", "state_id": 4008, "state_code": "MH", "state_name": "Maharashtra", "country_id": 101, "country_code": "IN", "country_name": "India", "latitude": "18.98921000", "longitude": "75.75634000", "wikiDataId": "Q814033" }, { "id": 58026, "name": "Beed", "state_id": 4008, "state_code": "MH", "state_name": "Maharashtra", "country_id": 101, "country_code": "IN", "country_name": "India", "latitude": "18.83333000", "longitude": "75.75000000", "wikiDataId": "Q814037" } ], "IN_India_MH_Maharashtra_Bhoom": [ { "id": 147709, "name": "Bhoom", "state_id": 4008, "state_code": "MH", "state_name": "Maharashtra", "country_id": 101, "country_code": "IN", "country_name": "India", "latitude": "18.45972222", "longitude": "75.66083333", "wikiDataId": "Q796520" }, { "id": 58024, "name": "Bhoom", "state_id": 4008, "state_code": "MH", "state_name": "Maharashtra", "country_id": 101, "country_code": "IN", "country_name": "India", "latitude": "18.45908000", "longitude": "75.65877000", "wikiDataId": "Q796520" } ], "NL_Netherlands_NH_North Holland_De Weere": [ { "id": 149883, "name": "De Weere", "state_id": 2612, "state_code": "NH", "state_name": "North Holland", "country_id": 156, "country_code": "NL", "country_name": "Netherlands", "latitude": "52.77444444", "longitude": "4.86694444", "wikiDataId": "Q2456668" }, { "id": 149884, "name": "De Weere", "state_id": 2612, "state_code": "NH", "state_name": "North Holland", "country_id": 156, "country_code": "NL", "country_name": "Netherlands", "latitude": "52.72280000", "longitude": "4.99860000", "wikiDataId": "Q1810945" } ], "NL_Netherlands_NH_North Holland_Driehuizen": [ { "id": 149890, "name": "Driehuizen", "state_id": 2612, "state_code": "NH", "state_name": "North Holland", "country_id": 156, "country_code": "NL", "country_name": "Netherlands", "latitude": "52.57890000", "longitude": "4.80890000", "wikiDataId": "Q2594292" }, { "id": 149891, "name": "Driehuizen", "state_id": 2612, "state_code": "NH", "state_name": "North Holland", "country_id": 156, "country_code": "NL", "country_name": "Netherlands", "latitude": "53.04666667", "longitude": "4.77611111", "wikiDataId": "Q2115818" } ], "NO_Norway_46_Vestland_Dale": [ { "id": 79099, "name": "Dale", "state_id": 1018, "state_code": "46", "state_name": "Vestland", "country_id": 165, "country_code": "NO", "country_name": "Norway", "latitude": "61.36353000", "longitude": "5.40036000", "wikiDataId": "Q1783017" }, { "id": 79098, "name": "Dale", "state_id": 1018, "state_code": "46", "state_name": "Vestland", "country_id": 165, "country_code": "NO", "country_name": "Norway", "latitude": "60.58639000", "longitude": "5.81888000", "wikiDataId": "Q1951189" } ], "NO_Norway_30_Viken_Nes": [ { "id": 79389, "name": "Nes", "state_id": 1011, "state_code": "30", "state_name": "Viken", "country_id": 165, "country_code": "NO", "country_name": "Norway", "latitude": "60.12252000", "longitude": "11.46570000", "wikiDataId": "Q488026" }, { "id": 79390, "name": "Nes", "state_id": 1011, "state_code": "30", "state_name": "Viken", "country_id": 165, "country_code": "NO", "country_name": "Norway", "latitude": "60.56804000", "longitude": "9.10169000", "wikiDataId": "Q483986" } ], "ES_Spain_AV_Ávila_Chamartín": [ { "id": 33614, "name": "Chamartín", "state_id": 1189, "state_code": "AV", "state_name": "Ávila", "country_id": 207, "country_code": "ES", "country_name": "Spain", "latitude": "40.70267000", "longitude": "-4.95769000", "wikiDataId": "Q732971" }, { "id": 33615, "name": "Chamartín", "state_id": 1189, "state_code": "AV", "state_name": "Ávila", "country_id": 207, "country_code": "ES", "country_name": "Spain", "latitude": "40.46206000", "longitude": "-3.67660000", "wikiDataId": "Q732971" } ], "ES_Spain_SA_Salamanca_Salamanca": [ { "id": 36744, "name": "Salamanca", "state_id": 1147, "state_code": "SA", "state_name": "Salamanca", "country_id": 207, "country_code": "ES", "country_name": "Spain", "latitude": "40.96882000", "longitude": "-5.66388000", "wikiDataId": "Q577585" }, { "id": 36745, "name": "Salamanca", "state_id": 1147, "state_code": "SA", "state_name": "Salamanca", "country_id": 207, "country_code": "ES", "country_name": "Spain", "latitude": "40.42972000", "longitude": "-3.67975000", "wikiDataId": "Q1773521" } ], "ES_Spain_ZA_Zamora_Fonfría": [ { "id": 34207, "name": "Fonfría", "state_id": 1161, "state_code": "ZA", "state_name": "Zamora", "country_id": 207, "country_code": "ES", "country_name": "Spain", "latitude": "40.99566000", "longitude": "-1.08475000", "wikiDataId": "Q574337" }, { "id": 34208, "name": "Fonfría", "state_id": 1161, "state_code": "ZA", "state_name": "Zamora", "country_id": 207, "country_code": "ES", "country_name": "Spain", "latitude": "41.63493000", "longitude": "-6.14081000", "wikiDataId": "Q1653149" } ], "ES_Spain_ZA_Zamora_Villaescusa": [ { "id": 38044, "name": "Villaescusa", "state_id": 1161, "state_code": "ZA", "state_name": "Zamora", "country_id": 207, "country_code": "ES", "country_name": "Spain", "latitude": "41.20617000", "longitude": "-5.46406000", "wikiDataId": "Q1922230" }, { "id": 38045, "name": "Villaescusa", "state_id": 1161, "state_code": "ZA", "state_name": "Zamora", "country_id": 207, "country_code": "ES", "country_name": "Spain", "latitude": "42.96262000", "longitude": "-4.16706000", "wikiDataId": "Q1922230" } ], "UA_Ukraine_53_Poltavska oblast_Baranivka": [ { "id": 149321, "name": "Baranivka", "state_id": 5071, "state_code": "53", "state_name": "Poltavska oblast", "country_id": 230, "country_code": "UA", "country_name": "Ukraine", "latitude": "50.30000000", "longitude": "27.66666667", "wikiDataId": "Q148917" }, { "id": 149322, "name": "Baranivka", "state_id": 5071, "state_code": "53", "state_name": "Poltavska oblast", "country_id": 230, "country_code": "UA", "country_name": "Ukraine", "latitude": "49.94809180", "longitude": "34.00827580", "wikiDataId": "Q891724" } ] }

[alex-gabor]

The following States also have duplicates: Petite Riviere, Mosfellsb, Bris e Verdi re, Clemencia, Dagoti re, Cr ve Coeur, Morcellement Saint Andre, Esperance Trebuchet, Riviere Des Anguilles, Bielsko-Biala.

[dr5hn]

@ManojKiranA I've removed duplicate cities. @alex-gabor Thanks for sharing list of states, will review and remove them soon!

[ManojKiranA]

@dr5hn I'll run it again and confim.

[alex-gabor]

@dr5hn City Mosfellsbaer in Iceland is still duplicate in the latest update, after the fix (see CIty ids 135200 and 135201). You may have missed it.