Trying to import with --import=all return errors

[matheusbrat]

Hello,

I'm trying to fetch data for all countries, the data that I need is basically, Country, Region, SubRegion and City.

I was using: CITIES_POSTAL_CODES = [] CITIES_LOCALES = ['en'] DATABASES['default']['ENGINE'] = 'django.contrib.gis.db.backends.postgis'

Also, I'm on mac os x, using postgresql 9.4.4 and Django 1.8.4 with python 3.

When running:

python manage.py cities --import=all --force

I receive a lot of messages saying that region was not found. So I decided to go step by step and see how that goes. I started with:

python manage.py cities --import=country --force which fetchted 250 countries to my database python manage.py cities --import=region --force which fetchted 3889 region to my database python manage.py cities --import=subregion --force which fetchted 39380 subregion to my database

Even for importing subregions there are some messages like: Subregion: Gldanskiy Rayon: Cannot find region: 00

If I run the fetch city command: python manage.py cities --import=city --force which fetched only 45771 cities to my database.

I also receive A LOT of messages

DE: Lachen-Speyerdorf: Cannot find subregion: 00 -- skipping DE: Osternienburger Land: Cannot find subregion: 00 -- skipping EH: Laayoune Plage: Cannot find region: 00 -- skipping FK: Stanley: Cannot find region: 00 -- skipping GE: Tsnori: Cannot find region: 00 -- skipping GE: Tqibuli: Cannot find region: 00 -- skipping GE: Surami: Cannot find region: 28 -- skipping GE: Naghvarevi: Cannot find region: 00 -- skipping GE: Gudauri: Cannot find region: 00 -- skipping GE: Gantiadi: Cannot find region: 00 -- skipping GE: Akhalk’alak’i: Cannot find region: 06 -- skipping GE: Agara: Cannot find region: 25 -- skipping GF: Macouria: Cannot find region: 00 -- skipping GH: Mpraeso: Cannot find region: 00 -- skipping GH: Kpandae: Cannot find region: 00 -- skipping GH: Kibi: Cannot find region: 00 -- skipping GH: Kete Krachi: Cannot find region: 00 -- skipping GH: Duayaw Nkwanta: Cannot find region: 00 -- skipping

Some says region other subregions. What could be wrong and how to fix it?

[cguethle]

I have found if you are unsuccessful on an --import=all at any point, then you have to blow away your installed django-cities module (pip/buildout/etc), reinstall, and run it again. I haven't looked at the exact issues there, but it seems to suffer from some state management issues if aborted/fails early.

Not sure if that helps you. My only attempts were complete import using cities 15000.

[cguethle]

I take it back some, I just ran a fresh import and get a lot of 'skipping' WARNINGs for subregion 00 as well. shrug

[coderholic]

--force should blow away any state.

Some warnings are expected, the matching isn't perfect. @x-warrior can you quantify how many you're seeing, and how many cities etc are successfully imported?

[matheusbrat]

When importing country, region, subregion and city which with a command the Stats is: countries: 250 imported, 0 warnings region: 3889 imported, 0 warnings subregion: 39381 imported, 8 warnings cities: 45772 imported, 1304 warning

When importing country, region, subregion and city with one command (python manage.py cities --import=country,region,subregion,city --force) the Stats is:

countries: 250 imported, 0 warnings region: 3889 imported, 0 warnings subregion: 39381 imported, 8 warnings cities: 45772 imported, 1305 warning

So the problem is not the difference between the commands.

Is this the correct number of warning that I should get? I thought it was too much yesterday, but maybe it is reasonable.

Thanks

[coderholic]

Yeah those numbers look fine.

[matheusbrat]

Sweet, sorry about the bad report, I was learning how it works.

I'm closing this. Thanks!