[x] I have verified that I am using a GIS-enabled database, such as PostGIS or Spatialite.
[x] I have verified that that issue exists against the master branch of django-cities.
[x] I have searched for similar issues in both open and closed tickets and cannot find a duplicate.
[x] I have reduced the issue to the simplest possible case.
[x] I have included a failing test as a pull request. (If you are unable to do so we can still accept the issue.)
Steps to reproduce
python manage.py cities --import=all
Expected behavior
Import of all data.
Actual behavior
Fails at the index build step.
python manage.py cities --import=all
Importing countries: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 250/250 [00:00<00:00, 505.84it/s]
Importing country neighbours: 100%|███████████████████████████████████████████████████████████████████████████████████████████████| 250/250 [00:00<00:00, 389.58it/s]
Building country index: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████| 250/250 [00:00<00:00, 26404.51it/s]
Importing regions: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 3972/3972 [00:07<00:00, 535.45it/s]
Building region index: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████| 3972/3972 [00:00<00:00, 18794.72it/s]
Importing subregions: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████| 45731/45731 [01:35<00:00, 480.91it/s]
Building region index: 0%| | 1/47993 [00:00<3:25:01, 3.90it/s]Traceback (most recent call last):
File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/backends/utils.py", line 85, in _execute
return self.cursor.execute(sql, params)
File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/backends/sqlite3/base.py", line 296, in execute
return Database.Cursor.execute(self, query, params)
sqlite3.OperationalError: too many SQL variables
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "manage.py", line 15, in <module>
execute_from_command_line(sys.argv)
File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
utility.execute()
File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/core/management/__init__.py", line 375, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/core/management/base.py", line 316, in run_from_argv
self.execute(*args, **cmd_options)
File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/core/management/base.py", line 353, in execute
output = self.handle(*args, **options)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/contextlib.py", line 74, in inner
return func(*args, **kwds)
File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/cities/management/commands/cities.py", line 160, in handle
func()
File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/cities/management/commands/cities.py", line 470, in import_city
self.build_region_index()
File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/cities/management/commands/cities.py", line 396, in build_region_index
desc="Building region index"):
File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/tqdm/_tqdm.py", line 979, in __iter__
for obj in iterable:
File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/models/query.py", line 268, in __iter__
self._fetch_all()
File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/models/query.py", line 1188, in _fetch_all
self._prefetch_related_objects()
File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/models/query.py", line 723, in _prefetch_related_objects
prefetch_related_objects(self._result_cache, *self._prefetch_related_lookups)
File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/models/query.py", line 1569, in prefetch_related_objects
obj_list, additional_lookups = prefetch_one_level(obj_list, prefetcher, lookup, level)
File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/models/query.py", line 1699, in prefetch_one_level
all_related_objects = list(rel_qs)
File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/models/query.py", line 268, in __iter__
self._fetch_all()
File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/models/query.py", line 1186, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/models/query.py", line 54, in __iter__
results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1065, in execute_sql
cursor.execute(sql, params)
File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/backends/utils.py", line 100, in execute
return super().execute(sql, params)
File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/backends/utils.py", line 68, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/backends/utils.py", line 85, in _execute
return self.cursor.execute(sql, params)
File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/utils.py", line 89, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/backends/utils.py", line 85, in _execute
return self.cursor.execute(sql, params)
File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/backends/sqlite3/base.py", line 296, in execute
return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: too many SQL variables
Checklist
masterbranch of django-cities.Steps to reproduce
python manage.py cities --import=allExpected behavior
Import of all data.
Actual behavior
Fails at the index build step.