Describe the bug
After rank imports and database check, the import process failed when warming up the database. I think this is due to https://github.com/osm-search/Nominatim/pull/3172 which sets a default query timeout of 10 seconds.
To Reproduce
Start a planet import and wait for it
Expected behavior
The import and warm up process should complete without errors
Screenshots & Logs
2023-09-30 05:50:36: Using project directory: /nominatim
2023-09-30 05:50:36: Checking database
Checking database connection ... OK
Checking for placex table ... OK
Checking for placex content ... OK
Checking that tokenizer works ... OK
Checking for wikipedia/wikidata data ... OK
Checking indexing status ... OK
Checking that database indexes are complete ... OK
Checking that all database indexes are valid ... OK
Checking TIGER external data table. ... OK
+ '[' https://ftp5.gwdg.de/pub/misc/openstreetmap/planet.openstreetmap.org/replication/day/ '!=' '' ']'
+ sudo -E -u nominatim nominatim replication --init
2023-09-30 05:50:42: Using project directory: /nominatim
2023-09-30 05:50:42: Initialising replication updates
2023-09-30 05:50:47: Updates initialised at sequence 4022 (2023-09-17 20:56:04+00:00)
2023-09-30 05:50:47: Create functions
+ '[' '' = true ']'
+ '[' '' = true ']'
+ sudo -H -E -u nominatim nominatim admin --warm
2023-09-30 05:50:48: Using project directory: /nominatim
2023-09-30 05:50:48: Warming database caches
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/sqlalchemy/ext/asyncio/engine.py", line 457, in execute
result = await greenlet_spawn(
File "/usr/lib/python3/dist-packages/sqlalchemy/util/_concurrency_py3k.py", line 136, in greenlet_spawn
result = context.switch(value)
File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1614, in _execute_20
return meth(self, args_10style, kwargs_10style, execution_options)
File "/usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection
return connection._execute_clauseelement(
File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1481, in _execute_clauseelement
ret = self._execute_context(
File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1845, in _execute_context
self._handle_dbapi_exception(
File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 2030, in _handle_dbapi_exception
util.raise_(exc_info[1], with_traceback=exc_info[2])
File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_
raise exception
File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context
self.dialect.do_execute(
File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute
cursor.execute(statement, parameters)
File "/usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 460, in execute
self._adapt_connection.await_(
File "/usr/lib/python3/dist-packages/sqlalchemy/util/_concurrency_py3k.py", line 76, in await_only
return current.driver.switch(awaitable)
File "/usr/lib/python3/dist-packages/sqlalchemy/util/_concurrency_py3k.py", line 129, in greenlet_spawn
value = await result
File "/usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 423, in _prepare_and_execute
self._rows = await prepared_stmt.fetch(*parameters)
File "/usr/lib/python3/dist-packages/asyncpg/prepared_stmt.py", line 176, in fetch
data = await self.__bind_execute(args, 0, timeout)
File "/usr/lib/python3/dist-packages/asyncpg/prepared_stmt.py", line 241, in __bind_execute
data, status, _ = await self.__do_execute(
File "/usr/lib/python3/dist-packages/asyncpg/prepared_stmt.py", line 230, in __do_execute
return await executor(protocol)
File "asyncpg/protocol/protocol.pyx", line 201, in bind_execute
asyncio.exceptions.CancelledError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.10/asyncio/tasks.py", line 456, in wait_for
return fut.result()
asyncio.exceptions.CancelledError
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/bin/nominatim", line 12, in <module>
exit(cli.nominatim(module_dir='/usr/local/lib/nominatim/module',
File "/usr/local/lib/nominatim/lib-python/nominatim/cli.py", line 225, in nominatim
return get_set_parser().run(**kwargs)
File "/usr/local/lib/nominatim/lib-python/nominatim/cli.py", line 121, in run
return args.command.run(args)
File "/usr/local/lib/nominatim/lib-python/nominatim/clicmd/admin.py", line 60, in run
return self._warm(args)
File "/usr/local/lib/nominatim/lib-python/nominatim/clicmd/admin.py", line 106, in _warm
api.search(word)
File "/usr/local/lib/nominatim/lib-python/nominatim/api/core.py", line 709, in search
return self._loop.run_until_complete(
File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
return future.result()
File "/usr/local/lib/nominatim/lib-python/nominatim/api/core.py", line 223, in search
return await geocoder.lookup(phrases)
File "/usr/local/lib/nominatim/lib-python/nominatim/api/search/geocoder.py", line 151, in lookup
results = await self.execute_searches(query, searches[:50])
File "/usr/local/lib/nominatim/lib-python/nominatim/api/search/geocoder.py", line 86, in execute_searches
for result in await search.lookup(self.conn, self.params):
File "/usr/local/lib/nominatim/lib-python/nominatim/api/search/db_searches.py", line 734, in lookup
for row in await conn.execute(sql, _details_to_bind_params(details)):
File "/usr/local/lib/nominatim/lib-python/nominatim/api/connection.py", line 63, in execute
return await asyncio.wait_for(self.connection.execute(sql, params), self.query_timeout)
File "/usr/lib/python3.10/asyncio/tasks.py", line 458, in wait_for
raise exceptions.TimeoutError() from exc
asyncio.exceptions.TimeoutError
Desktop / Server (please complete the following information):
OS & Version: Arch Linux kernel 6.5.4-arch2-1
Docker Version: 24.0.5
Nominatim Version: 4.3.0
Additional context
I will try another planet import with NOMINATIM_QUERY_TIMEOUT set to 600. If that fails, I will disable it entirely while warming up and re-set it afterwards as this mechanism is a good idea. If everything is works out, I will submit a PR with said fixes.
Describe the bug After rank imports and database check, the import process failed when warming up the database. I think this is due to https://github.com/osm-search/Nominatim/pull/3172 which sets a default query timeout of 10 seconds.
To Reproduce Start a planet import and wait for it
Expected behavior The import and warm up process should complete without errors
Screenshots & Logs
Desktop / Server (please complete the following information):
Arch Linux kernel 6.5.4-arch2-1
24.0.5
4.3.0
Additional context I will try another planet import with
NOMINATIM_QUERY_TIMEOUT
set to 600. If that fails, I will disable it entirely while warming up and re-set it afterwards as this mechanism is a good idea. If everything is works out, I will submit a PR with said fixes.