mediagis / nominatim-docker

100% working container for Nominatim
Creative Commons Zero v1.0 Universal
1.03k stars 434 forks source link

Problem with updating the database #512

Open KarolPlewka opened 7 months ago

KarolPlewka commented 7 months ago

Describe the bug When I try to update the database as it's described in readme

docker exec -it nominatim sudo -u nominatim nominatim replicati on --project-dir /nominatim

I've got following error:

`2023-12-15 11:03:18: Using project directory: /nominatim 2023-12-15 11:03:22: Starting indexing rank (0 to 4) using 1 threads 2023-12-15 11:03:22: Starting rank 1 (using batch size 1) 2023-12-15 11:03:22: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 1

2023-12-15 11:03:22: Starting rank 2 (using batch size 1) 2023-12-15 11:03:22: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 2

2023-12-15 11:03:22: Starting rank 3 (using batch size 1) 2023-12-15 11:03:22: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 3

2023-12-15 11:03:22: Starting rank 4 (using batch size 1) 2023-12-15 11:03:22: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 4

2023-12-15 11:03:22: Starting indexing boundaries using 1 threads 2023-12-15 11:03:22: Starting boundaries rank 4 (using batch size 1) 2023-12-15 11:07:32: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 4

2023-12-15 11:07:32: Starting boundaries rank 5 (using batch size 1) 2023-12-15 11:07:34: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 5

2023-12-15 11:07:34: Starting boundaries rank 6 (using batch size 1) 2023-12-15 11:07:34: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 6

2023-12-15 11:07:34: Starting boundaries rank 7 (using batch size 1) 2023-12-15 11:07:34: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 7

2023-12-15 11:07:34: Starting boundaries rank 8 (using batch size 1) 2023-12-15 11:07:54: Done 1/1 in 19 @ 0.052 per second - FINISHED boundaries ran k 8

2023-12-15 11:07:54: Starting boundaries rank 9 (using batch size 1) 2023-12-15 11:07:54: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 9

2023-12-15 11:07:54: Starting boundaries rank 10 (using batch size 1) 2023-12-15 11:07:54: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 10

2023-12-15 11:07:54: Starting boundaries rank 11 (using batch size 1) 2023-12-15 11:07:54: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 11

2023-12-15 11:07:54: Starting boundaries rank 12 (using batch size 1) 2023-12-15 11:07:56: Done 10/10 in 1 @ 5.226 per second - FINISHED boundaries ra nk 12

2023-12-15 11:07:56: Starting boundaries rank 13 (using batch size 1) 2023-12-15 11:07:56: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 13

2023-12-15 11:07:56: Starting boundaries rank 14 (using batch size 1) 2023-12-15 11:07:56: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 14

2023-12-15 11:07:56: Starting boundaries rank 15 (using batch size 1) 2023-12-15 11:07:56: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 15

2023-12-15 11:07:56: Starting boundaries rank 16 (using batch size 1) 2023-12-15 11:07:58: Done 30 in 2 @ 13.975 per second - boundaries rank 16 ETA ( seconds): 3.79 2023-12-15 11:07:59: Done 43 in 2 @ 15.975 per second - boundaries rank 16 ETA ( seconds): 2.50 2023-12-15 11:07:59: Done 58 in 3 @ 18.211 per second - boundaries rank 16 ETA ( seconds): 1.37 2023-12-15 11:08:00: Done 76 in 3 @ 19.508 per second - boundaries rank 16 ETA ( seconds): 0.36 2023-12-15 11:08:00: Done 83/83 in 4 @ 20.342 per second - FINISHED boundaries r ank 16

2023-12-15 11:08:00: Starting boundaries rank 17 (using batch size 1) 2023-12-15 11:08:00: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 17

2023-12-15 11:08:00: Starting boundaries rank 18 (using batch size 1) 2023-12-15 11:08:00: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 18

2023-12-15 11:08:00: Starting boundaries rank 19 (using batch size 1) 2023-12-15 11:08:00: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 19

2023-12-15 11:08:00: Starting boundaries rank 20 (using batch size 1) 2023-12-15 11:08:00: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 20

2023-12-15 11:08:00: Starting boundaries rank 21 (using batch size 1) 2023-12-15 11:08:00: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 21

2023-12-15 11:08:00: Starting boundaries rank 22 (using batch size 1) 2023-12-15 11:08:00: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 22

2023-12-15 11:08:00: Starting boundaries rank 23 (using batch size 1) 2023-12-15 11:08:01: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 23

2023-12-15 11:08:01: Starting boundaries rank 24 (using batch size 1) 2023-12-15 11:08:01: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 24

2023-12-15 11:08:01: Starting boundaries rank 25 (using batch size 1) 2023-12-15 11:08:01: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 25

2023-12-15 11:08:01: Starting indexing rank (5 to 25) using 1 threads 2023-12-15 11:08:01: Starting rank 5 (using batch size 1) 2023-12-15 11:08:01: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 5

2023-12-15 11:08:01: Starting rank 6 (using batch size 1) 2023-12-15 11:08:01: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 6

2023-12-15 11:08:01: Starting rank 7 (using batch size 1) 2023-12-15 11:08:01: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 7

2023-12-15 11:08:01: Starting rank 8 (using batch size 1) 2023-12-15 11:08:01: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 8

2023-12-15 11:08:01: Starting rank 9 (using batch size 1) 2023-12-15 11:08:01: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 9

2023-12-15 11:08:01: Starting rank 10 (using batch size 1) 2023-12-15 11:08:01: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 10

2023-12-15 11:08:01: Starting rank 11 (using batch size 1) 2023-12-15 11:08:01: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 11

2023-12-15 11:08:01: Starting rank 12 (using batch size 1) 2023-12-15 11:08:01: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 12

2023-12-15 11:08:01: Starting rank 13 (using batch size 1) 2023-12-15 11:08:01: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 13

2023-12-15 11:08:01: Starting rank 14 (using batch size 1) 2023-12-15 11:08:01: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 14

2023-12-15 11:08:01: Starting rank 15 (using batch size 1) 2023-12-15 11:08:01: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 15

2023-12-15 11:08:01: Starting rank 16 (using batch size 1) 2023-12-15 11:08:04: Done 30 in 3 @ 9.773 per second - rank 16 ETA (seconds): 1. 43 2023-12-15 11:08:07: Done 39 in 6 @ 6.188 per second - rank 16 ETA (seconds): 0. 81 2023-12-15 11:08:08: Done 44/44 in 6 @ 6.471 per second - FINISHED rank 16

2023-12-15 11:08:08: Starting rank 17 (using batch size 1) 2023-12-15 11:08:08: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 17

2023-12-15 11:08:08: Starting rank 18 (using batch size 1) 2023-12-15 11:08:09: Done 42/42 in 1 @ 28.632 per second - FINISHED rank 18

2023-12-15 11:08:09: Starting rank 19 (using batch size 1) 2023-12-15 11:08:09: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 19

2023-12-15 11:08:09: Starting rank 20 (using batch size 1) 2023-12-15 11:08:12: Done 30 in 2 @ 12.033 per second - rank 20 ETA (seconds): 4 .57 2023-12-15 11:08:12: Done 42 in 2 @ 15.544 per second - rank 20 ETA (seconds): 2 .77 2023-12-15 11:08:13: Done 57 in 3 @ 16.251 per second - rank 20 ETA (seconds): 1 .72 2023-12-15 11:08:13: Done 73 in 4 @ 18.217 per second - rank 20 ETA (seconds): 0 .66 2023-12-15 11:08:14: Done 85/85 in 4 @ 18.828 per second - FINISHED rank 20

2023-12-15 11:08:14: Starting rank 21 (using batch size 1) 2023-12-15 11:08:14: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 21

2023-12-15 11:08:14: Starting rank 22 (using batch size 1) 2023-12-15 11:08:16: Done 36/36 in 2 @ 16.190 per second - FINISHED rank 22

2023-12-15 11:08:16: Starting rank 23 (using batch size 1) 2023-12-15 11:08:16: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 23

2023-12-15 11:08:16: Starting rank 24 (using batch size 1) 2023-12-15 11:08:16: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 24

2023-12-15 11:08:16: Starting rank 25 (using batch size 1) 2023-12-15 11:08:17: Done 6/6 in 0 @ 11.812 per second - FINISHED rank 25

2023-12-15 11:08:17: Starting indexing rank (26 to 30) using 1 threads 2023-12-15 11:08:17: Starting rank 26 (using batch size 1) 2023-12-15 11:08:17: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 26

2023-12-15 11:08:17: Starting rank 27 (using batch size 1) 2023-12-15 11:08:17: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 27

2023-12-15 11:08:17: Starting rank 28 (using batch size 1) 2023-12-15 11:08:17: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 28

2023-12-15 11:08:17: Starting rank 29 (using batch size 1) 2023-12-15 11:08:17: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 29

2023-12-15 11:08:17: Starting rank 30 (using batch size 20) 2023-12-15 11:08:17: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 30

2023-12-15 11:08:17: Starting rank 0 (using batch size 1) 2023-12-15 11:08:19: Done 190 in 2 @ 91.970 per second - rank 0 ETA (seconds): 4 0.00 2023-12-15 11:08:19: Done 281 in 2 @ 101.934 per second - rank 0 ETA (seconds): 35.20 2023-12-15 11:08:20: Done 382 in 3 @ 113.848 per second - rank 0 ETA (seconds): 30.63 2023-12-15 11:08:21: Done 495 in 3 @ 126.950 per second - rank 0 ETA (seconds): 26.58 2023-12-15 11:08:21: Done 621 in 4 @ 141.106 per second - rank 0 ETA (seconds): 23.02 2023-12-15 11:08:22: Done 762 in 4 @ 157.434 per second - rank 0 ETA (seconds): 19.74 2023-12-15 11:08:22: Done 919 in 5 @ 178.687 per second - rank 0 ETA (seconds): 16.51 2023-12-15 11:08:22: Done 1097 in 5 @ 197.430 per second - rank 0 ETA (seconds): 14.04 2023-12-15 11:08:23: Done 1294 in 5 @ 217.052 per second - rank 0 ETA (seconds): 11.86 2023-12-15 11:08:23: Done 1511 in 6 @ 230.583 per second - rank 0 ETA (seconds): 10.23 2023-12-15 11:08:24: Done 1741 in 7 @ 236.004 per second - rank 0 ETA (seconds): 9.02 2023-12-15 11:08:25: Done 1977 in 8 @ 243.116 per second - rank 0 ETA (seconds): 7.78 2023-12-15 11:08:25: Done 2220 in 8 @ 252.347 per second - rank 0 ETA (seconds): 6.53 2023-12-15 11:08:27: Done 2472 in 10 @ 243.733 per second - rank 0 ETA (seconds) : 5.73 2023-12-15 11:08:28: Done 2715 in 10 @ 250.063 per second - rank 0 ETA (seconds) : 4.61 2023-12-15 11:08:28: Done 2965 in 11 @ 260.127 per second - rank 0 ETA (seconds) : 3.48 2023-12-15 11:08:29: Done 3225 in 12 @ 259.552 per second - rank 0 ETA (seconds) : 2.48 2023-12-15 11:08:31: Done 3484 in 14 @ 242.811 per second - rank 0 ETA (seconds) : 1.59 2023-12-15 11:08:34: Done 3726 in 16 @ 221.532 per second - rank 0 ETA (seconds) : 0.65 2023-12-15 11:08:35: Done 3869/3869 in 18 @ 214.391 per second - FINISHED rank 0

2023-12-15 11:08:35: Starting interpolation lines (location_property_osmline) (u sing batch size 20) Traceback (most recent call last): File "/usr/local/bin/nominatim", line 14, in exit(cli.nominatim(module_dir='/usr/local/lib/nominatim/module', File "/usr/local/lib/nominatim/lib-python/nominatim/cli.py", line 264, in nomi natim return parser.run(**kwargs) File "/usr/local/lib/nominatim/lib-python/nominatim/cli.py", line 126, in run return args.command.run(args) File "/usr/local/lib/nominatim/lib-python/nominatim/clicmd/replication.py", li ne 196, in run self._update(args) File "/usr/local/lib/nominatim/lib-python/nominatim/clicmd/replication.py", li ne 161, in _update indexer.index_full(analyse=False) File "/usr/local/lib/nominatim/lib-python/nominatim/indexer/indexer.py", line 140, in index_full if self.index_by_rank(26, 30) > 1000: File "/usr/local/lib/nominatim/lib-python/nominatim/indexer/indexer.py", line 178, in index_by_rank total += self._index(runners.InterpolationRunner(analyzer), 20) File "/usr/local/lib/nominatim/lib-python/nominatim/indexer/indexer.py", line 234, in _index runner.index_places(pool.next_free_worker(), part) File "/usr/local/lib/nominatim/lib-python/nominatim/db/async_connection.py", l ine 201, in next_free_worker return next(self.free_workers) File "/usr/local/lib/nominatim/lib-python/nominatim/db/async_connection.py", l ine 209, in _yield_free_worker if thread.is_done(): File "/usr/local/lib/nominatim/lib-python/nominatim/db/async_connection.py", l ine 159, in is_done if self.conn.poll() == psycopg2.extensions.POLLOK: psycopg2.errors.InternalError: Splitting of Point geometries is unsupported CONTEXT: PL/pgSQL function osmline_update() line 77 at assignment

` Container was run few months ago with following parameters:

docker run -it --shm-size=1g -e PBF_URL=https://ftp5.gwdg.de/pub/misc/openstreetmap/planet.openstreetmap.org/pbf/planet-latest.osm.pbf -e REPLICATION_URL=https://ftp5.gwdg.de/pub/misc/openstreetmap/planet.openstreetmap.org/replication/day/ -e IMPORT_STYLE=extratags -e UPDATE_MODE=continuous -v nominatim-data:/datadrive -p 8080:8080 --name nominatim mediagis/nominatim:4.2 To Reproduce docker exec -it nominatim sudo -u nominatim nominatim replicati on --project-dir /nominatim

Expected behavior Update should be completed without any error

Desktop / Server (please complete the following information):