mediagis / nominatim-docker

100% working container for Nominatim
Creative Commons Zero v1.0 Universal
1.08k stars 442 forks source link

Stuck with no progress on Update #146

Closed d3mon187 closed 3 years ago

d3mon187 commented 3 years ago

I'm trying to get a planet import to do an osmosis update, but it appears to freeze after processing nodes, ways, and relations. Looking at the database, the only active query is - COPY "place" FROM STDIN. According to my server stats I can see that the query is using tons of disk IO at 200-400MB a second. Still, I have let it run for over 2 days, and the osmosis import showed no updates on progress. I tried using the latest osm2pgsql version from master, as well as updating Nominatim from 3.5.1 to 3.5.2. Am I missing something? Thanks!

sudo docker exec -it nominatim sudo -u postgres ./src/build/utils/update.php --verbose --import-osmosis --osm2pgsql-cache 22000
Currently at sequence 2928 (17/09/2020 23:58:04) - 1 indexed
'/usr/local/bin/pyosmium-get-changes' --server 'https://planet.openstreetmap.org/replication/day' --outfile '/app/src/build/osmosischange.osc' --size 20 '--start-id' 2928
'/app/src/build/osm2pgsql/osm2pgsql' --hstore --latlong --append --slim '--number-processes' 1 --cache 22000 --output gazetteer --style '/app/src/settings/import-full.style' --database nominatim --port 5432 '/app/src/build/osmosischange.osc'
osm2pgsql version 1.2.0 (v3.5.2)

Allocating memory for dense node cache
Allocating dense node cache in one big chunk
Allocating memory for sparse node cache
Sharing dense sparse
Node-cache: cache=22000MB, maxblocks=352000*65536, allocation method=11
Mid: pgsql, cache=22000
Parsing gazetteer style file '/app/src/settings/import-full.style'.
Using projection SRS 4326 (Latlong)

Reading in file: /app/src/build/osmosischange.osc
Using XML parser.
Processing: Node(3588k 66.4k/s) Way(831k 16.62k/s) Relation(12342 726.0/s)  parse time: 121s
Node stats: total(3588852), max(7921640261) in 54s
Way stats: total(831818), max(849053464) in 50s
Relation stats: total(12342), max(11643024) in 17s
mtmail commented 3 years ago

Duplicate of https://github.com/osm-search/Nominatim/issues/1983 where the maintainer of Nominatim and osm2pgsql answered.

d3mon187 commented 3 years ago

The check_import_finished.php script at the end of init seemed to indicate that everything was ok, but after running it again it stated that indexes were missing. After running "--index --create-search-indices --create-country-names --index-noanalyse" two more times, the check_import_finished.php script finally said that all indices were good.