search

robjuz / helm-charts

https://robjuz.github.io/helm-charts/index.yaml
34 stars 30 forks source link

Failed to connect to the database #23

Open mauriciocirelli opened 1 year ago

mauriciocirelli commented 1 year ago

Dear,

I am trying to deploy this helm chart in a cluster and the init process is failing with the following logs:

Defaulted container "nominatim" out of: nominatim, download-wikipedia (init), download-pbf (init)
2022-09-18 23:39:22: Using project directory: /nominatim
2022-09-18 23:39:24: Creating database
createdb: error: connection to server at "nominatim-postgresql" (10.10.183.116), port 5432 failed: FATAL:  password authentication failed for user "postgres"

I have just configured the size and StorageClass of the database parameters. Everything else is the default:

postgresql:
  primary:
      persistence: 
        enabled: true
        storageClass: my-storage-class
        accessModes: 
          - ReadWriteMany
        size: "150Gi"
      extendedConfiguration: |
        shared_buffers = 2GB
        maintenance_work_mem = 10GB
        autovacuum_work_mem = 2GB
        work_mem = 50MB
        effective_cache_size = 24GB
        synchronous_commit = off
        max_wal_size = 1GB
        checkpoint_timeout = 10min
        checkpoint_completion_target = 0.9
        fsync = off
        full_page_writes = off

It seems that the database password is wrong somewhere in the chart or on its dependencies.

robjuz commented 1 year ago

Thanks for reporting. I will take a look.

robjuz commented 1 year ago

Hallo @mauriciocirelli I added a init container to ensure the database is ready for connections.

Please give it a try now

mauriciocirelli commented 1 year ago

Hi @robjuz!

The problem persists:

Defaulted container "nominatim" out of: nominatim, wait-for-db (init), download-wikipedia (init), download-pbf (init)
2022-09-23 12:27:13: Using project directory: /nominatim
2022-09-23 12:27:16: Creating database
createdb: error: connection to server at "nominatim-postgresql" (10.10.195.98), port 5432 failed: FATAL:  password authentication failed for user "postgres"
2022-09-23 12:27:16: FATAL: Creating new database failed.

It seems that waiting for the db is not the issue... It seems that nominatim DB credentials are inconsistent with the ones used by Bitnami's postgres chart.

robjuz commented 1 year ago

Can you show me your values file?

robjuz commented 1 year ago

Have you tried with a fresh install and a small extract like Monaco?

mauriciocirelli commented 1 year ago

Yes, I am trying a fresh install, and I am trying to extract South-America only, which is the region we work on. We have a nominatim deployment in a VM, so I know 150G is enough for this region.

nominatimInitialize:
  enabled: true
  pbfUrl: "https://download.geofabrik.de/south-america-latest.osm.pbf"
  threads: 8
  importWikipedia: true
  importGB_Postcode: false
  importUS_Postcode: false
  importStyle: full  

nominatimReplications:
  enabled: true
  replicationUrl: "https://download.geofabrik.de/south-america-updates"

replicaCount: 1

postgresql:
  primary:
      persistence: 
        enabled: true
        storageClass: azurefile-csi
        accessModes: 
          - ReadWriteMany
        size: "150Gi"
      extendedConfiguration: |
        shared_buffers = 2GB
        maintenance_work_mem = 10GB
        autovacuum_work_mem = 2GB
        work_mem = 50MB
        effective_cache_size = 24GB
        synchronous_commit = off
        max_wal_size = 1GB
        checkpoint_timeout = 10min
        checkpoint_completion_target = 0.9
        fsync = off
        full_page_writes = off
robjuz commented 1 year ago

Maybe this is a stupid question, but are you using the last version of the chart?

mauriciocirelli commented 1 year ago

Not stupid at all. The first attempt today I had forgotten to run helm repo update hehehe..

helm search repo
robjuz/nominatim        3.0.2           4.1                     A Helm chart for Kubernetes
mauriciocirelli commented 1 year ago

Perhaps bitnami has changed the way it creates database credentials?

robjuz commented 1 year ago

I tested the chart on my cluster after Updated it. It was working fine.

I will try to debug it.

robjuz commented 1 year ago

Hello @mauriciocirelli

I just tested this chart on a fresh environment

Create a server on Hetzner

hcloud server create --image debian-11 --type cpx11 --name nominatim

Install apparmor

apt update
apt install apparmor

Install k3s

curl https://get.k3s.io | - sh

Install helm

curl  https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash -

Install nominatim

cat << EOF > values.yaml
nominatimInitialize:
  enabled: true
  pbfUrl: https://download.geofabrik.de/europe/monaco-latest.osm.pbf
  importWikipedia: false
  threads: 1

postgresql:
  enabled: true
  primary:
    persistence:
      size: 1Gi
EOF
helm repo add robjuz https://robjuz.github.io/helm-charts
helm install nominatim robjuz/nominatim -f values.yaml

Check the logs

kubectl logs nominatim-init-7647n --all-containers
wait-for-it.sh: waiting 15 seconds for nominatim-postgresql:5432
wait-for-it.sh: nominatim-postgresql:5432 is available after 0 seconds
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  475k  100  475k    0     0  2176k      0 --:--:-- --:--:-- --:--:-- 2170k
2022-09-29 06:45:34: Using project directory: /nominatim
2022-09-29 06:45:37: Creating database
2022-09-29 06:45:37: Setting up country tables
2022-09-29 06:45:39: Importing OSM data file
2022-09-29 06:45:39  osm2pgsql version 1.6.0
2022-09-29 06:45:39  Database version: 14.4
2022-09-29 06:45:39  PostGIS version: 3.1
2022-09-29 06:45:39  Parsing gazetteer style file '/usr/local/etc/nominatim/import-full.style'.
2022-09-29 06:45:40  Reading input files done in 1s.                                      
2022-09-29 06:45:40    Processed 28083 nodes in 0s - 28k/s
2022-09-29 06:45:40    Processed 4549 ways in 1s - 5k/s
2022-09-29 06:45:40    Processed 262 relations in 0s - 262/s
2022-09-29 06:45:40  Done postprocessing on table 'planet_osm_nodes' in 0s
2022-09-29 06:45:40  Done postprocessing on table 'planet_osm_ways' in 0s
2022-09-29 06:45:40  Done postprocessing on table 'planet_osm_rels' in 0s
2022-09-29 06:45:40  osm2pgsql took 1s overall.
2022-09-29 06:45:40: Create functions (1st pass)
2022-09-29 06:45:40: Create tables
2022-09-29 06:45:40: Create functions (2nd pass)
2022-09-29 06:45:40: Create table triggers
2022-09-29 06:45:40: Create partition tables
2022-09-29 06:45:43: Create functions (3rd pass)
2022-09-29 06:45:43: Importing wikipedia importance data
2022-09-29 06:45:43: Wikipedia importance dump file not found. Will be using default importances.
2022-09-29 06:45:43: Initialise tables
2022-09-29 06:45:44: Load data into placex table
2022-09-29 06:45:46: Setting up tokenizer
2022-09-29 06:45:46: Calculate postcodes
2022-09-29 06:45:46: Indexing places
2022-09-29 06:45:46: Starting indexing rank (0 to 4) using 1 threads
2022-09-29 06:45:46: Starting rank 1 (using batch size 1)
2022-09-29 06:45:46: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 1

2022-09-29 06:45:46: Starting rank 2 (using batch size 1)
2022-09-29 06:45:47: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 2

2022-09-29 06:45:47: Starting rank 3 (using batch size 1)
2022-09-29 06:45:47: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 3

2022-09-29 06:45:47: Starting rank 4 (using batch size 1)
2022-09-29 06:45:47: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 4

2022-09-29 06:45:47: Starting indexing boundaries using 1 threads
2022-09-29 06:45:47: Starting boundaries rank 4 (using batch size 1)
2022-09-29 06:45:47: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 4

2022-09-29 06:45:47: Starting boundaries rank 5 (using batch size 1)
2022-09-29 06:45:47: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 5

2022-09-29 06:45:47: Starting boundaries rank 6 (using batch size 1)
2022-09-29 06:45:47: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 6

2022-09-29 06:45:47: Starting boundaries rank 7 (using batch size 1)
2022-09-29 06:45:47: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 7

2022-09-29 06:45:47: Starting boundaries rank 8 (using batch size 1)
2022-09-29 06:45:47: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 8

2022-09-29 06:45:47: Starting boundaries rank 9 (using batch size 1)
2022-09-29 06:45:47: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 9

2022-09-29 06:45:47: Starting boundaries rank 10 (using batch size 1)
2022-09-29 06:45:47: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 10

2022-09-29 06:45:47: Starting boundaries rank 11 (using batch size 1)
2022-09-29 06:45:47: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 11

2022-09-29 06:45:47: Starting boundaries rank 12 (using batch size 1)
2022-09-29 06:45:47: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 12

2022-09-29 06:45:47: Starting boundaries rank 13 (using batch size 1)
2022-09-29 06:45:47: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 13

2022-09-29 06:45:47: Starting boundaries rank 14 (using batch size 1)
2022-09-29 06:45:47: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 14

2022-09-29 06:45:47: Starting boundaries rank 15 (using batch size 1)
2022-09-29 06:45:47: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 15

2022-09-29 06:45:47: Starting boundaries rank 16 (using batch size 1)
2022-09-29 06:45:47: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 16

2022-09-29 06:45:47: Starting boundaries rank 17 (using batch size 1)
2022-09-29 06:45:47: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 17

2022-09-29 06:45:47: Starting boundaries rank 18 (using batch size 1)
2022-09-29 06:45:47: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 18

2022-09-29 06:45:47: Starting boundaries rank 19 (using batch size 1)
2022-09-29 06:45:47: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 19

2022-09-29 06:45:47: Starting boundaries rank 20 (using batch size 1)
2022-09-29 06:45:48: Done 6/6 in 0 @ 40.462 per second - FINISHED boundaries rank 20

2022-09-29 06:45:48: Starting boundaries rank 21 (using batch size 1)
2022-09-29 06:45:48: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 21

2022-09-29 06:45:48: Starting boundaries rank 22 (using batch size 1)
2022-09-29 06:45:48: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 22

2022-09-29 06:45:48: Starting boundaries rank 23 (using batch size 1)
2022-09-29 06:45:48: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 23

2022-09-29 06:45:48: Starting boundaries rank 24 (using batch size 1)
2022-09-29 06:45:48: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 24

2022-09-29 06:45:48: Starting boundaries rank 25 (using batch size 1)
2022-09-29 06:45:48: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 25

2022-09-29 06:45:48: Starting indexing rank (5 to 25) using 1 threads
2022-09-29 06:45:48: Starting rank 5 (using batch size 1)
2022-09-29 06:45:48: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 5

2022-09-29 06:45:48: Starting rank 6 (using batch size 1)
2022-09-29 06:45:48: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 6

2022-09-29 06:45:48: Starting rank 7 (using batch size 1)
2022-09-29 06:45:48: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 7

2022-09-29 06:45:48: Starting rank 8 (using batch size 1)
2022-09-29 06:45:48: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 8

2022-09-29 06:45:48: Starting rank 9 (using batch size 1)
2022-09-29 06:45:48: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 9

2022-09-29 06:45:48: Starting rank 10 (using batch size 1)
2022-09-29 06:45:48: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 10

2022-09-29 06:45:48: Starting rank 11 (using batch size 1)
2022-09-29 06:45:48: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 11

2022-09-29 06:45:48: Starting rank 12 (using batch size 1)
2022-09-29 06:45:48: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 12

2022-09-29 06:45:48: Starting rank 13 (using batch size 1)
2022-09-29 06:45:48: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 13

2022-09-29 06:45:48: Starting rank 14 (using batch size 1)
2022-09-29 06:45:48: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 14

2022-09-29 06:45:48: Starting rank 15 (using batch size 1)
2022-09-29 06:45:48: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 15

2022-09-29 06:45:48: Starting rank 16 (using batch size 1)
2022-09-29 06:45:49: Done 1/1 in 0 @ 3.303 per second - FINISHED rank 16

2022-09-29 06:45:49: Starting rank 17 (using batch size 1)
2022-09-29 06:45:49: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 17

2022-09-29 06:45:49: Starting rank 18 (using batch size 1)
2022-09-29 06:45:49: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 18

2022-09-29 06:45:49: Starting rank 19 (using batch size 1)
2022-09-29 06:45:49: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 19

2022-09-29 06:45:49: Starting rank 20 (using batch size 1)
2022-09-29 06:45:49: Done 9/9 in 0 @ 68.306 per second - FINISHED rank 20

2022-09-29 06:45:49: Starting rank 21 (using batch size 1)
2022-09-29 06:45:49: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 21

2022-09-29 06:45:49: Starting rank 22 (using batch size 1)
2022-09-29 06:45:49: Done 4/4 in 0 @ 29.749 per second - FINISHED rank 22

2022-09-29 06:45:49: Starting rank 23 (using batch size 1)
2022-09-29 06:45:49: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 23

2022-09-29 06:45:49: Starting rank 24 (using batch size 1)
2022-09-29 06:45:49: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 24

2022-09-29 06:45:49: Starting rank 25 (using batch size 1)
2022-09-29 06:45:49: Done 6/6 in 0 @ 44.225 per second - FINISHED rank 25

2022-09-29 06:45:50: Starting indexing rank (26 to 30) using 1 threads
2022-09-29 06:45:50: Starting rank 26 (using batch size 1)
2022-09-29 06:45:52: Done 711/711 in 1 @ 398.493 per second - FINISHED rank 26

2022-09-29 06:45:52: Starting rank 27 (using batch size 1)
2022-09-29 06:45:52: Done 180/180 in 0 @ 318.565 per second - FINISHED rank 27

2022-09-29 06:45:52: Starting rank 28 (using batch size 1)
2022-09-29 06:45:52: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 28

2022-09-29 06:45:52: Starting rank 29 (using batch size 1)
2022-09-29 06:45:52: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 29

2022-09-29 06:45:52: Starting rank 30 (using batch size 20)
2022-09-29 06:45:54: Done 1100 in 2 @ 545.309 per second - rank 30 ETA (seconds): 2.28
2022-09-29 06:45:56: Done 1640 in 3 @ 529.050 per second - rank 30 ETA (seconds): 1.33
2022-09-29 06:45:56: Done 2180 in 4 @ 540.888 per second - rank 30 ETA (seconds): 0.30
2022-09-29 06:45:57: Done 2341/2341 in 4 @ 543.736 per second - FINISHED rank 30

2022-09-29 06:45:57: Starting rank 0 (using batch size 1)
2022-09-29 06:45:57: Done 28/28 in 0 @ 158.801 per second - FINISHED rank 0

2022-09-29 06:45:57: Starting interpolation lines (location_property_osmline) (using batch size 20)
2022-09-29 06:45:57: Done 0/0 in 0 @ 0.000 per second - FINISHED interpolation lines (location_property_osmline)

2022-09-29 06:45:58: Starting indexing postcodes using 1 threads
2022-09-29 06:45:58: Starting postcodes (location_postcode) (using batch size 20)
2022-09-29 06:45:58: Done 4/4 in 0 @ 61.620 per second - FINISHED postcodes (location_postcode)

2022-09-29 06:45:58: Post-process tables
2022-09-29 06:45:59: Create search index for default country names.
2022-09-29 06:46:00: Recompute word counts
2022-09-29 06:46:00: Setup website at /nominatim/website
..

2022-09-29 06:46:01: Using project directory: /nominatim
2022-09-29 06:46:04: Starting indexing boundaries using 1 threads
2022-09-29 06:46:04: Starting boundaries rank 4 (using batch size 1)
2022-09-29 06:46:04: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 4

2022-09-29 06:46:04: Starting boundaries rank 5 (using batch size 1)
2022-09-29 06:46:04: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 5

2022-09-29 06:46:04: Starting boundaries rank 6 (using batch size 1)
2022-09-29 06:46:04: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 6

2022-09-29 06:46:04: Starting boundaries rank 7 (using batch size 1)
2022-09-29 06:46:04: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 7

2022-09-29 06:46:04: Starting boundaries rank 8 (using batch size 1)
2022-09-29 06:46:04: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 8

2022-09-29 06:46:04: Starting boundaries rank 9 (using batch size 1)
2022-09-29 06:46:04: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 9

2022-09-29 06:46:04: Starting boundaries rank 10 (using batch size 1)
2022-09-29 06:46:04: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 10

2022-09-29 06:46:04: Starting boundaries rank 11 (using batch size 1)
2022-09-29 06:46:04: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 11

2022-09-29 06:46:04: Starting boundaries rank 12 (using batch size 1)
2022-09-29 06:46:04: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 12

2022-09-29 06:46:04: Starting boundaries rank 13 (using batch size 1)
2022-09-29 06:46:04: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 13

2022-09-29 06:46:04: Starting boundaries rank 14 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 14

2022-09-29 06:46:05: Starting boundaries rank 15 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 15

2022-09-29 06:46:05: Starting boundaries rank 16 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 16

2022-09-29 06:46:05: Starting boundaries rank 17 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 17

2022-09-29 06:46:05: Starting boundaries rank 18 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 18

2022-09-29 06:46:05: Starting boundaries rank 19 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 19

2022-09-29 06:46:05: Starting boundaries rank 20 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 20

2022-09-29 06:46:05: Starting boundaries rank 21 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 21

2022-09-29 06:46:05: Starting boundaries rank 22 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 22

2022-09-29 06:46:05: Starting boundaries rank 23 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 23

2022-09-29 06:46:05: Starting boundaries rank 24 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 24

2022-09-29 06:46:05: Starting boundaries rank 25 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 25

2022-09-29 06:46:05: Starting indexing rank (0 to 30) using 1 threads
2022-09-29 06:46:05: Starting rank 1 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 1

2022-09-29 06:46:05: Starting rank 2 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 2

2022-09-29 06:46:05: Starting rank 3 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 3

2022-09-29 06:46:05: Starting rank 4 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 4

2022-09-29 06:46:05: Starting rank 5 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 5

2022-09-29 06:46:05: Starting rank 6 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 6

2022-09-29 06:46:05: Starting rank 7 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 7

2022-09-29 06:46:05: Starting rank 8 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 8

2022-09-29 06:46:05: Starting rank 9 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 9

2022-09-29 06:46:05: Starting rank 10 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 10

2022-09-29 06:46:05: Starting rank 11 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 11

2022-09-29 06:46:05: Starting rank 12 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 12

2022-09-29 06:46:05: Starting rank 13 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 13

2022-09-29 06:46:05: Starting rank 14 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 14

2022-09-29 06:46:05: Starting rank 15 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 15

2022-09-29 06:46:05: Starting rank 16 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 16

2022-09-29 06:46:05: Starting rank 17 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 17

2022-09-29 06:46:05: Starting rank 18 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 18

2022-09-29 06:46:05: Starting rank 19 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 19

2022-09-29 06:46:05: Starting rank 20 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 20

2022-09-29 06:46:05: Starting rank 21 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 21

2022-09-29 06:46:05: Starting rank 22 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 22

2022-09-29 06:46:05: Starting rank 23 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 23

2022-09-29 06:46:05: Starting rank 24 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 24

2022-09-29 06:46:05: Starting rank 25 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 25

2022-09-29 06:46:05: Starting rank 26 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 26

2022-09-29 06:46:05: Starting rank 27 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 27

2022-09-29 06:46:05: Starting rank 28 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 28

2022-09-29 06:46:05: Starting rank 29 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 29

2022-09-29 06:46:05: Starting rank 30 (using batch size 20)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 30

2022-09-29 06:46:05: Starting rank 0 (using batch size 1)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 0

2022-09-29 06:46:05: Starting interpolation lines (location_property_osmline) (using batch size 20)
2022-09-29 06:46:05: Done 0/0 in 0 @ 0.000 per second - FINISHED interpolation lines (location_property_osmline)

2022-09-29 06:46:06: Using project directory: /nominatim
2022-09-29 06:46:06: Checking database
Checking database connection ... OK
Checking for placex table ... OK
Checking for placex content ... OK
Checking that tokenizer works ... OK
Checking for wikipedia/wikidata data ... WARNING

Wikipedia/Wikidata importance tables missing.
Quality of search results may be degraded. Reverse geocoding is unaffected.
See https://nominatim.org/release-docs/latest/admin/Import/#wikipediawikidata-rankings

Checking indexing status ... OK
Checking that database indexes are complete ... OK
Checking that all database indexes are valid ... OK
Checking TIGER external data table. ... not applicable
kubectl get pods
NAME                     READY   STATUS      RESTARTS   AGE
nominatim-postgresql-0   1/1     Running     0          5m49s
nominatim-init-7647n     0/1     Completed   0          5m49s

As you can see, its working.

Can you tell me more about your infrastructure?

mauriciocirelli commented 1 year ago

Hi,

I am using Helm on Windows and Azure Kubernetes Service (Ubuntu 18.04 nodes). I will try to run Helm on Linux to check if there are any differences.

robjuz commented 1 year ago

Can you please show me logs from the Postgres pod?

robjuz commented 1 year ago

You can additionally set postgres.image.debug = true

mauriciocirelli commented 1 year ago

Hi @robjuz,

We have found the issue. We created a namespace "nominatim" for this deployment and configured kubectl context to run commands there. Changing to the default namespace worked.

This is something we can live with, but would be nice if we could deploy it on any namespace.

Thank you!

mauriciocirelli commented 1 year ago

Can you please show me logs from the Postgres pod?

DETAIL: User "postgres" has no password assigned.

It repeats that line continuously.

robjuz commented 1 year ago

Please try installing the chart with helm install --namespace nominatim ... I‘m not sure how Helm will handle the installation then the namespace is set in kube config

mauriciocirelli commented 1 year ago

Please try installing the chart with helm install --namespace nominatim ... I‘m not sure how Helm will handle the installation then the namespace is set in kube config

Tried that.

The nominatim logs are the following:

Defaulted container "nominatim" out of: nominatim, wait-for-db (init), download-wikipedia (init), download-pbf (init)
2022-10-03 14:20:02: Using project directory: /nominatim
2022-10-03 14:20:04: Creating database
createdb: error: connection to server at "nominatim-postgresql" (10.10.160.63), port 5432 failed: FATAL:  password authentication failed for user "postgres"
2022-10-03 14:20:05: FATAL: Creating new database failed.

The PG logs are the following:

2022-10-03 14:16:42.033 GMT [352] FATAL:  password authentication failed for user "postgres"
2022-10-03 14:16:42.033 GMT [352] DETAIL:  User "postgres" has no password assigned.
        Connection matched pg_hba.conf line 1: "host     all             all             0.0.0.0/0               md5"

It seems that there is a user misconfiguration when running on a different namespace...

robjuz commented 1 year ago

@mauriciocirelli I just did some refactor. Please try the version 3.1.0 if the problem is resolved

mauriciocirelli commented 1 year ago

Hi!

The database has been installed using v 3.1.0. However, replication is not working as expected. It is stuck at an error loop:

2022-10-20 12:35:47: Using project directory: /nominatim
2022-10-20 12:35:51: Replication not set up. Please run 'nominatim replication --init' first.
2022-10-20 12:35:51: FATAL: Replication not set up.
waster commented 1 year ago

Please try installing the chart with helm install --namespace nominatim ... I‘m not sure how Helm will handle the installation then the namespace is set in kube config

Tried that.

The nominatim logs are the following:

Defaulted container "nominatim" out of: nominatim, wait-for-db (init), download-wikipedia (init), download-pbf (init)
2022-10-03 14:20:02: Using project directory: /nominatim
2022-10-03 14:20:04: Creating database
createdb: error: connection to server at "nominatim-postgresql" (10.10.160.63), port 5432 failed: FATAL:  password authentication failed for user "postgres"
2022-10-03 14:20:05: FATAL: Creating new database failed.

The PG logs are the following:

2022-10-03 14:16:42.033 GMT [352] FATAL:  password authentication failed for user "postgres"
2022-10-03 14:16:42.033 GMT [352] DETAIL:  User "postgres" has no password assigned.
        Connection matched pg_hba.conf line 1: "host     all             all             0.0.0.0/0               md5"

It seems that there is a user misconfiguration when running on a different namespace...

Still the same problem here while installing into the specific namespace with PVC from scratch.