search

Jean-Baptiste-Lasselle / jbl-osm-original

I started from https://shinesolutions.com/2016/12/30/generating-high-res-maps-with-mapnik-and-docker/
Apache License 2.0
0 stars 0 forks source link

RENDERER_TO_POSTGIS #5

Open Jean-Baptiste-Lasselle opened 5 years ago

Jean-Baptiste-Lasselle commented 5 years ago

It looks like renderer believes PostGIS database server is not initialized : 

[jibl@pc-100 carto-proto]$ docker logs -f postgis
LOG:  database system was shut down at 2018-10-22 15:21:19 UTC
LOG:  MultiXact member wraparound protections are now enabled
LOG:  database system is ready to accept connections
LOG:  autovacuum launcher started
^C
[jibl@pc-100 carto-proto]$ docker logs -f rendereurpoulet
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
^C

yet, the project.yaml file in the renderer container IS modified with Database Connection settings Is there a different config param for Database network host ID, depending it's a domain name/hostname, or an IP address (like in libpg / postgres's main lib , with PGHOSTADDR and PGHOST)

Jean-Baptiste-Lasselle commented 5 years ago

Ohh you oiece of *** ! :skull: :100: =>

oh you piece of ***

Is that what you deliver as a checkhealth to your customers ???

Jean-Baptiste-Lasselle commented 5 years ago

And there we go :+1: 

[jibl@pc-100 carto-proto]$ docker logs -f rendereurpoulet
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing..
[...].=> I cut the cra, just many more "Waiting while database is initializing..."  logs
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
Waiting while database is initializing...
^C
[jibl@pc-100 carto-proto]$ docker exec -it rendereurpoulet sh -c "ls -all /var/lib/postgresql/data"
total 60
drwx------. 19  999  999  4096 Oct 22 16:37 .
drwxr-xr-x.  3 root root    18 Oct 22 16:37 ..
-rw-------.  1  999  999     4 Oct 21 18:47 PG_VERSION
drwx------.  7  999  999    67 Oct 21 18:47 base
drwx------.  2  999  999  4096 Oct 22 16:38 global
drwx------.  2  999  999    18 Oct 21 18:47 pg_clog
drwx------.  2  999  999     6 Oct 21 18:47 pg_commit_ts
drwx------.  2  999  999     6 Oct 21 18:47 pg_dynshmem
-rw-------.  1  999  999  4507 Oct 21 18:47 pg_hba.conf
-rw-------.  1  999  999  1636 Oct 21 18:47 pg_ident.conf
drwx------.  4  999  999    39 Oct 21 18:47 pg_logical
drwx------.  4  999  999    36 Oct 21 18:47 pg_multixact
drwx------.  2  999  999    18 Oct 22 16:37 pg_notify
drwx------.  2  999  999     6 Oct 21 18:47 pg_replslot
drwx------.  2  999  999     6 Oct 21 18:47 pg_serial
drwx------.  2  999  999     6 Oct 21 18:47 pg_snapshots
drwx------.  2  999  999     6 Oct 22 16:37 pg_stat
drwx------.  2  999  999    63 Oct 22 17:43 pg_stat_tmp
drwx------.  2  999  999    18 Oct 21 18:47 pg_subtrans
drwx------.  2  999  999     6 Oct 21 18:47 pg_tblspc
drwx------.  2  999  999     6 Oct 21 18:47 pg_twophase
drwx------.  3  999  999   124 Oct 21 21:18 pg_xlog
-rw-------.  1  999  999    88 Oct 21 18:47 postgresql.auto.conf
-rw-------.  1  999  999 21279 Oct 21 18:47 postgresql.conf
-rw-------.  1  999  999    37 Oct 22 16:37 postmaster.opts
-rw-------.  1  999  999    85 Oct 22 16:37 postmaster.pid
[jibl@pc-100 carto-proto]$ docker exec -it postgis sh -c "ls -all /var/lib/postgresql/data"
total 60
drwx------. 19 postgres postgres  4096 Oct 22 16:37 .
drwxr-xr-x.  1 postgres postgres    18 Oct 16 05:07 ..
drwx------.  7 postgres postgres    67 Oct 21 18:47 base
drwx------.  2 postgres postgres  4096 Oct 22 16:38 global
drwx------.  2 postgres postgres    18 Oct 21 18:47 pg_clog
drwx------.  2 postgres postgres     6 Oct 21 18:47 pg_commit_ts
drwx------.  2 postgres postgres     6 Oct 21 18:47 pg_dynshmem
-rw-------.  1 postgres postgres  4507 Oct 21 18:47 pg_hba.conf
-rw-------.  1 postgres postgres  1636 Oct 21 18:47 pg_ident.conf
drwx------.  4 postgres postgres    39 Oct 21 18:47 pg_logical
drwx------.  4 postgres postgres    36 Oct 21 18:47 pg_multixact
drwx------.  2 postgres postgres    18 Oct 22 16:37 pg_notify
drwx------.  2 postgres postgres     6 Oct 21 18:47 pg_replslot
drwx------.  2 postgres postgres     6 Oct 21 18:47 pg_serial
drwx------.  2 postgres postgres     6 Oct 21 18:47 pg_snapshots
drwx------.  2 postgres postgres     6 Oct 22 16:37 pg_stat
drwx------.  2 postgres postgres    63 Oct 22 17:43 pg_stat_tmp
drwx------.  2 postgres postgres    18 Oct 21 18:47 pg_subtrans
drwx------.  2 postgres postgres     6 Oct 21 18:47 pg_tblspc
drwx------.  2 postgres postgres     6 Oct 21 18:47 pg_twophase
-rw-------.  1 postgres postgres     4 Oct 21 18:47 PG_VERSION
drwx------.  3 postgres postgres   124 Oct 21 21:18 pg_xlog
-rw-------.  1 postgres postgres    88 Oct 21 18:47 postgresql.auto.conf
-rw-------.  1 postgres postgres 21279 Oct 21 18:47 postgresql.conf
-rw-------.  1 postgres postgres    37 Oct 22 16:37 postmaster.opts
-rw-------.  1 postgres postgres    85 Oct 22 16:37 postmaster.pid
[jibl@pc-100 carto-proto]$ docker exec -it postgis sh -c "touch /var/lib/postgresql/data/DN_INITED"
[jibl@pc-100 carto-proto]$ docker exec -it rendereurpoulet sh -c "ls -all /var/lib/postgresql/data"
total 60
drwx------. 19  999  999  4096 Oct 22 17:45 .
drwxr-xr-x.  3 root root    18 Oct 22 16:37 ..
-rw-r--r--.  1 root root     0 Oct 22 17:45 DN_INITED
-rw-------.  1  999  999     4 Oct 21 18:47 PG_VERSION
drwx------.  7  999  999    67 Oct 21 18:47 base
drwx------.  2  999  999  4096 Oct 22 16:38 global
drwx------.  2  999  999    18 Oct 21 18:47 pg_clog
drwx------.  2  999  999     6 Oct 21 18:47 pg_commit_ts
drwx------.  2  999  999     6 Oct 21 18:47 pg_dynshmem
-rw-------.  1  999  999  4507 Oct 21 18:47 pg_hba.conf
-rw-------.  1  999  999  1636 Oct 21 18:47 pg_ident.conf
drwx------.  4  999  999    39 Oct 21 18:47 pg_logical
drwx------.  4  999  999    36 Oct 21 18:47 pg_multixact
drwx------.  2  999  999    18 Oct 22 16:37 pg_notify
drwx------.  2  999  999     6 Oct 21 18:47 pg_replslot
drwx------.  2  999  999     6 Oct 21 18:47 pg_serial
drwx------.  2  999  999     6 Oct 21 18:47 pg_snapshots
drwx------.  2  999  999     6 Oct 22 16:37 pg_stat
drwx------.  2  999  999    63 Oct 22 17:44 pg_stat_tmp
drwx------.  2  999  999    18 Oct 21 18:47 pg_subtrans
drwx------.  2  999  999     6 Oct 21 18:47 pg_tblspc
drwx------.  2  999  999     6 Oct 21 18:47 pg_twophase
drwx------.  3  999  999   124 Oct 21 21:18 pg_xlog
-rw-------.  1  999  999    88 Oct 21 18:47 postgresql.auto.conf
-rw-------.  1  999  999 21279 Oct 21 18:47 postgresql.conf
-rw-------.  1  999  999    37 Oct 22 16:37 postmaster.opts
-rw-------.  1  999  999    85 Oct 22 16:37 postmaster.pid
[jibl@pc-100 carto-proto]$ docker exec -it postgis sh -c "touch /var/lib/postgresql/data/DB_INITED"
[jibl@pc-100 carto-proto]$ docker exec -it rendereurpoulet sh -c "ls -all /var/lib/postgresql/data"
total 60
drwx------. 19  999  999  4096 Oct 22 17:45 .
drwxr-xr-x.  3 root root    18 Oct 22 16:37 ..
-rw-r--r--.  1 root root     0 Oct 22 17:45 DB_INITED
-rw-r--r--.  1 root root     0 Oct 22 17:45 DN_INITED
-rw-------.  1  999  999     4 Oct 21 18:47 PG_VERSION
drwx------.  7  999  999    67 Oct 21 18:47 base
drwx------.  2  999  999  4096 Oct 22 16:38 global
drwx------.  2  999  999    18 Oct 21 18:47 pg_clog
drwx------.  2  999  999     6 Oct 21 18:47 pg_commit_ts
drwx------.  2  999  999     6 Oct 21 18:47 pg_dynshmem
-rw-------.  1  999  999  4507 Oct 21 18:47 pg_hba.conf
-rw-------.  1  999  999  1636 Oct 21 18:47 pg_ident.conf
drwx------.  4  999  999    39 Oct 21 18:47 pg_logical
drwx------.  4  999  999    36 Oct 21 18:47 pg_multixact
drwx------.  2  999  999    18 Oct 22 16:37 pg_notify
drwx------.  2  999  999     6 Oct 21 18:47 pg_replslot
drwx------.  2  999  999     6 Oct 21 18:47 pg_serial
drwx------.  2  999  999     6 Oct 21 18:47 pg_snapshots
drwx------.  2  999  999     6 Oct 22 16:37 pg_stat
drwx------.  2  999  999    63 Oct 22 17:44 pg_stat_tmp
drwx------.  2  999  999    18 Oct 21 18:47 pg_subtrans
drwx------.  2  999  999     6 Oct 21 18:47 pg_tblspc
drwx------.  2  999  999     6 Oct 21 18:47 pg_twophase
drwx------.  3  999  999   124 Oct 21 21:18 pg_xlog
-rw-------.  1  999  999    88 Oct 21 18:47 postgresql.auto.conf
-rw-------.  1  999  999 21279 Oct 21 18:47 postgresql.conf
-rw-------.  1  999  999    37 Oct 22 16:37 postmaster.opts
-rw-------.  1  999  999    85 Oct 22 16:37 postmaster.pid
[jibl@pc-100 carto-proto]$ docker logs -f rendereurpoulet
Waiting while database is initializing...
Waiting while database is initializing...
[...].=> I cut the cra, just many more "Waiting while database is initializing..."  logs
Waiting while database is initializing...
Waiting while database is initializing...
DB successfully created, waiting for restart
Starting renderer
2018/10/22 17:46:21 app.go:266: [INFO] Serving debug data (/debug/vars) on %s... :9090
2018/10/22 17:46:21 app.go:267: [INFO] Serving monitoring xml data on %s... :9090
2018/10/22 17:46:21 app.go:266: [INFO] Serving debug data (/debug/vars) on %s... :9080
2018/10/22 17:46:21 app.go:267: [INFO] Serving monitoring xml data on %s... :9080
2018/10/22 17:46:21 renderselector.go:209: [DEBUG] ping error %v dial tcp 127.0.0.1:8090: getsockopt: connection refused
2018/10/22 17:46:21 renderselector.go:117: [DEBUG] '%v' is %v localhost:8090 Offline
2018/10/22 17:46:21 main.go:118: [INFO] Starting on %s... :8080
2018/10/22 17:46:23 render.go:35: [ERROR] Render child error: %v Exception: Postgis Plugin: FATAL:  role "postgres" does not exist

2018/10/22 17:46:23 render.go:35: [ERROR] Render child error: %v 

2018/10/22 17:46:23 render.go:35: [ERROR] Render child error: %v Connection string: 'host=postgis port=5432 dbname=gis user=postgres connect_timeout=4'

2018/10/22 17:46:23 render.go:35: [ERROR] Render child error: %v   encountered during parsing of layer 'landcover-low-zoom' in Layer at line 334 of '/openstreetmap-carto/stylesheet.xml'

2018/10/22 17:46:23 render.go:35: [ERROR] Render child error: %v Exception: Postgis Plugin: FATAL:  role "postgres" does not exist

2018/10/22 17:46:23 render.go:35: [ERROR] Render child error: %v 

2018/10/22 17:46:23 render.go:35: [ERROR] Render child error: %v Connection string: 'host=postgis port=5432 dbname=gis user=postgres connect_timeout=4'

2018/10/22 17:46:23 render.go:35: [ERROR] Render child error: %v   encountered during parsing of layer 'landcover-low-zoom' in Layer at line 334 of '/openstreetmap-carto/stylesheet.xml'

2018/10/22 17:46:23 main.go:91: [CRITICAL] Failed to create tile server: Failed to create some renders: [Invalid read uint64: EOF Invalid read uint64: EOF]
2018/10/22 17:46:51 renderselector.go:209: [DEBUG] ping error %v dial tcp 127.0.0.1:8090: getsockopt: connection refused
2018/10/22 17:46:51 renderselector.go:117: [DEBUG] '%v' is %v localhost:8090 Offline
2018/10/22 17:47:21 renderselector.go:209: [DEBUG] ping error %v dial tcp 127.0.0.1:8090: getsockopt: connection refused
2018/10/22 17:47:21 renderselector.go:117: [DEBUG] '%v' is %v localhost:8090 Offline
2018/10/22 17:47:51 renderselector.go:209: [DEBUG] ping error %v dial tcp 127.0.0.1:8090: getsockopt: connection refused
2018/10/22 17:47:51 renderselector.go:117: [DEBUG] '%v' is %v localhost:8090 Offline
2018/10/22 17:48:21 renderselector.go:209: [DEBUG] ping error %v dial tcp 127.0.0.1:8090: getsockopt: connection refused
2018/10/22 17:48:21 renderselector.go:117: [DEBUG] '%v' is %v localhost:8090 Offline
2018/10/22 17:48:51 renderselector.go:209: [DEBUG] ping error %v dial tcp 127.0.0.1:8090: getsockopt: connection refused
2018/10/22 17:48:51 renderselector.go:117: [DEBUG] '%v' is %v localhost:8090 Offline
2018/10/22 17:49:21 renderselector.go:209: [DEBUG] ping error %v dial tcp 127.0.0.1:8090: getsockopt: connection refused
2018/10/22 17:49:21 renderselector.go:117: [DEBUG] '%v' is %v localhost:8090 Offline

So you can see better :

with colors

My Boy, we're nto finished yet... => credentials are missing to renderer app, to connect to PostGres, so there i'll have to update ./project.yaml with proper credentials, just for the test, then, we'll care about password variable interpolation.

Update : I had already included user and pasword credentials in project.yaml, so they didn't apply. Nevertheless, error messages about credentials really look like PostGreSQL 's libpg.... So I'll apply using PGPASSWORD and PGUSER env variables with renderer's container, in ./docker-compsoe.yml, just as I did a few commits before, for the postgis container

Jean-Baptiste-Lasselle commented 5 years ago

nope, I tried setting PGUSER and PGPASSWORD, but it dpoesn't have any effect on Mapnik : 

CONTAINER ID        IMAGE                         COMMAND                  CREATED             STATUS                    PORTS                                                                          NAMES
845211392b07        carto-proto_renderer          "/bin/sh -c /entrypo…"   2 seconds ago       Up 1 second               0.0.0.0:8080->8080/tcp, 0.0.0.0:9090->9090/tcp                                 rendereurpoulet
81da3f99dbae        carto-proto_postgis           "docker-entrypoint.s…"   3 seconds ago       Up 2 seconds              5432/tcp                                                                       postgis
d4c91982ecc3        nginx:1.11-alpine             "nginx -g 'daemon of…"   3 seconds ago       Up 3 seconds              443/tcp, 0.0.0.0:8888->80/tcp                                                  carto-proto_web_1
5a7c86a2cd3c        nginx                         "nginx -g 'daemon of…"   5 days ago          Up 12 hours               80/tcp, 0.0.0.0:1222->322/tcp, 0.0.0.0:1443->7443/tcp, 0.0.0.0:801->8030/tcp   marguerite_reverseproxy
fc3e83c41a29        marguerite/meteor-ide:1.0.0   "/bin/sh -c $WORKSPA…"   5 days ago          Up 12 hours (healthy)     0.0.0.0:6000->6000/tcp                                                         ide_meteor_marguerite
d5701330a0f2        marguerite/mongo:1.0.0        "docker-entrypoint.s…"   5 days ago          Up 12 hours (unhealthy)   0.0.0.0:27018->27017/tcp                                                       ide_mongo_marguerite
[jibl@pc-100 carto-proto]$ docker logs -f rendereurpoulet
 ------------------------------------------------------------------------------------------- 
 VERIFICATION RENDERER ENNTRYPOINT : [MAPNIK_POSTGRES_USER=renderer-user] 
 VERIFICATION RENDERER ENNTRYPOINT : [MAPNIK_POSTGRES_PASSWORD=whereischarlie] 
 VERIFICATION RENDERER ENNTRYPOINT : [PGUSER=renderer-user] 
 VERIFICATION RENDERER ENNTRYPOINT  : [PGPASSWORD=whereischarlie] 
 ------------------------------------------------------------------------------------------- 
DB successfully created, waiting for restart
Starting renderer
2018/10/22 19:06:53 app.go:266: [INFO] Serving debug data (/debug/vars) on %s... :9080
2018/10/22 19:06:53 app.go:267: [INFO] Serving monitoring xml data on %s... :9080
2018/10/22 19:06:53 renderselector.go:209: [DEBUG] ping error %v dial tcp 127.0.0.1:8090: getsockopt: connection refused
2018/10/22 19:06:53 renderselector.go:117: [DEBUG] '%v' is %v localhost:8090 Offline
2018/10/22 19:06:53 main.go:118: [INFO] Starting on %s... :8080
2018/10/22 19:06:53 app.go:266: [INFO] Serving debug data (/debug/vars) on %s... :9090
2018/10/22 19:06:53 app.go:267: [INFO] Serving monitoring xml data on %s... :9090
2018/10/22 19:06:54 render.go:35: [ERROR] Render child error: %v Exception: Postgis Plugin: FATAL:  role "postgres" does not exist

2018/10/22 19:06:54 render.go:35: [ERROR] Render child error: %v 

2018/10/22 19:06:54 render.go:35: [ERROR] Render child error: %v Connection string: 'host=postgis port=5432 dbname=gis user=postgres connect_timeout=4'

2018/10/22 19:06:54 render.go:35: [ERROR] Render child error: %v   encountered during parsing of layer 'landcover-low-zoom' in Layer at line 334 of '/openstreetmap-carto/stylesheet.xml'

2018/10/22 19:06:54 render.go:35: [ERROR] Render child error: %v Exception: Postgis Plugin: FATAL:  role "postgres" does not exist

2018/10/22 19:06:54 render.go:35: [ERROR] Render child error: %v 

2018/10/22 19:06:54 render.go:35: [ERROR] Render child error: %v Connection string: 'host=postgis port=5432 dbname=gis user=postgres connect_timeout=4'

2018/10/22 19:06:54 render.go:35: [ERROR] Render child error: %v   encountered during parsing of layer 'landcover-low-zoom' in Layer at line 334 of '/openstreetmap-carto/stylesheet.xml'

2018/10/22 19:06:54 main.go:91: [CRITICAL] Failed to create tile server: Failed to create some renders: [Invalid read uint64: EOF Invalid read uint64: EOF]
2018/10/22 19:07:23 renderselector.go:209: [DEBUG] ping error %v dial tcp 127.0.0.1:8090: getsockopt: connection refused
2018/10/22 19:07:23 renderselector.go:117: [DEBUG] '%v' is %v localhost:8090 Offline
2018/10/22 19:07:53 renderselector.go:209: [DEBUG] ping error %v dial tcp 127.0.0.1:8090: getsockopt: connection refused