stefanodecillis
commented
3 years ago Hello, I would also add that these are the tiles that the server is providing
Closed stefanodecillis closed 3 years ago
stefanodecillis
commented
3 years ago Hello, I would also add that these are the tiles that the server is providing
Istador
commented
3 years ago According to your screenshot, the server does render and serve tiles, but they are empty. This'd suggest that the database is empty. E.g. because the import failed or hasn't finished yet.
The full log from the import might be helpful.
I don't see any obvious mistake in your configuration, though it might be problematic, depending on how you use it, what you have changed or have left out.
command: "import" is a bit odd, but as long as you change the command to run after successfully importing it and then restart (docker-compose up -d), you should be fine. (Otherwise it'd import again on every start of the container and then exit after the import, without ever running the http server. Though you posted run logs and a screenshot, so you somehow overcome that - without telling how. Or you didn't import and are just running with an empty database now, then the screenshot looks acurate.) Because you have the database as a volume mount, the database files should be kept even when the service is recreated.
If your host system is windows, you might have permission issues mounting the host filesystem as volumes (depending on where ./ is, the directory might not be allowed to use for volumes).
Please make sure to delete ./osm-tile/render/tile, so that you don't see old tiles from previous attempts.
If you want to start completely fresh, make sure to also delete ./osm-tile/render/data and the container with docker-compose down
stefanodecillis
commented
3 years ago @Istador thank you for your reply. Actually I tried the following guide https://switch2osm.org/serving-tiles/using-a-docker-container/ and it looks pretty simple. Even here I end with the same result. (I forgot to mention that in my docker-compose I change the command in run).
In particular, I used this command
time docker run -v /home/renderaccount/zambia-latest.osm.pbf:/data.osm.pbf -v openstreetmap-data:/var/lib/postgresql/12/main overv/openstreetmap-tile-server:1.3.10 import
and this is my output:
+ cat /etc/postgresql/12/main/conf.d/postgresql.custom.conf,
# Suggested minimal settings from,
# https://ircama.github.io/osm-carto-tutorials/tile-server-ubuntu/,
,
shared_buffers = 128MB,
min_wal_size = 1GB,
max_wal_size = 2GB,
maintenance_work_mem = 256MB,
,
# Suggested settings from,
# https://github.com/openstreetmap/chef/blob/master/roles/tile.rb#L38-L45,
,
max_connections = 250,
temp_buffers = 32MB,
work_mem = 128MB,
wal_buffers = 1024kB,
wal_writer_delay = 500ms,
commit_delay = 10000,
# checkpoint_segments = 60 # unrecognized in psql 10.7.1,
max_wal_size = 2880MB,
random_page_cost = 1.1,
track_activity_query_size = 16384,
autovacuum_vacuum_scale_factor = 0.05,
autovacuum_analyze_scale_factor = 0.02,
,
listen_addresses = '*',
autovacuum = on,
+ service postgresql start,
* Starting PostgreSQL 12 database server,
...done.,
+ sudo -u postgres createuser renderer,
+ sudo -u postgres createdb -E UTF8 -O renderer gis,
+ sudo -u postgres psql -d gis -c 'CREATE EXTENSION postgis;',
CREATE EXTENSION,
+ sudo -u postgres psql -d gis -c 'CREATE EXTENSION hstore;',
CREATE EXTENSION,
+ sudo -u postgres psql -d gis -c 'ALTER TABLE geometry_columns OWNER TO renderer;',
ALTER TABLE,
+ sudo -u postgres psql -d gis -c 'ALTER TABLE spatial_ref_sys OWNER TO renderer;',
ALTER TABLE,
+ setPostgresPassword,
+ sudo -u postgres psql -c 'ALTER USER renderer PASSWORD '\''renderer'\''',
ALTER ROLE,
+ '[' '!' -f /data.osm.pbf ']',
+ '[' disabled = enabled ']',
+ '[' -f /data.poly ']',
+ sudo -u renderer osm2pgsql -d gis --create --slim -G --hstore --tag-transform-script /home/renderer/src/openstreetmap-carto/openstreetmap-carto.lua --number-processes 4 -S /home/renderer/src/openstreetmap-carto/openstreetmap-carto.style /data.osm.pbf,
osm2pgsql version 1.2.0 (64 bit id space),
,
Allocating memory for dense node cache,
Allocating dense node cache in one big chunk,
Allocating memory for sparse node cache,
Sharing dense sparse,
Node-cache: cache=800MB, maxblocks=12800*65536, allocation method=11,
Mid: pgsql, cache=800,
Setting up table: planet_osm_nodes,
Setting up table: planet_osm_ways,
Setting up table: planet_osm_rels,
Using lua based tag processing pipeline with script /home/renderer/src/openstreetmap-carto/openstreetmap-carto.lua,
Using projection SRS 3857 (Spherical Mercator),
Setting up table: planet_osm_point,
Setting up table: planet_osm_line,
Setting up table: planet_osm_polygon,
Setting up table: planet_osm_roads,
,
Reading in file: /data.osm.pbf,
Using PBF parser.,
Processing: Node(70k 70.0k/s) Way(0k 0.00k/s) Relation(0 0.00/s)
Processing: Node(210k 105.0k/s) Way(0k 0.00k/s) Relation(0 0.00/s)
Processing: Node(330k 110.0k/s) Way(0k 0.00k/s) Relation(0 0.00/s)
Processing: Node(470k 117.5k/s) Way(0k 0.00k/s) Relation(0 0.00/s)
Processing: Node(660k 132.0k/s) Way(0k 0.00k/s) Relation(0 0.00/s)
Processing: Node(810k 135.0k/s) Way(0k 0.00k/s) Relation(0 0.00/s)
Processing: Node(980k 140.0k/s) Way(0k 0.00k/s) Relation(0 0.00/s)
Processing: Node(1110k 138.8k/s) Way(0k 0.00k/s) Relation(0 0.00/s)
[...]
Processing: Node(502880k 232.1k/s) Way(0k 0.00k/s) Relation(0 0.00/s)result COPY END for planet_osm_nodes failed: ERROR: duplicate key value violates unique constraint "planet_osm_nodes_pkey",
DETAIL: Key (id)=(29673302) already exists.,
CONTEXT: COPY planet_osm_nodes, line 431374,
,
DB copy thread failed: Ending COPY mode,
+ sudo -u postgres psql -d gis -f indexes.sql,
CREATE INDEX,
CREATE INDEX,
CREATE INDEX,
CREATE INDEX,
CREATE INDEX,
CREATE INDEX,
CREATE INDEX,
CREATE INDEX,
CREATE INDEX,
CREATE INDEX,
CREATE INDEX,
CREATE INDEX,
CREATE INDEX,
+ touch /var/lib/mod_tile/planet-import-complete,
+ service postgresql stop,
* Stopping PostgreSQL 12 database server,
...done.,
+ exit 0,then I use this command
docker run -p 8080:80 -v openstreetmap-data:/var/lib/postgresql/12/main -d overv/openstreetmap-tile-server:1.3.10 run
and logs are:
renderd[143]: DEBUG: Loading font: /usr/share/fonts/truetype/dejavu/DejaVuSans-Oblique.ttf
renderd[143]: DEBUG: Loading font: /usr/share/fonts/truetype/dejavu/DejaVuSansMono-Bold.ttf
renderd[143]: DEBUG: Loading font: /usr/share/fonts/truetype/dejavu/DejaVuSerif.ttf
renderd[143]: DEBUG: Loading font: /usr/share/fonts/truetype/dejavu/DejaVuSans-ExtraLight.ttf
renderd[143]: DEBUG: Loading font: /usr/share/fonts/truetype/dejavu/DejaVuSerifCondensed.ttf
renderd[143]: DEBUG: Loading font: /usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf
Running in foreground mode...
renderd[143]: Starting stats thread
debug: init_storage_backend: initialising file storage backend at: /var/lib/mod_tile
debug: init_storage_backend: initialising file storage backend at: /var/lib/mod_tile
renderd[143]: Loading parameterization function for
debug: init_storage_backend: initialising file storage backend at: /var/lib/mod_tile
renderd[143]: Loading parameterization function for
renderd[143]: Loading parameterization function for
debug: init_storage_backend: initialising file storage backend at: /var/lib/mod_tile
renderd[143]: Loading parameterization function for
Mapnik LOG> 2021-03-31 16:23:06: warning: unable to find face-name 'Noto Sans CJK JP Regular' in FontSet 'fontset-0'
Mapnik LOG> 2021-03-31 16:23:06: warning: unable to find face-name 'Noto Sans CJK JP Regular' in FontSet 'fontset-0'
[...]
Mapnik LOG> 2021-03-31 16:23:06: warning: unable to find face-name 'HanaMinA Regular' in FontSet 'fontset-2'
Mapnik LOG> 2021-03-31 16:23:06: warning: unable to find face-name 'HanaMinB Regular' in FontSet 'fontset-2'
Mapnik LOG> 2021-03-31 16:23:06: warning: unable to find face-name 'unifont Medium' in FontSet 'fontset-2'
Mapnik LOG> 2021-03-31 16:23:06: warning: unable to find face-name 'Noto Serif Tibetan Bold' in FontSet 'fontset-2'
Mapnik LOG> 2021-03-31 16:23:06: warning: unable to find face-name 'Noto Sans CJK JP Regular' in FontSet 'fontset-2'
Mapnik LOG> 2021-03-31 16:23:06: warning: unable to find face-name 'Noto Serif Tibetan Regular' in FontSet 'fontset-2'
Mapnik LOG> 2021-03-31 16:23:06: warning: unable to find face-name 'Noto Emoji Regular' in FontSet 'fontset-2'
Mapnik LOG> 2021-03-31 16:23:06: warning: unable to find face-name 'HanaMinA Regular' in FontSet 'fontset-2'
Mapnik LOG> 2021-03-31 16:23:06: warning: unable to find face-name 'HanaMinB Regular' in FontSet 'fontset-2'
Mapnik LOG> 2021-03-31 16:23:06: warning: unable to find face-name 'unifont Medium' in FontSet 'fontset-2'
renderd[143]: Using web mercator projection settings
renderd[143]: Using web mercator projection settings
renderd[143]: Using web mercator projection settings
renderd[143]: Using web mercator projection settingsThe result looks the same:
I checked the osm.pbf file and it's 4.5gb so I'm sure that in some part of the world I should see some tile 😂. Am I forgetting something?
stefanodecillis
commented
3 years ago I will add that I tried the Dockerfile and it fails on the OpenStreetMap-carto. I guess it looks on a file that has been moved
Failed to load file /home/renderer/src/openstreetmap-carto/palette.mss.
The command '/bin/sh -c mkdir -p /home/renderer/src && cd /home/renderer/src && git clone --single-branch --branch v5.2.0 https://github.com/gravitystorm/openstreetmap-carto.git --depth 1 && cd openstreetmap-carto && rm -rf .git && npm install -g carto@0.18.2 && wget https://raw.githubusercontent.com/mapbox/osm-bright/master/osm-bright/osm-bright.osm2pgsql.mml && carto osm-bright.osm2pgsql.mml > mapnik.xml && scripts/get-shapefiles.py && rm /home/renderer/src/openstreetmap-carto/data/*.zip' returned a non-zero code: 1Actually I tried the following guide https://switch2osm.org/serving-tiles/using-a-docker-container/ and it looks pretty simple.
Ah, cool I didn't know https://switch2osm.org/ had a guide for this docker image.
It is using version 1.3.10 which is 13 months old. But that shouldn't be an an issue in itself. The latest tag is v1.6.0 though from one month ago.
The import had an error:
Processing: Node(502880k 232.1k/s) Way(0k 0.00k/s) Relation(0 0.00/s)result COPY END for planet_osm_nodes failed: ERROR: duplicate key value violates unique constraint "planet_osm_nodes_pkey",
DETAIL: Key (id)=(29673302) already exists.,
CONTEXT: COPY planet_osm_nodes, line 431374,
,
DB copy thread failed: Ending COPY mode,I will add that I tried the Dockerfile and it fails on the OpenStreetMap-carto. I guess it looks on a file that has been moved
Failed to load file /home/renderer/src/openstreetmap-carto/palette.mss. The command '/bin/sh -c mkdir -p /home/renderer/src && cd /home/renderer/src && git clone --single-branch --branch v5.2.0 https://github.com/gravitystorm/openstreetmap-carto.git --depth 1 && cd openstreetmap-carto && rm -rf .git && npm install -g carto@0.18.2 && wget https://raw.githubusercontent.com/mapbox/osm-bright/master/osm-bright/osm-bright.osm2pgsql.mml && carto osm-bright.osm2pgsql.mml > mapnik.xml && scripts/get-shapefiles.py && rm /home/renderer/src/openstreetmap-carto/data/*.zip' returned a non-zero code: 1
That is not the Dockerfile of this project. It is trying to install another style named osm-bright instead of openstreetmap-carto. But it downloads osm-bright's .mml file directly, and is not checking out the whole git repository. Therefore it's missing additional files that are required.
This is the current Dockerfile:
https://github.com/Overv/openstreetmap-tile-server/blob/75e7714f200cf452bfbe2f249453125632773e48/Dockerfile#L122-L131
stefanodecillis
commented
3 years ago Thanks for pointing out the error @Istador ! Do you think that is a problem of my file? I merged two files osm.pbf - I don't know if that is the error.
For the Dockerfile, I copied the one in this repo but I'm also trying to customise the style. In one of the old issues, the owner of the project wrote that I need to run carto project.mml > Mapnik.mml that is why I modified that. What else do I need?
Sorry to bother you so much but I'm really newbie at this.
Istador
commented
3 years ago Thanks for pointing out the error @Istador ! Do you think that is a problem of my file? I merged two files osm.pbf - I don't know if that is the error.
Might be, the error message'd suppose that. If merging two regions directly next to each other, without removing duplicate osm_id's, then that'll be a problem during the import.
Which program do you use to merge the PBF files, and what command do you run exactly?
osmconvert - https://github.com/Overv/openstreetmap-tile-server/issues/2#issuecomment-468008764 and https://github.com/Overv/openstreetmap-tile-server/issues/2#issuecomment-468393785osmium - https://github.com/Overv/openstreetmap-tile-server/issues/20#issuecomment-469074287osmosisI only ever used osmconvert and had no issues merging Europe and Africa (3 years ago).
For the Dockerfile, I copied the one in this repo but I'm also trying to customise the style. In one of the old issues, the owner of the project wrote that I need to run
carto project.mml > Mapnik.mmlthat is why I modified that. What else do I need?
I'd first try to get the server working before trying to change the style.
And then I'd replace the the whole style block with my own. You don't need to install openstreetmap-carto, if you aren't using it at all. Clone the whole osm-bright repo with a specific branch, tag or commit hash (like osm carto). Make sure to follow the setup instructions from that project and if you have issues with it, better direct them to that project.
Or rather than manipulating the Dockerfile and build everything myself, I'd take it as a base image as is for my own Dockerfile (starting with FROM overv/openstreetmap-tile-server:v1.6.0), install osm-bright additionally in another directory and change the configuration to use that instead (maybe even configure it to work with both styles).
stefanodecillis
commented
3 years ago Thank you @Istador for your tips. I guess you helped me with my two problems! I will close the issue with this comment!
Thank you again!
Hello,
sorry to bother but I'm facing some issue regarding the docker image. In particular, I make the import of my .pbf and the run without problems (I guess), the demo-leaflet.html runs but it doesn't render any tile. Also it fails in making single tile requests. My
docker-compose.ymllooks like:In the README.md uses a volume - I tried also the volume but it doesn't work. I think this is only an error that it occurs to me but it looks everything fine. Can you help me with it?
I'm leaving here some logs: