systemed / tilemaker

Make OpenStreetMap vector tiles without the stack
https://tilemaker.org/
Other
1.46k stars 230 forks source link

Ocean and land area is missing #720

Closed BinoyKadakkal closed 4 months ago

BinoyKadakkal commented 4 months ago

Ocean and land area is missing while converting .osm.pbf file to .mbtiles. and shows the error

_### Unable to open coastline/water_polygons.shp or coastline/water_polygons.SHP. Reading shapefile urban_areas Unable to open landcover/ne_10m_urban_areas/ne_10m_urban_areas.shp or landcover/ne_10m_urban_areas/ne_10m_urban_areas.SHP. Reading shapefile ice_shelf Unable to open landcover/ne_10m_antarctic_ice_shelves_polys/ne_10m_antarctic_ice_shelves_polys.shp or landcover/ne_10m_antarctic_ice_shelves_polys/ne_10m_antarctic_ice_shelves_polys.SHP. Reading shapefile glacier Unable to open landcover/ne_10m_glaciated_areas/ne_10m_glaciated_areas.shp or landcover/ne_10m_glaciated_areas/ne_10m_glaciatedareas.SHP. Screenshot from 2024-05-13 16-12-04

Tilemaker is running in docker

systemed commented 4 months ago

From the readme:

To include sea tiles, create a directory called coastline in the same place you're running tilemaker from, and then save the files from https://osmdata.openstreetmap.de/download/water-polygons-split-4326.zip in it, such that tilemaker can find a file at coastline/water_polygons.shp.

The error you are getting shows that you don't have a file at the location where tilemaker is expecting to find it. Make sure that you don't have any extra levels of directory in there.

(Please use Github discussions, not issues, for help questions. Thank you :) )

BinoyKadakkal commented 4 months ago

i saw that and i had created the folder as shown in the image and downloaded the data from https://osmdata.openstreetmap.de. and done several trials by changing the location, rebuilding the docker image and extracting the file. It still shows the same error.

Screenshot from 2024-05-13 16-25-06

systemed commented 4 months ago

Ah, you're using Docker. Sorry, I don't know how the filesystem works there, hopefully someone else will.

baurzhan commented 4 months ago

Run container without --rm option. docker run -it --name=tilemaker -v $(pwd):/data ghcr.io/systemed/tilemaker:master /data/monaco-latest.osm.pbf /data/monaco-latest.mbtiles Wait for the container to stop. Copy folders into container and restart the container: docker cp landcover tilemaker:/usr/src/app/ docker cp coastline tilemaker:/usr/src/app/ docker start tilemaker

BinoyKadakkal commented 4 months ago

it still shows the same problem. after following the instructions i tried to run docker run -v $(pwd):/data tilemaker /data/southern-zone-latest.osm.pbf /data/india.mbtiles

It also doesn't work

systemed commented 4 months ago

@daniel-j-h Could I trouble you for any thoughts on this?

daniel-j-h commented 4 months ago

I believe this has to do with absolute vs relative paths; in your config.json can you check if you have relative paths such as

and turn them into absolute paths such as

Then run it like

docker run -it --rm --pull always -v $(pwd):/data ghcr.io/systemed/tilemaker:master /data/monaco-latest.osm.pbf --output /data/monaco-latest.pmtiles --config /data/config-coastline.json --process /data/process-coastline.lua

I'm not sure where exactly tilemaker looks for those files otherwise - is it relative to the tilemaker binary by chance @systemed?


Two learnings here

  1. Should we have a script users can run to download this data, unzip it, put it in the right place; I have something like that already for my own experimentation and can upstream it if I have a bit more time throughout the week, and
  2. Should tilemaker get flags for coastline and landcover directories; I believe it's okay to have it in the config like this but maybe we need to put it into the documentation
systemed commented 4 months ago

I'm not sure where exactly tilemaker looks for those files otherwise - is it relative to the tilemaker binary by chance @systemed?

Everything is relative to the current working directory.

As far as tilemaker is concerned, coastline and landcover are just another datasource - there's no special handling. The config for a particular set of vector tiles might have both, or none, or lots more! So I don't think we'd want to add special handling for these paths.

Would something like a --base-dir option, which specifies the root path for the source data, be helpful? That way you could invoke it with something like

tilemaker input.osm.pbf output.mbtiles --base-dir /data

and it would then look for /data/input.osm.pbf, /data/coastline/water_polygons.shp, etc. etc.

daniel-j-h commented 4 months ago

Not sure we need an extra option for tilemaker when you can change the paths in the config file and use both absolute and relative paths there. In addition there is a -w option (docs) for docker to change the current workdir, so we can just pass -w /data and make sure tilemaker sees the mounted data dir as the current working dir.

With relative paths in the configs and changing the workdir the command looks as follows

docker run -it --rm --pull always -v $(pwd):/data -w /data ghcr.io/systemed/tilemaker:master /data/monaco-latest.osm.pbf --output /data/monaco-latest.pmtiles --config /data/config-coastline.json --process /data/process-coastline.lua
baurzhan commented 4 months ago

it still shows the same problem. after following the instructions i tried to run docker run -v $(pwd):/data tilemaker /data/southern-zone-latest.osm.pbf /data/india.mbtiles

It also doesn't work

"docker run" creates new container. In my instructions you shouldn't create new container. Just restart existing container. After docker restart tilemaker run docker logs -f tilemaker to see output

baurzhan commented 4 months ago

Updated instructions.

  1. Create container: docker create -i -t --name=tilemaker -v $(pwd):/data ghcr.io/systemed/tilemaker:master /data/monaco-latest.osm.pbf /data/monaco-latest.mbtiles
  2. Copy folders into container: docker cp landcover tilemaker:/usr/src/app/ docker cp coastline tilemaker:/usr/src/app/
  3. Start container docker start tilemaker && docker logs -f tilemaker
daniel-j-h commented 4 months ago

I'm capturing learnings from here and https://github.com/systemed/tilemaker/issues/720#issuecomment-2112735478 in

BinoyKadakkal commented 4 months ago

It works, Thank you for the support