search

systemed / tilemaker

Make OpenStreetMap vector tiles without the stack
https://tilemaker.org/
Other
1.44k stars 229 forks source link

Unable to open landcover SHP #573

Closed felipemendes1994 closed 8 months ago

felipemendes1994 commented 10 months ago

Hi,

I've been trying to generate a whole planet mbtiles based on OSM pbf file, but I've been facing the following error:

Layer place (z0-14)
Layer boundary (z0-14)
Layer poi (z12-14)
Layer poi_detail (z14-14) -> poi
Layer housenumber (z14-14)
Layer waterway (z8-14)
Layer waterway_detail (z12-14) -> waterway
Layer transportation (z4-14)
Layer transportation_name (z8-14)
Layer transportation_name_mid (z12-14) -> transportation_name
Layer transportation_name_detail (z14-14) -> transportation_name
Layer building (z13-14)
Layer water (z6-14)
Layer ocean (z0-14) -> water
Layer water_name (z14-14)
Layer water_name_detail (z14-14) -> water_name
Layer aeroway (z11-14)
Layer aerodrome_label (z10-14)
Layer park (z11-14)
Layer landuse (z4-14)
Layer urban_areas (z4-8) -> landuse
Layer landcover (z0-14)
Layer ice_shelf (z0-9) -> landcover
Layer glacier (z2-9) -> landcover
Layer mountain_peak (z11-14)
Bounding box -180, -85.06, 180, 85.06
Reading .shp ocean
Reading .shp urban_areas
Reading .shp ice_shelf
Reading .shp glacier
Sorting loaded shapes
Reading .pbf /pbfs/planet-231030.osm.pbf
Unable to open coastline/water_polygons.shp or coastline/water_polygons.SHP.
Unable to open landcover/ne_10m_urban_areas/ne_10m_urban_areas.shp or landcover/ne_10m_urban_areas/ne_10m_urban_areas.SHP.
Unable to open landcover/ne_10m_antarctic_ice_shelves_polys/ne_10m_antarctic_ice_shelves_polys.shp or landcover/ne_10m_antarctic_ice_shelves_polys/ne_10m_antarctic_ice_shelves_polys.SHP.
Unable to open landcover/ne_10m_glaciated_areas/ne_10m_glaciated_areas.shp or landcover/ne_10m_glaciated_areas/ne_10m_glaciated_areas.SHP.

I'm using the standard configuration based in tag 2.4.0 inside a docker image.

I also tried with different pbf files downloaded directly from OSM website. So I'm pretty sure the file is not corrupted.

Any idea of what may been causing it?

systemed commented 10 months ago

Sorry, the landcover files aren't documented and should be. They're optional files that you can download and put in those locations. You can download them from naturalearthdata.com - start at https://www.naturalearthdata.com/downloads/ and look for the appropriate 1:10m files, and put them in the locations tilemaker says it wants.

You also need the coastline files and that's documented in the README.

felipemendes1994 commented 10 months ago

Hi @systemed ,

No problem at all. I'll follow your instructions, and return you the result asap. But my doubt is, is this absence determinant for process crash, once those files are optional?

systemed commented 10 months ago

No, it won't cause a crash - it'll just mean that these features aren't present in the map.

felipemendes1994 commented 10 months ago

But my mbtile creation process crashes.

May this crash be caused by my resources quantity? Currently I have 110Gi Ram, and a PVC of 200Gi in my k8s cluster, and that's not a limit. I can upgrade it more if needed. I'm asking this, cause, in my pod resources doesn't touch the max availability before crashing.

One more thing. I've been searching about coastline in docs, and README but I found no mentions to it. Would you point me a link?

I think this scenario may be a nice lab for tilemaker planet mbtile generation, and I do want to collaborate with the project.

systemed commented 10 months ago

In the README it says:

If you want to include sea tiles, then create a directory called coastline in the same place you're running tilemaker from, and then save the files from https://osmdata.openstreetmap.de/download/water-polygons-split-4326.zip in it, such that tilemaker can find a file at coastline/water_polygons.shp.

For memory requirements - we have some comments in #315. In February 2022 it was possible to generate the planet using 118GB RAM with these provisos:

OSM will have grown in the intervening 18 months, but you get the general order of magnitude. You don't have to use --compact or exclude the poles, but you'll need more memory if you don't.

499 should reduce memory requirements substantially but won't land for another couple of months.

systemed commented 8 months ago

landcover docs and memory requirements both fixed in v3.