Overv / openstreetmap-tile-server

Docker file for a minimal effort OpenStreetMap tile server
Apache License 2.0
1.23k stars 486 forks source link

How to download more than one .osm.pbf? #367

Open gztchan opened 1 year ago

gztchan commented 1 year ago
docker run \
    -e DOWNLOAD_PBF=https://download.geofabrik.de/europe/luxembourg-latest.osm.pbf \
    -e DOWNLOAD_POLY=https://download.geofabrik.de/europe/luxembourg.poly \
    -v osm-data:/data/database/ \
    overv/openstreetmap-tile-server \
    import

If I rerun the above command, and try to download another .osm.pdf, it shows that createuser: error: creation of new role failed: ERROR: role "renderer" already exists

I checked out the file and found that import always creates a new renderer user in Postgres

Istador commented 1 year ago

import is only expected to be run once, because even if it would continue past the role "renderer" already exists error, it would not "add" to the existing data, but replace everything that is already imported.

You need to download and merge the files for multiple regions locally before importing them, or call osm2pgsql with append manually (SLOW).

VicBancroft3D commented 9 months ago

We have a similar issue in our usage of the tile server. In our use case, we would like to import additional regions and have it load it into the prior postgis database. In this case, the database has already been initialized with the renderer role and the gis database has been created and extended with both postgis and hstore . . .

One solution is to introduce an environment variable, SKIP_DB_INIT, such that if set, skips the initialization procedures. For example, the relevant snippet from run.sh would be as follows.

    if [ -z ${SKIP_DB_INIT} ] ; then
        sudo -u postgres createuser renderer
        sudo -u postgres createdb -E UTF8 -O renderer gis
        sudo -u postgres psql -d gis -c "CREATE EXTENSION postgis;"
        sudo -u postgres psql -d gis -c "CREATE EXTENSION hstore;"
        sudo -u postgres psql -d gis -c "ALTER TABLE geometry_columns OWNER TO renderer;"
        sudo -u postgres psql -d gis -c "ALTER TABLE spatial_ref_sys OWNER TO renderer;"
        setPostgresPassword
    else
        echo "SKIP_DB_INIT is set."
    fi 

The environment variable could be set as follows.

% docker run -e SKIP_DB_INIT=true \
   -v /opt/assets/osm/north-carolina-latest.osm.pbf:/data/region.osm.pbf \
   -v osm-data:/data/database/ \
   overv/openstreetmap-tile-server:debug \
   import

Is there a down side to this ?

Istador commented 9 months ago

Is there a down side to this ?

Yes,

it would not "add" to the existing data, but replace everything that is already imported.

For this to work you'd also need to change the osm2pgsql command to append, which would run very slow compared to a import.


The recommended solution for multiple regions is to

download and merge the files for multiple regions locally before importing them