How does the update process work?

[devopsnot]

By just adding it to the compose it already saw that it was doing an automatic update?

How do I know that an update has been performed?

"-e REPLICATION_URL=https://planet.openstreetmap.org/replication/minute/ \ -e MAX_INTERVAL_SECONDS=60 \ -e UPDATES=enabled \ "

[Istador]

By just adding it to the compose it already saw that it was doing an automatic update?

UPDATES=enabled needs to be set during the import or otherwise it won't work later when UPDATES=enabled is set for the run command.

See the Readme 1 and 2.

How do I know that an update has been performed?

There should be log entries at /var/log/tiles/run.log inside of the docker container.

[devopsnot]

Got it, I performed the definitions inside my docker-compose impose and server, I'm having the following log inside the path " /var/log/tiles/run.log

/home/renderer/src/regional/trim_osc.py:211: FutureWarning: GzipFile was opened for writing, but this will change in future Python releases. Specify the mode argument for opening it for writing. of = gzip.GzipFile(fileobj=of) [2022-12-22 19:01:18] 632 importing diff [2022-12-22 19:01:34] 632 expiring tiles [2022-12-22 19:01:34] 632 Done with import [2022-12-22 19:02:01] 782 start import from seq-nr 90041, replag is 154921s [2022-12-22 19:02:01] 782 downloading diff [2022-12-22 19:02:10] 782 filtering diff

Is this record correct? Has the update been performed?

[SenorKarlos]

Having a 'related' issue, I did the import so long ago now I almost don't remember, but in my compose updates were enabled (or so I thought) and it has not updated anything literally ever

I decided to dig into this finally, and I notice when it starts up the line for updates reports enabled = enabled... no matter if I try to string enabled in my compose

osmtileserver    | 2023-01-14T15:08:32.240294010Z + '[' 1 -ne 1 ']'
osmtileserver    | 2023-01-14T15:08:32.240345199Z + '[' run = import ']'
osmtileserver    | 2023-01-14T15:08:32.240412919Z + '[' run = run ']'
osmtileserver    | 2023-01-14T15:08:32.240502327Z + rm -rf /tmp/hsperfdata_root
osmtileserver    | 2023-01-14T15:08:32.242447297Z + chown postgres:postgres /var/lib/postgresql -R
osmtileserver    | 2023-01-14T15:08:32.260308156Z + '[' enabled == enabled ']'
osmtileserver    | 2023-01-14T15:08:32.260348660Z + echo 'export APACHE_ARGUMENTS='\''-D ALLOW_CORS'\'''

    environment:
      PGPASSWORD: Redacted
      OSM2PGSQL_EXTRA_ARGS: -C 16384
      THREADS: 9
      REPLICATION_URL: https://planet.openstreetmap.org/replication/minute/
      MAX_INTERVAL_SECONDS: 60
      UPDATES: "enabled"  //(started as enabled, tried 'enabled' as well)
      ALLOW_CORS: enabled

[Istador]

You should be able to figure out which version of the image you use by inspecting the image:

docker image inspect overv/openstreetmap-tile-server

Newer images since version 2.0.0 should have information about what they are build on in the labels. E.g. "org.opencontainers.image.revision": "10571b77b756bea8046f59de08d27f84c4faffe9" for the latest image (or even e.g. "org.opencontainers.image.version": "2.2.0" for specific version releases). For older images you might be able to figure out based on the Created value, when it was build and what version was active then.

---

Based on the log output you posted, I assume it's a version earlier than 2.0.0 (a version between Feb 12, 2020 and Mar 31, 2022) (before commit 0f229a0cbdd6bf00ffde1b029da57560e20aee00 and at or after commit f9a29fb23b4ddc8be8441ee6be7e86706ec12008).

The log output that you posted (except the first two lines which are earlier) belong to these source code lines: https://github.com/Overv/openstreetmap-tile-server/blob/f1203714c16be885b2aaa1a9d2790b40d4047a83/run.sh#L95-L104

So the osmtileserver | 2023-01-14T15:08:32.260308156Z + '[' enabled == enabled ']' belongs to the ALLOW_CORS and not to the UPDATES check which comes further down in the script: https://github.com/Overv/openstreetmap-tile-server/blob/f1203714c16be885b2aaa1a9d2790b40d4047a83/run.sh#L116-L119

[SenorKarlos]

Yeah It's definitely between those dates (so pre 2.0) but the directions I'm reading look pretty well the same as my patchy recollections, and it's never ran updates. I'm going to try pulling (literally have never done so since firing it up) and see if I get any indication of it trying to update.

[SenorKarlos]

and that proved to be a mistake.... error after error trying to bring it back up. First two related to timezone mounts I use on like every docker I have, and now postgres not working "ERROR: role "renderer" does not exist"

this is what I get for troubleshooting... every. damn. time.

[Istador]

Well, the 2.0.0 version has breaking changes (hence the major version number bump), so when upgrading to a later version you have to start from scratch or manually migrate your volumes and adjust your commands/configuration to the new mount points.

[SenorKarlos]

Scratch it'll probably be, just sucks in meantime I'm hitting the main OSM for my service. Good thing it's at a low use point and CloudFlare caching etc. Thanks for the pointers!

[Istador]

You could try to get back to the 1.8.2 version, which was the last "release" before 2.0.0, to see if it still works for you. (Use overv/openstreetmap-tile-server:v1.8.2 instead of just overv/openstreetmap-tile-server)

Though I'm not sure if the run.sh from 2.0.0 already partially migrated the file structure in the volumes for you or not, which might make it incompatible with earlier versions. https://github.com/Overv/openstreetmap-tile-server/blob/10571b77b756bea8046f59de08d27f84c4faffe9/run.sh#L136-L146

[SenorKarlos]

So I think this new setup is going to do updates, the initial logs show it pulling a state file and setting that up. It then did a few steps that took anywhere from a few minutes to few hours, but has been 'stuck' on Analyzing table 'planet_osm_polygon'... for 23 hours now. I'm only doing north america and recently set up a new updated nominatim in about a day, so am hoping this is normal and not stuck. My resources aren't being utilized much but the HDD lights are goin like mad lol. It's a mirrored striped raid over Dell SAS drives, so not NVMe but not bad either.

EDIT at just over 24h that stage completed and it's on to more indexing, fingers crossed LOL

Question about the tile updating, could I move my old saved tiles over between volumes, and it'll update those as needed/viewed? Not sure how much of the apache/mod_tile/etc might have changed if at all, or if I should just re-render them. Lat time I used a pearl script to render several full layers, then specific boxes for high use areas on the higher zoom levels. If i don't have to start from scratch that would be ideal I think :D

[Istador]

24 hours to import a whole continent with HDD instead of SSD or NVMe sounds normal to me (even a bit quick IMHO).

The mirrored RAID doesn't help you much with the import, because it's mostly writing to disk and not reading to it. Though, it should be faster rendering compared to just one HDD. (Though compared to SSD/NVMe it should still be significantly slower.)

I'm not sure if the rendered tiles can be reused. Likely yes, because they are just image files. Moving/copying them over might change the file timestamps, so the tiles might be considered longer "fresh" than they should be. Because you have a version change I'd advise to delete all .meta files recursively in the tiles directory. IDK if the meta tiles are compatible between versions or not.

[SenorKarlos]

It was 24 on that one stage, whole run was 36 apparently. Thanks for the advice on the tiles! I was thinking that I might even just be able to get away with a volume rename, then delete the .meta files as advised (those will be re-created I presume?) to avoid the timestamp issue. I'm going to fire it up and let it render a couple low zooms to compare the file structure first I think.

[SenorKarlos]

So I'm not sure, will probably just re-render. When I look at the old volume, there's a folder for ajt (new volume has default instead) with a large file structure, each dir chain ending in one or more meta files. I see no standard image files at all. So deleting those meta files seems like I'd be deleting all the tiles anyways? The osmosis replication is running constantly and properly I believe, so I've won there 😂

[SenorKarlos]

Seeing some strange behavior, will open an issue if it doesn't self-resolve. Despite pre-rendering showing complete on 0-8 so far, 9 in progress, it will not show me zoom 0 or 1 on the slippy map (and it used to) and I've got a huge blank spot on my main service areas (ofc right) at I think z7. Might just be cloudflare/caching etc, so not calling it one just yet

[Istador]

there's a folder for ajt (new volume has default instead) with a large file structure,

Oh right, the name of the folder where the tiles are saved to was changed from ajt to default.

each dir chain ending in one or more meta files. I see no standard image files at all.

The .png files should be on the lowest directory level, e.g. .../0/0/0.png. Meta files in the folders above contain only .meta files, because they contain meta data for bigger areas than individual tiles.

[SenorKarlos]

This is what I see in my WinSCP browser on the current tileserver. The old data is now gone, but every layer I investigated ended the same, like this with one or more meta files, never any png's. I thought it rather odd as well. I know layer 3 is fully rendered, mostly blank world with a little North America detail

[SenorKarlos]

yeah I'm still hella confused but it just, works? render_list reports back making meta and tiles, and my slippy map is loading fast n true, but there are no png's anywhere in this file tree

When I run the same command with .meta as the search it immediately floods the console with results.

[balchen]

A meta tile is a number of regular tiles bunched together. My tile server has 8x8 tiles in a meta tile.