Error building tiles for Russian Federation

[yashindmitry]

System: Ubuntu 20.10 Docker-Compose File:

version: '3.0'
services:
  valhalla:
    image: gisops/valhalla:latest
    ports:
      - "8002:8002"
    volumes:
      - ./custom_files/:/custom_files
    environment:
      # The tile_file must be located in the `custom_files` folder.
      # The tile_file has priority and is used when valid.
      # If the tile_file doesn't exist, the url is used instead.
      # Don't blank out tile_url when you use tile_file and vice versa.
      - tile_urls=http://download.geofabrik.de/russia-latest.osm.pbf http://download.geofabrik.de/europe-latest.osm.pbf
      # Get correct bounding box from e.g. https://boundingbox.klokantech.com/
      - min_x=-4 # -> Albania | -180 -> World
      - min_y=40 # -> Albania | -90  -> World
      - max_x=80 # -> Albania |  180 -> World
      - max_y=61 # -> Albania |  90  -> World
      - use_tiles_ignore_pbf=True
      - force_rebuild=False
      - force_rebuild_elevation=False
      - build_elevation=False
      - build_admins=True
      - build_time_zones=True

Docker Logs:

...
valhalla_1  | 2021/12/12 22:56:01.057929 [WARN] Restrictions mask exceeds allowable limit: 256
valhalla_1  | 2021/12/12 22:56:01.057965 [WARN] Restrictions mask exceeds allowable limit on wayid: 184551491
valhalla_1  | 2021/12/12 22:56:01.057974 [WARN] Restrictions mask exceeds allowable limit: 256
valhalla_1  | 2021/12/12 22:56:01.632808 [WARN] Restrictions mask exceeds allowable limit on wayid: 35854074
valhalla_1  | 2021/12/12 22:56:01.632841 [WARN] Restrictions mask exceeds allowable limit: 1013
valhalla_1  | 2021/12/12 22:56:01.632850 [WARN] Restrictions mask exceeds allowable limit on wayid: 35854074
valhalla_1  | 2021/12/12 22:56:01.632855 [WARN] Restrictions mask exceeds allowable limit: 1013
valhalla_1  | 2021/12/12 23:02:06.898457 [INFO] Finished
valhalla_1  | terminate called after throwing an instance of 'std::exception'
valhalla_1  |   what():  std::exception
valhalla_1  | /valhalla/scripts/configure_valhalla.sh: line 171:    49 Aborted                 (core dumped) valhalla_build_tiles -c ${CONFIG_FILE} -e build ${files}
gisops_docker_valhalla_1 exited with code 1
root@mart:~/gisops_docker#

[nilsnolde]

every time some country doesn't work that crosses the antimeridian I'm wondering if it's that.. couple of issues lately in upstream valhalla (see antimeridian in issues). @yashindmitry to test that, can you clip the osm file with e.g. osmium to a longitude of 179° (but the rest of the bounding box should stay the same): https://docs.osmcode.org/osmium/latest/osmium-extract.html

where did you get the osm file from?

[nilsnolde]

oh and one more thing: please use the -s, --strategy=STRATEGY "simple" (it defaults to complete_ways). the reason is that there might be some ferry line crossing the antimeridian, we wouldn't want that in the final extract.

[nilsnolde]

just saw the line - tile_urls=http://download.geofabrik.de/russia-latest.osm.pbf http://download.geofabrik.de/europe-latest.osm.pbf

europe includes a part of russia I guess.. best always check geofabrik in case of such ambiguous datasets: http://download.geofabrik.de/europe.html

[yashindmitry]

Thanks for your tips. I tried to remove two files from the config: `

• tile_urls=http://download.geofabrik.de/russia-latest.osm.pbf ` And I get the following error:

  valhalla_1  | 2021/12/13 17:04:42.211276 [WARN] (stat):  No such file or directory
  valhalla_1  | 2021/12/13 17:04:42.211291 [WARN] Traffic tile extract could not be loaded
  valhalla_1  | 2021/12/13 17:04:42.232133 [INFO] Tile extract successfully loaded with tile count: 27543
  valhalla_1  | 2021/12/13 17:04:42.232190 [WARN] (stat):  No such file or directory
  valhalla_1  | 2021/12/13 17:04:42.232198 [WARN] Traffic tile extract could not be loaded
  valhalla_1  | 2021/12/13 17:04:42.233404 [INFO] Tile extract successfully loaded with tile count: 27543
  valhalla_1  | 2021/12/13 17:04:42.233452 [WARN] (stat):  No such file or directory
  valhalla_1  | 2021/12/13 17:04:42.233474 [WARN] Traffic tile extract could not be loaded
  valhalla_1  | 2021/12/13 17:04:42.298467 [INFO] Tile extract successfully loaded with tile count: 27543
  valhalla_1  | 2021/12/13 17:04:42.298545 [WARN] (stat):  No such file or directory
  valhalla_1  | 2021/12/13 17:04:42.298554 [WARN] Traffic tile extract could not be loaded
  valhalla_1  | Too many open files (src/ipc_listener.cpp:358)
  valhalla_1  | Too many open files (src/epoll.cpp:65)
  valhalla_1  | Too many open files (src/epoll.cpp:65)
  valhalla_1  | /valhalla/scripts/run.sh: line 9: 30547 Aborted                 sudo -E $1
  gisops_docker_valhalla_1 exited with code 1
  root@mart:~/gisops_docker# ulimit -n
  1000000
  root@mart:~/gisops_docker# cat /proc/sys/fs/file-max
  9223372036854775807

[nilsnolde]

shows that we have a tiny bug: Traffic tile extract could not be loaded. you're not loading a tar file but you're using the plain graph files on the filesystem. it's bcs you use an old docker-compose.yml; we added build_tar recently which controls the tar building.

it's a bug in that I was aiming to be backwards-compatible in all those changes recently but that one's not. if the new build_tar env var is True this won't happen, you can try that. I just pushed a fix, so in like 10 mins you can download a new latest image which should work with your old docker-compose..

those config files don't age very well generally. I recommend checking into the repo template every now and then.

[yashindmitry]

Thank you very much! I'll try it tomorrow and tell you the result.

[yashindmitry]

I tried starting from the very beginning and it didn't work out for me:

valhalla_1  | 2021/12/14 09:04:31.978068 [INFO] Tile extract successfully loaded with tile count: 27551
valhalla_1  | 2021/12/14 09:04:31.978138 [WARN] (stat):  No such file or directory
valhalla_1  | 2021/12/14 09:04:31.978156 [WARN] Traffic tile extract could not be loaded
valhalla_1  | 2021/12/14 09:04:31.980623 [INFO] Tile extract successfully loaded with tile count: 27551
valhalla_1  | 2021/12/14 09:04:31.980698 [WARN] (stat):  No such file or directory
valhalla_1  | 2021/12/14 09:04:31.980713 [WARN] Traffic tile extract could not be loaded
valhalla_1  | 2021/12/14 09:04:31.997857 [INFO] Tile extract successfully loaded with tile count: 27551
valhalla_1  | 2021/12/14 09:04:31.997924 [WARN] (stat):  No such file or directory
valhalla_1  | 2021/12/14 09:04:31.997935 [WARN] Traffic tile extract could not be loaded
valhalla_1  | 2021/12/14 09:04:32.032202 [INFO] Tile extract successfully loaded with tile count: 27551
valhalla_1  | 2021/12/14 09:04:32.032270 [WARN] (stat):  No such file or directory
valhalla_1  | 2021/12/14 09:04:32.032279 [WARN] Traffic tile extract could not be loaded
valhalla_1  | 2021/12/14 09:04:32.091898 [INFO] Tile extract successfully loaded with tile count: 27551
valhalla_1  | 2021/12/14 09:04:32.091976 [WARN] (stat):  No such file or directory
valhalla_1  | 2021/12/14 09:04:32.091984 [WARN] Traffic tile extract could not be loaded
valhalla_1  | Too many open files (src/epoll.cpp:65)
valhalla_1  | /valhalla/scripts/run.sh: line 9: 31021 Aborted                 sudo -E $1
gisops_docker2_valhalla_1 exited with code 1
root@mart:~/gisops_docker2# ls -ll 
custom_files/       docker-compose.yml  
root@mart:~/gisops_docker2# ls -ll custom_files/
total 6018068
drwxrwxr-x  2 root root       4096 Dec 14 11:00 admin_data
-rw-rw-r--  1 root root       1632 Dec 14 12:14 duplicateways.txt
drwxrwxr-x 45 root root       4096 Dec 14 12:28 elevation_data
-rw-rw-r--  1 root root         65 Dec 14 13:03 file_hashes.txt
-rw-rw-r--  1 root root 3197651446 Dec 14 10:52 russia-latest.osm.pbf
drwxrwxr-x  2 root root       4096 Dec 14 11:00 timezone_data
-rw-rw-r--  1 root root       6870 Dec 14 10:52 valhalla.json
drwxrwxr-x  5 root root       4096 Dec 14 13:03 valhalla_tiles
-rw-rw-r--  1 root root 2964807680 Dec 14 13:03 valhalla_tiles.tar
root@mart:~/gisops_docker2#

[nilsnolde]

jep, that's possibly because I made a mistake in yesterday's commit and I think there was never a new image for you to try. now there is though, after fixing #46 I realized the bug and rectified it. the new latest image is already on dockerhub. can you please try again @yashindmitry ? sorry for the hassle..

[yashindmitry]

Thank you very much for your help. Sorry, but I still can't get it:

version: '3.0'
services:
  valhalla:
    image: gisops/valhalla:latest
    ports:
      - "8002:8002"
    volumes:
      - ./custom_files/:/custom_files
    environment:
      # The tile_file must be located in the `custom_files` folder.
      # The tile_file has priority and is used when valid.
      # If the tile_file doesn't exist, the url is used instead.
      # Don't blank out tile_url when you use tile_file and vice versa.
      - tile_urls=http://download.geofabrik.de/russia-latest.osm.pbf
      # Get correct bounding box from e.g. https://boundingbox.klokantech.com/
      - min_x=-4 # -> Albania | -180 -> World
      - min_y=40 # -> Albania | -90  -> World
      - max_x=80 # -> Albania |  180 -> World
      - max_y=61 # -> Albania |  90  -> World
      - server_threads=30
      - use_tiles_ignore_pbf=True
      - force_rebuild=False
      - force_rebuild_elevation=False
      - build_elevation=False
      - build_admins=False
      - build_time_zones=False
      - build_tar=True # build an indexed tar file from the tile_dir for faster graph loading times

root@mart:~/gisops_docker3# docker-compose up --build
Creating network "gisops_docker3_default" with the default driver
Pulling valhalla (gisops/valhalla:latest)...
latest: Pulling from gisops/valhalla
7b1a6ab2e44d: Pull complete
a0b5f8e7de97: Pull complete
a2e0bc77c203: Pull complete
bd785ff04e67: Pull complete
0e8c381e8413: Pull complete
93c1aac31dfc: Pull complete
a0f12522528c: Pull complete
4f4fb700ef54: Pull complete
Digest: sha256:5e83862d1c6f7c12332758492320669cd9fa5960f5a0511c6ac8b9dec2f923e8
Status: Downloaded newer image for gisops/valhalla:latest
Creating gisops_docker3_valhalla_1 ... done
Attaching to gisops_docker3_valhalla_1
valhalla_1  | 
valhalla_1  | INFO: Running container with user valhalla UID 59999 and GID 59999.
valhalla_1  | 
valhalla_1  | WARNING: No routing tiles found, starting new build..
valhalla_1  | WARNING: No local files and no valhalla_tiles.tar found. Downloading links: http://download.geofabrik.de/russia-latest.osm.pbf!
valhalla_1  | 
valhalla_1  | ===============================================================
valhalla_1  |  Downloading  http://download.geofabrik.de/russia-latest.osm.pbf
valhalla_1  | ===============================================================
valhalla_1  |   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
valhalla_1  |                                  Dload  Upload   Total   Spent    Left  Speed

.....
.....
.....

valhalla_1  | 2021/12/14 18:18:37.217922 [INFO] Tile extract successfully loaded with tile count: 27551
valhalla_1  | 2021/12/14 18:18:37.217991 [WARN] (stat):  No such file or directory
valhalla_1  | 2021/12/14 18:18:37.218014 [WARN] Traffic tile extract could not be loaded
valhalla_1  | 2021/12/14 18:18:37.254359 [INFO] Tile extract successfully loaded with tile count: 27551
valhalla_1  | 2021/12/14 18:18:37.254432 [WARN] (stat):  No such file or directory
valhalla_1  | 2021/12/14 18:18:37.254438 [WARN] Traffic tile extract could not be loaded
valhalla_1  | terminate called after throwing an instance of 'std::runtime_error'
valhalla_1  | terminate called recursively
valhalla_1  |   what():  /custom_files/elevation_data/N65/N65E176.hgt(mmap): Cannot allocate memory
valhalla_1  | terminate called recursively
valhalla_1  | terminate called recursively
valhalla_1  | terminate called recursively
valhalla_1  | /valhalla/scripts/run.sh: line 9: 30988 Aborted                 sudo -E $1

Tell me, what am I doing wrong?

[TimMcCauley]

From what I can see your memory is not sufficient:

what(): /custom_files/elevation_data/N65/N65E176.hgt(mmap): Cannot allocate memory

Can you disable elevation and try again?

[yashindmitry]

I have 64 gigabytes of RAM, of which 58 are free. And the hard disk is more than 500 GB free. I did not set any limits on memory. Only for the number of processor cores (- server_threads = 30). Is it an error in RAM or hard disk space? How can this be diagnosed?

[TimMcCauley]

I assume this is memory. How much did you allocate to Docker?

[yashindmitry]

I have not limited memory for docker. Just installed it from the repository https://download.docker.com/linux/ubuntu After your last post, I tried adding mem_limit to config, but docker gave an error:

version: '3.0'
services:
  valhalla:
    image: gisops/valhalla:latest
    ports:
      - "8002:8002"
    mem_limit: 16384m

ERROR: The Compose file './docker-compose.yml' is invalid because:
Unsupported config option for services.valhalla: 'mem_limit'

Besides, I didn’t limit my memory in any way.

[TimMcCauley]

Please use version 2.4 - https://linuxhint.com/docker_compose_memory_limits/ and try again

[nilsnolde]

Hm that’s strange and there’s been an issue about that before.. It looks to me like you’re on Linux right? In that case there’d be no point fiddling with the memory settings of docker, that’s only important for osx & win where docker has to be virtualized.

I’ll look for the issue tmrw, not sure if it was resolved back then..

[yashindmitry]

It looks to me like you’re on Linux right?

Yes, I use ubuntu 20.10 and ubuntu 20.04

[nilsnolde]

now that I really look at your config: as I said before pls use the current docker-compose.yml. the way you're doing it you're still pulling in elevation since you're using old/deprecated env vars (the min_x etc), doesn't matter what build_elevation says in that case.

the error makes absolutely no sense to me though, no matter what's the config.. anyways, nuke the elevation folder if you don't need it. probably start from scratch entirely

[yashindmitry]

I start from scratch every time (the directories are different gisops_docker, gisops_docker2, gisops_docker3 ...). Now I will try in the new directory gisops_docker4 using your config with russia-latest.osm.pbf:

version: "3.0"
services:
  valhalla:
    image: gisops/valhalla:latest
    container_name: valhalla_latest
    ports:
      - 8002:8002
    #build:
    #  context: .
    #  args:
    #    - VALHALLA_UID=1000
    #    - VALHALLA_GID=1000
    volumes:
      - ./custom_files/:/custom_files
    environment:
      # Auto-download PBFs from Geofabrik
      # - "tile_urls=https://download.geofabrik.de/north-america/us/california-latest.osm.pbf https://download.geofabrik.de/europe/austria-latest.osm.pbf"
      - tile_urls=http://download.geofabrik.de/russia-latest.osm.pbf
      - server_threads=30 # determines how many threads will be used to run the valhalla server
      - use_tiles_ignore_pbf=True # load existing valhalla_tiles.tar directly
      - build_elevation=False # build elevation with "True" or "Force": will download only the elevation for areas covered by the graph tiles
      - build_admins=False # build admins db with "True" or "Force"
      - build_time_zones=False # build timezone db with "True" or "Force"
      - build_tar=True # build an indexed tar file from the tile_dir for faster graph loading times
      - force_rebuild=False # forces a rebuild of the routing tiles with "True"
      - path_extension=graphs # this path will be internally appended to /custom_files; no leading or trailing path separator!

as a result of the command "docker-compose up --build", I got the following:

...
valhalla_latest | 2021/12/15 11:02:10.834115 [WARN] Traffic tile extract could not be loaded
valhalla_latest | 2021/12/15 11:02:10.838155 [INFO] Tile extract successfully loaded with tile count: 27551
valhalla_latest | 2021/12/15 11:02:10.838238 [WARN] (stat):  No such file or directory
valhalla_latest | 2021/12/15 11:02:10.838248 [WARN] Traffic tile extract could not be loaded
valhalla_latest | 2021/12/15 11:02:10.884948 [INFO] Tile extract successfully loaded with tile count: 27551
valhalla_latest | 2021/12/15 11:02:10.885063 [WARN] (stat):  No such file or directory
valhalla_latest | 2021/12/15 11:02:10.885083 [WARN] Traffic tile extract could not be loaded
valhalla_latest | 2021/12/15 11:02:10.906261 [INFO] Tile extract successfully loaded with tile count: 27551
valhalla_latest | 2021/12/15 11:02:10.906343 [WARN] (stat):  No such file or directory
valhalla_latest | 2021/12/15 11:02:10.906351 [WARN] Traffic tile extract could not be loaded
valhalla_latest | 2021/12/15 11:02:10.927518 [INFO] Tile extract successfully loaded with tile count: 27551
valhalla_latest | 2021/12/15 11:02:10.927594 [WARN] (stat):  No such file or directory
valhalla_latest | 2021/12/15 11:02:10.927604 [WARN] Traffic tile extract could not be loaded
valhalla_latest | 2021/12/15 11:02:11.113971 [WARN] /custom_files/graphs/elevation_data currently has no elevation tiles
valhalla_latest | 2021/12/15 11:02:11.133647 [WARN] /custom_files/graphs/elevation_data currently has no elevation tiles
valhalla_latest | 2021/12/15 11:02:11.133913 [WARN] /custom_files/graphs/elevation_data currently has no elevation tiles
valhalla_latest | 2021/12/15 11:02:11.134578 [WARN] /custom_files/graphs/elevation_data currently has no elevation tiles
valhalla_latest | 2021/12/15 11:02:11.135768 [WARN] /custom_files/graphs/elevation_data currently has no elevation tiles
valhalla_latest | 2021/12/15 11:02:11.135877 [WARN] /custom_files/graphs/elevation_data currently has no elevation tiles
valhalla_latest | 2021/12/15 11:02:11.136346 [WARN] /custom_files/graphs/elevation_data currently has no elevation tiles
valhalla_latest | 2021/12/15 11:02:11.136680 [WARN] /custom_files/graphs/elevation_data currently has no elevation tiles
valhalla_latest | 2021/12/15 11:02:11.137926 [WARN] /custom_files/graphs/elevation_data currently has no elevation tiles
valhalla_latest | 2021/12/15 11:02:11.138448 [WARN] /custom_files/graphs/elevation_data currently has no elevation tiles
valhalla_latest | 2021/12/15 11:02:11.138612 [WARN] /custom_files/graphs/elevation_data currently has no elevation tiles
valhalla_latest | 2021/12/15 11:02:11.138784 [WARN] /custom_files/graphs/elevation_data currently has no elevation tiles
valhalla_latest | 2021/12/15 11:02:11.138924 [WARN] /custom_files/graphs/elevation_data currently has no elevation tiles
valhalla_latest | 2021/12/15 11:02:11.141413 [WARN] /custom_files/graphs/elevation_data currently has no elevation tiles
valhalla_latest | 2021/12/15 11:02:11.143622 [WARN] /custom_files/graphs/elevation_data currently has no elevation tiles
valhalla_latest | 2021/12/15 11:02:11.148903 [WARN] /custom_files/graphs/elevation_data currently has no elevation tiles
valhalla_latest | 2021/12/15 11:02:11.150604 [WARN] /custom_files/graphs/elevation_data currently has no elevation tiles
valhalla_latest | 2021/12/15 11:02:11.153014 [WARN] /custom_files/graphs/elevation_data currently has no elevation tiles
valhalla_latest | 2021/12/15 11:02:11.154242 [WARN] /custom_files/graphs/elevation_data currently has no elevation tiles
valhalla_latest | 2021/12/15 11:02:11.154607 [WARN] /custom_files/graphs/elevation_data currently has no elevation tiles
valhalla_latest | 2021/12/15 11:02:11.155681 [WARN] /custom_files/graphs/elevation_data currently has no elevation tiles
valhalla_latest | 2021/12/15 11:02:11.155713 [WARN] /custom_files/graphs/elevation_data currently has no elevation tiles
valhalla_latest | 2021/12/15 11:02:11.161329 [WARN] /custom_files/graphs/elevation_data currently has no elevation tiles
valhalla_latest | 2021/12/15 11:02:11.164797 [WARN] /custom_files/graphs/elevation_data currently has no elevation tiles
valhalla_latest | 2021/12/15 11:02:11.167096 [WARN] /custom_files/graphs/elevation_data currently has no elevation tiles
valhalla_latest | 2021/12/15 11:02:11.168405 [WARN] /custom_files/graphs/elevation_data currently has no elevation tiles
valhalla_latest | 2021/12/15 11:02:11.168961 [WARN] /custom_files/graphs/elevation_data currently has no elevation tiles
valhalla_latest | 2021/12/15 11:02:11.169672 [WARN] /custom_files/graphs/elevation_data currently has no elevation tiles
valhalla_latest | 2021/12/15 11:02:11.171115 [WARN] /custom_files/graphs/elevation_data currently has no elevation tiles
valhalla_latest | 2021/12/15 11:02:11.172417 [WARN] /custom_files/graphs/elevation_data currently has no elevation tiles

And then the process does not go, nothing happens

root@mart:~/gisops_docker4# ps aux | grep valhalla
59999     978757  0.0  0.0   3976  3100 ?        Ss   14:21   0:00 bash /valhalla/scripts/run.sh build_tiles
root     1009555  0.0  0.0   5000  3512 ?        S    15:02   0:00 sudo -E valhalla_service /custom_files/graphs/valhalla.json 30
root     1009556  3.2  1.5 189363696 1019720 ?   Sl   15:02   0:22 valhalla_service /custom_files/graphs/valhalla.json 30
root     1010377  0.0  0.0   5192  2416 pts/1    S+   15:13   0:00 grep --color=auto valhalla

[nilsnolde]

this is the process valhalla_service /custom_files/graphs/valhalla.json 30. well there's a few too many parent processes IMO, could use some execs I guess, but doesn't really matter..

don't understand what you expect to happen.. it's a daemon process.. you'll only know if it works if you actually request something..

[yashindmitry]

I'm sorry, I didn't know how this should work. So the process was successful. Thank you very much. It turns out that the exceptions from the parameter config min_x,min_y,max_x,max_y helped me. Thanks again.