tileserver-gl-light working locally, but not in docker

[kochis]

Hello,

When starting the locally installed (npm install -g tileserver-gl-light) version, it starts up and serves the tiles correctly

$ tileserver-gl-light
Starting tileserver-gl-light v4.4.7
No MBTiles specified, using osm-2023-03-03.mbtiles
[INFO] Automatically creating config file for osm-2023-03-03.mbtiles
[INFO] Only a basic preview style will be used.
[INFO] See documentation to learn how to create config.json file.
Run with --verbose to see the config file here.
Starting server
Listening at http://[::]:8080/
Startup complete
GET /data/v3/1/0/0.pbf 200 7.150 ms - 87765
GET /data/v3/1/1/0.pbf 200 7.122 ms - 105995
GET /data/v3/1/1/1.pbf 200 7.733 ms - 84153
GET /data/v3/1/0/1.pbf 200 8.468 ms - 82323

However, when starting up via docker, I'm getting 500's for serving vector data. I was able to add some additional logging, and looks like its an SQLite access error

$ docker run --rm -it -v $(pwd):/data -p 8080:8080 maptiler/tileserver-gl-light
Starting tileserver-gl-light v4.4.8
No MBTiles specified, using osm-2023-03-03.mbtiles
[INFO] Automatically creating config file for osm-2023-03-03.mbtiles
[INFO] Only a basic preview style will be used.
[INFO] See documentation to learn how to create config.json file.
Starting server
Listening at http://[::]:8080/
Startup complete
GET /data/v3/1/0/0.pbf 500 2.816 ms - 28
[Error: SQLITE_IOERR: disk I/O error] {
  errno: 10,
  code: 'SQLITE_IOERR'
}
GET /data/v3/1/0/1.pbf 500 4.362 ms - 28
[Error: SQLITE_IOERR: disk I/O error] {
  errno: 10,
  code: 'SQLITE_IOERR'
}

I don't think there's any issue with the .mbtiles file since it works correctly when running locally.

I am running on MacOS with ARM64, so I wonder if something there might be the issue. Just wondering if anyone else has come across this before, or has an idea what might be the issue here.

Thanks.

System Info

MacOS Ventura 13.1 M1 Max Docker 4.15

[acalcutt]

Does the non-light version have the same issue?

[acalcutt]

One thing to note is you are using a older version in the first test. The repo version should be v4.4.8 also, but your first test is using v4.4.7. If you do a 'git pull' to update it does it then have the same issue as the second test?

[kochis]

tileserver-gl-light@4.4.8 also works fine while running locally.

Non-light has the same issue (local version works, docker version does not).

[acalcutt]

Do you know what the last version of that worked for you was (in the light version)

[kochis]

I haven't been able to get it to work with docker on my local machine yet (I'm also just trying this for the first time).

My suspicion is this is an ARM64 thing. Going to try and deploy this in a linux environment and see if that resolves it.

[acalcutt]

It seems to me more like a permissions issue. When running inside docker it runs as a "node" user, so I'd imagine if the permissions don't allow that user to read the mbtiles files, you may get an error like that.

The reason I am asking about older versions is we just made a few changes to the dockerfile. The light version got changed in v4.4.8 and the full version was changed in v4.4.7, so if it worked before those versions it could be related to those changes.

Arm64 should not matter. The full version should work with arm64 on Ubuntu 20.02 and MacOS 12. The Light version does not use maplibre-native, so it shouldn't matter if it runs on arm64 (the requirements of the light version are pretty minimal). As far as I am aware arm64 support has been working since v4.3.2

[kochis]

Ah interesting, yeah let me see if one of the older docker versions works.

[kochis]

I'm getting the same error on all docker versions (went all the way down to v4.4.0)

[acalcutt]

So not the recent changes then.

It still sounds like permissions to me. What permissions are set on the mbtiles file? I'm assuming it would need something like 644 (read for everyone) for the node user inside the docker to be able to read the file.

When running without the docker, it is using whatever user you are logged in as, so it is a little different permissions wise.

[kochis]

Yeah, it does seem like it can't read the file for some reason. I did just set it to 777 just to see, and still can't access it

node@cb0515ed9e2c:/data$ ls -al
total 81297668
drwxr-xr-x 3 node node          96 Mar  6 19:31 .
drwxr-xr-x 1 root root        4096 Mar  6 21:13 ..
-rwxrwxrwx 1 node node 83238203392 Mar  3 18:21 osm-2023-03-03.mbtiles

[acalcutt]

If you use a different mbtiles file does it make a difference?

wget https://github.com/maptiler/tileserver-gl/releases/download/v1.3.0/zurich_switzerland.mbtiles
docker run --rm -it -v $(pwd):/data -p 8080:8080 maptiler/tileserver-gl --mbtiles zurich_switzerland.mbtiles

[kochis]

Interesting, I just tried with tileserver-gl:v4.4.0 (non-light) and it worked 🤷

[kochis]

I'm also getting a mix of 200's and 500's, so maybe something is off with the file. I'm gonna add the logging back in to see if it's still the same sqlite error.

GET /data/v3/12/1238/1514.pbf 500 28 - 15.154 ms
GET /data/v3/12/1240/1514.pbf 500 28 - 17.316 ms
GET /data/v3/12/1238/1515.pbf 500 28 - 2.412 ms
GET /data/v3/13/2477/3030.pbf 200 80822 - 1.719 ms
GET /data/v3/13/2480/3030.pbf 200 9097 - 3.091 ms
GET /data/v3/13/2477/3029.pbf 200 79356 - 3.600 ms
GET /data/v3/13/2481/3029.pbf 200 7247 - 4.395 ms

Oddly I'm not seeing those 500's when just running locally

[kochis]

Alright, I'm not sure entirely what the issue is, but it must be something about reading the sqlite file through the volume mount.

I was able to copy the file into the image during docker build and that seemed to work fine. I wonder if this is a Mac thing, or maybe something with the file I'm using. As long as this isn't an issue in linux I think it should be fine.

Happy to try and debug more, but not sure what next things to try.

[acalcutt]

Works for me here on Ubuntu 20.04/Docker version 20.10.18 with my planetiler generated osm file.

tserver@mapgen2:/opt/data_test$ docker run --rm -it -v $(pwd):/data -p 8080:8080 maptiler/tileserver-gl-light:v4.4.8
Unable to find image 'maptiler/tileserver-gl-light:v4.4.8' locally
v4.4.8: Pulling from maptiler/tileserver-gl-light
Digest: sha256:d287bd14e10a4ad9b8b0b9612630726e4285641092d0c9916bdb56cffdcdd15a
Status: Downloaded newer image for maptiler/tileserver-gl-light:v4.4.8
Starting tileserver-gl-light v4.4.8
No MBTiles specified, using output.mbtiles
[INFO] Automatically creating config file for output.mbtiles
[INFO] Only a basic preview style will be used.
[INFO] See documentation to learn how to create config.json file.
Run with --verbose to see the config file here.
Starting server
Listening at http://[::]:8080/
Startup complete
GET / 304 - - 42.636 ms
GET /index.css 304 - - 1.293 ms
GET /images/logo.png 304 - - 0.403 ms
GET /images/placeholder.png 304 - - 0.938 ms
GET /images/maptiler-logo.svg 304 - - 0.656 ms
GET /images/header-map-1280px.png 304 - - 0.556 ms
GET /fonts/OpenSans-Bold.ttf 304 - - 0.926 ms
GET /fonts/OpenSans-Regular.ttf 304 - - 0.791 ms
GET /health 200 2 - 1.019 ms
GET /data/v3/ 304 - - 12.363 ms
GET /maplibre-gl.css 304 - - 0.495 ms
GET /maplibre-gl-inspect.css 304 - - 0.472 ms
GET /maplibre-gl.js 304 - - 0.340 ms
GET /maplibre-gl-inspect.min.js 304 - - 0.665 ms
GET /data/v3.json 304 - - 1.618 ms
GET /data/v3/1/1/1.pbf 200 88628 - 60.030 ms
GET /data/v3/1/0/1.pbf 200 86637 - 58.201 ms
GET /data/v3/1/0/0.pbf 200 91920 - 74.273 ms
GET /data/v3/1/1/0.pbf 200 110317 - 74.384 ms
GET /health 200 2 - 0.490 ms
^CCaught signal SIGINT, stopping gracefully
tserver@mapgen2:/opt/data_test$

I ran as a non-root users and the file is 644 permission it looks like

tserver@mapgen2:/opt/data_test$ ls -l
total 80753584
-rw-r--r-- 1 root root 82691657728 Oct  6 17:54 output.mbtiles
tserver@mapgen2:/opt/data_test$ docker run --entrypoint /bin/bash --rm -it -v $(pwd):/data -p 8080:8080 maptiler/tileserver-gl-light:v4.4.8
node@d716870e80b2:/data$ ls -l
total 80753584
-rw-r--r-- 1 root root 82691657728 Oct  6 21:54 output.mbtiles
node@d716870e80b2:/data$

[acalcutt]

I tested this on a amazon t4g.small arm64 Ubuntu 20.04 and it seemed to work fine there too

ubuntu@ip-172-31-83-46:~$ uname -a
Linux ip-172-31-83-46 5.15.0-1028-aws #32~20.04.1-Ubuntu SMP Mon Jan 9 18:02:12 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux
ubuntu@ip-172-31-83-46:~$ docker run --rm -it -v $(pwd):/data -p 443:8080 maptiler/tileserver-gl-light --mbtiles zurich_switzerland.mbtiles
Unable to find image 'maptiler/tileserver-gl-light:latest' locally
latest: Pulling from maptiler/tileserver-gl-light
455c32d33260: Already exists
ecbb47d6397c: Pull complete
e7752554baf7: Pull complete
4f4fb700ef54: Pull complete
d078c5dbb742: Pull complete
c12406d5e61f: Pull complete
Digest: sha256:d287bd14e10a4ad9b8b0b9612630726e4285641092d0c9916bdb56cffdcdd15a
Status: Downloaded newer image for maptiler/tileserver-gl-light:latest
Starting tileserver-gl-light v4.4.8
[INFO] Automatically creating config file for zurich_switzerland.mbtiles
[INFO] Only a basic preview style will be used.
[INFO] See documentation to learn how to create config.json file.
Run with --verbose to see the config file here.
Starting server
Listening at http://[::]:8080/
Startup complete
GET /data/v3/ 304 - - 37.740 ms
GET /maplibre-gl.css 200 70412 - 4.298 ms
GET /maplibre-gl-inspect.css 200 1877 - 1.326 ms
GET /maplibre-gl.js 200 745464 - 1.194 ms
GET /maplibre-gl-inspect.min.js 200 23455 - 1.144 ms
GET /data/v3.json 304 - - 2.511 ms
GET /favicon.ico 200 5430 - 1.109 ms
GET / 200 2290 - 64.841 ms
GET /index.css 200 3320 - 1.025 ms
GET /images/logo.png 200 6407 - 1.445 ms
GET /images/placeholder.png 200 2084 - 0.810 ms
GET /images/maptiler-logo.svg 200 5136 - 2.030 ms
GET /fonts/OpenSans-Bold.ttf 200 224592 - 1.058 ms
GET /images/header-map-1280px.png 200 174969 - 2.049 ms
GET /fonts/OpenSans-Regular.ttf 200 217360 - 1.312 ms
GET /data/v3/ 304 - - 1.721 ms
GET /maplibre-gl.css 304 - - 0.867 ms
GET /maplibre-gl-inspect.css 304 - - 0.850 ms
GET /maplibre-gl.js 304 - - 0.470 ms
GET /maplibre-gl-inspect.min.js 304 - - 0.489 ms
GET /data/v3.json 304 - - 0.908 ms
GET /health 200 2 - 0.740 ms

[kochis]

I just tried running the tileserver-gl-light on amazon linux, and got the following error

$ docker run --rm -it -v $(pwd):/data -p 80:80 maptiler/tileserver-gl-light
Starting tileserver-gl-light v4.4.10
node:internal/fs/utils:347
    throw err;
    ^

Error: EACCES: permission denied, stat 'planet.mbtiles'
    at Object.statSync (node:fs:1615:3)
    at file:///usr/src/app/src/main.js:180:35
    at FSReqCallback.oncomplete (node:fs:208:21) {
  errno: -13,
  syscall: 'stat',
  code: 'EACCES',
  path: 'planet.mbtiles'
}

Node.js v18.15.0

I've given the file full permissions, and for some reason it still cant access:

ls -al planet.mbtiles
-rwxrwxrwx 1 ec2-user ec2-user 54776152064 Aug 31  2017 planet.mbtiles

FWIW, tileserver-gl works fine, just not tileserver-gl-light.

$ docker run --rm -it -v $(pwd):/data -p 80:80 maptiler/tileserver-gl
Starting tileserver-gl v3.1.1
No MBTiles specified, using planet.mbtiles
[INFO] Automatically creating config file for planet.mbtiles
[INFO] Only a basic preview style will be used.
[INFO] See documentation to learn how to create config.json file.
Run with --verbose to see the config file here.
Starting server
Listening at http://[::]:80/
Startup complete

(just realized that's running a much older version)

[kochis]

Looks like moving to an older version worked:

Starting tileserver-gl-light v4.0.0
No MBTiles specified, using planet.mbtiles
[INFO] Automatically creating config file for planet.mbtiles
[INFO] Only a basic preview style will be used.
[INFO] See documentation to learn how to create config.json file.
Run with --verbose to see the config file here.
Starting server
Listening at http://[::]:80/
Startup complete