First Run script broken on arm32v7 architecture

mikenye commented 4 years ago

Error is as follows:

$ docker run --rm -it --name=flightairmap -p 8077:80 -e TZ=America/Chicago -e BASESTATIONHOST=readsb -e FAM_INSTALLPASSWORD="very_secure_password_12345" mikenye/flightairmap:latest
Unable to find image 'mikenye/flightairmap:latest' locally
latest: Pulling from mikenye/flightairmap
f83522bf96d7: Already exists
61eb9f50a77b: Pull complete
bb0b29ce0263: Pull complete
Digest: sha256:5af65849de8417caca99b1890d2e6ecaea9debff0b7895185c5a07a98c511e1d
Status: Downloaded newer image for mikenye/flightairmap:latest
[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 01-sanitycheck: executing...
INFO: MYSQLHOSTNAME not set, using local database
[cont-init.d] 01-sanitycheck: exited 0.
[cont-init.d] 02-firstrun: executing...

New installation detected. Performing first-run tasks...
This takes a very long time, please be patient!

Starting temporary services...
Waiting for database...
.
Database ready.
Running install/index.php
 - Installation status: "database_import" / "Create and import tables"
 - Installation status: "populate" / "Populate aircraft_modes table with externals data for ADS-B"
 - Installation status: "populate_flarm" / "Populate aircraft_modes table with externals data for FLARM"
 - Installation status: "routes" / "Populate routes table with externals data"
parse error: Invalid numeric literal at line 1, column 7
parse error: Invalid numeric literal at line 1, column 7
parse error: Invalid numeric literal at line 1, column 7
parse error: Invalid numeric literal at line 1, column 7

[cont-init.d] 02-firstrun: exited 1.
[cont-finish.d] executing container finish scripts...
[cont-finish.d] done.
[s6-finish] waiting for services.
[s6-finish] sending all processes the TERM signal.
[s6-finish] sending all processes the KILL signal and exiting.

Links that may help:

mikenye commented 4 years ago

The problem is that the output from this command:

https://github.com/mikenye/docker-flightairmap/blob/01c3f6bacdbd9920bd96008ca27ca91561910911/etc/cont-init.d/02-firstrun#L263

...is expected to be JSON.

For some reason, seemingly at the end of the install process on arm32v7 platform, it returns an HTML error:

+ curl http://127.0.0.1/install/install-action.php --silent --cookie /tmp/install_process_cookies --cookie-jar /tmp/install_process_cookies -H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'Referer: http://127.0.0.1/install/index.php?246642094&next=database_create' --compressed -H 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive' --cookie-jar /tmp/install_process_cookies
+ echo ''

+ cat /tmp/install_process_status_json
<html>
<head><title>502 Bad Gateway</title></head>
<body bgcolor="white">
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx</center>
</body>
</html>

...which then causes issues on the following lines that use jq:

# Get info from json
  INSTALLERROR=$(jq ".error" < /tmp/install_process_status_json)
  INSTALLNEXT=$(jq ".next" < /tmp/install_process_status_json)
  INSTALLPHASE=$(jq ".install" < /tmp/install_process_status_json)

mikenye commented 4 years ago

Fixed

nRaecheR commented 4 years ago

Could it be that the bug is still present? I'm trying to set the docker container up on a AMD64 host. This is the log output:

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 01-sanitycheck: executing... 
[cont-init.d] 01-sanitycheck: exited 0.
[cont-init.d] 02-firstrun: executing... 
Waiting for database...

Database ready.

New installation detected. Performing first-run tasks...
This takes a very long time, please be patient!

Starting temporary services... 
Running install/index.php
parse error: Invalid numeric literal at line 1, column 5
parse error: Invalid numeric literal at line 1, column 5
parse error: Invalid numeric literal at line 1, column 5
parse error: Invalid numeric literal at line 1, column 5

[cont-init.d] 02-firstrun: exited 1.
[cont-finish.d] executing container finish scripts...
[cont-finish.d] done.
[s6-finish] waiting for services.
[s6-finish] sending all processes the TERM signal.
[s6-finish] sending all processes the KILL signal and exiting.

Tried "development" and "latest" tags....

nRaecheR commented 4 years ago

I've enabled VERBOSE_LOGGING and find that problem:

+ date
Wed Oct  7 11:01:49 CEST 2020
+ curl http://127.0.0.1/install/install-action.php --verbose --progress-bar --cookie /tmp/install_process_cookies --cookie-jar /tmp/install_process_cookies -H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'Referer: http://127.0.0.1/install/' --compressed -H 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive' --cookie-jar /tmp/install_process_cookies
* Expire in 0 ms for 6 (transfer 0x5654b4c18e80)
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x5654b4c18e80)
* Connected to 127.0.0.1 (127.0.0.1) port 80 (#0)
> GET /install/install-action.php HTTP/1.1
> Host: 127.0.0.1
> User-Agent: curl/7.64.0
> Accept-Encoding: deflate, gzip
> Accept: application/json, text/javascript, */*; q=0.01
> Referer: http://127.0.0.1/install/
> X-Requested-With: XMLHttpRequest
> Connection: keep-alive
> 
< HTTP/1.1 404 Not Found
< Server: nginx
< Date: Wed, 07 Oct 2020 09:01:49 GMT
< Content-Type: text/html; charset=UTF-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< 
{ [27 bytes data]
* Connection #0 to host 127.0.0.1 left intact

+ date
Wed Oct  7 11:01:49 CEST 2020
+ echo 'Response JSON:'
+ cat /tmp/install_process_status_json
Response JSON:
File not found.
+ echo 'End response JSON.'
End response JSON.
++ jq .error
parse error: Invalid numeric literal at line 1, column 5
+ INSTALLERROR=
++ jq .next
parse error: Invalid numeric literal at line 1, column 5
+ INSTALLNEXT=
++ jq .install
parse error: Invalid numeric literal at line 1, column 5
+ INSTALLPHASE=
+ '[' '' = '""' ']'
+ jq .
parse error: Invalid numeric literal at line 1, column 5

It seems that the install-action.php isn't there or not served.

nRaecheR commented 4 years ago

Okay, I was able to "fix" it myself. The problem is that the following line :

  -v flightairmap_webapp:/var/www/flightairmap \

It prevents the access to the webapp from the docker image itself, so nginx won't find the files to serve because the volume is empty at first start. I've just deleted the line and now it seems to work. Haven't tried an update to the container though.

mikenye commented 4 years ago

Was the flightairmap_webapp volume left over from a previous deployment? I'm wondering if it was a permissions issue?

nRaecheR commented 4 years ago

No, I've mapped an empty directory as data volume and it stayed empty.

mikenye commented 4 years ago

OK, I'm just running through the first-run process on a local install using an empty directory for the webapp path to try and recreate the same problem. Will let you know what I find.

mikenye commented 4 years ago

This works as expected:

docker volume create flightairmap_db
docker volume create flightairmap_webapp
docker run -d \
    --name=flightairmap \
    -p 8080:80 \
    -e BASESTATIONHOST=readsb \
    -e TZ=Australia/Perth \
    -e FAM_INSTALLPASSWORD="very_secure_password_12345" \
    -v flightairmap_db:/var/lib/mysql \
    -v flightairmap_webapp:/var/www/flightairmap \
    mikenye/flightairmap

This does not work:

docker run -d \
    --name=flightairmap \
    -p 8080:80 \
    -e BASESTATIONHOST=readsb \
    -e TZ=Australia/Perth \
    -e FAM_INSTALLPASSWORD="very_secure_password_12345" \
    -v /opt/flightairmap/db:/var/lib/mysql \
    -v /opt/flightairmap/webapp:/var/www/flightairmap \
    mikenye/flightairmap

Looking into this, it is discussed in more detail here: https://github.com/moby/moby/issues/17470, but the short version is:

The reason is that when using a "bind mounted" directory from the host, you're telling docker that you want to take a file or directory from your host and use it in your container. Docker should not modify those files/directories, unless you explicitly do so. For example, you don't want -v /home/user/:/var/lib/mysql to result in your home-directory being replaced with a MySQL database.

Did your -v command actually point to a docker volume created with docker volume create?

nRaecheR commented 4 years ago

No, I'm always using -v with directories and not with data volumes. I haven't got any problems with multiple containers and I'm puzzled why this be a problem here. Why is the data container needed for the WebApp? As far as I understand it, all the data goes into the database and any setup configuration will be passed as environment variables. What will be stored in the data container?

mikenye commented 4 years ago

I haven't got any problems with multiple containers and I'm puzzled why this be a problem here.

This is because the image has data in /var/www/flightairmap from the get-go. When you create the container using a directory (a bind mount) instead of a docker volume, the contents of /var/www/flightairmap isn't copied over to your directory. This then causes the container to fail as it is missing files.

Other images (that you're not havinig a problem with) likely don't have anything in the directory you're mapping to - as is the case for /var/lib/mysql in this flightairmap container.

Why is the data container needed for the WebApp?

By "data container" I'm assuming you mean "docker volume". This is because of what I quoted in my previous message:

The reason is that when using a "bind mounted" directory from the host, you're telling docker that you want to take a file or directory from your host and use it in your container. Docker should not modify those files/directories, unless you explicitly do so. For example, you don't want -v /home/user/:/var/lib/mysql to result in your home-directory being replaced with a MySQL database.

The flightairmap image already contains data within /var/www/flightairmap, so docker won't copy it over to the directory you specify.

As far as I understand it, all the data goes into the database and any setup configuration will be passed as environment variables.

Ordinarily this would be the case. However, flightairmap downloads a whole bunch of stuff on first run and also on a regular schedule. You can see this taking place during first run if you keep an eye on the container logs. For example:

Models from FlightAirMap website : Download...Check files...
Downloading model 320.glb ...
Downloading model 320.gltf ...
Downloading model 321.glb ...
Downloading model 330.glb ...
Downloading model 340.glb ...
Downloading model 707.glb ...
Downloading model 727.glb ...

...and so on.

Also, if we take a look at the number of files and disk usage of the /var/www/flightairmap folder before and aftere firstrun:

Before firstrun:

# Disk space consumed by /var/www/flightairmap:
root@34ae9f0a5bad:/var/www/flightairmap# du -hs .
194M    .

# Number of files under /var/www/flightairmap:
root@34ae9f0a5bad:/var/www/flightairmap# find . -type f | wc -l
6040

After firstrun:

# Disk space consumed by /var/www/flightairmap:
root@ec2d8f16b199:/var/www/flightairmap# du -hs
707M    .

# Number of files under /var/www/flightairmap:
root@ec2d8f16b199:/var/www/flightairmap# find . -type f | wc -l
6263

As you can see, flightairmap has downloaded ~500MB of additional files during firstrun.

Also, some of the app's settings are in /var/www/flightairmap/htdocs/require/settings.php, so any configuration changes (that are stored in this file) performed outside of container environment variables may be lost.

What will be stored in the data container?

The contents of the container's /var/www/flightairmap directory.

If you want to see what's in here, we can start a temporary instance of the container and bypass the normal startup by setting a different entrypoint:

$ docker run --rm -it --entrypoint bash mikenye/flightairmap
root@80ac6733e086:/# cd /var/www/flightairmap
root@80ac6733e086:/var/www/flightairmap# ls
htdocs

...and so on.

If you're concerned with losing the visibility of your data when using docker volumes, don't be.

You can inspect a volume to see exactly where docker is storing the data within the volume, and you can then browse the directory structure (as root):

$ docker volume ls | grep flightairmap
local                 flightairmap_db
local                 flightairmap_webapp

$ docker volume inspect flightairmap_webapp
[
    {
        "CreatedAt": "2020-10-08T19:40:45+08:00",
        "Driver": "local",
        "Labels": null,
        "Mountpoint": "/var/lib/docker/volumes/flightairmap_webapp/_data",
        "Name": "flightairmap_webapp",
        "Options": null,
        "Scope": "local"
    }
]

$ sudo ls /var/lib/docker/volumes/flightairmap_webapp/_data
htdocs

If you want your data to be at a specific place within your filesystem (and not hidden away under /var/lib/docker), you can perform a docker volume create with some specific options:

$ docker volume create --opt type=none --opt device=/opt/flightairmap/webapp --opt o=bind flightairmap_webapp
flightairmap_webapp

$ docker volume inspect flightairmap_webapp
[
    {
        "CreatedAt": "2020-10-08T20:30:57+08:00",
        "Driver": "local",
        "Labels": {},
        "Mountpoint": "/var/lib/docker/volumes/flightairmap_webapp/_data",
        "Name": "flightairmap_webapp",
        "Options": {
            "device": "/opt/flightairmap/webapp",
            "o": "bind",
            "type": "none"
        },
        "Scope": "local"
    }
]

nRaecheR commented 4 years ago

mikenye, thank you for your help. I've migrated the four changed directories with the settings and downloaded data to a docker data volume and now it works with this one too.

I really don't like data volumes not only because of the lack of visibility of the data files but also because they make backups and change tracking very difficult. And I must confess that I wasn't aware of the differences between bind mounts and data volumes related to the overlay nature of it. The flightairmap container needs this to overlay the changed/downloaded files over the files from the container. This is something I've never seen before.

But now it works, thank you very much for your help and your work here!

mikenye commented 4 years ago

I fully understand. The flightairmap container was a challenging one to put together. Instead of having to use volumes, I could've had a firstrun task to move the contents of /var/www/flightairmap into place, however docker volumes does this automatically so this is the directon I went down.

Anyway, I'm glad you've got it working. :-)

sdr-enthusiasts / docker-flightairmap

First Run script broken on arm32v7 architecture #8