Closed jaydrogers closed 1 month ago
skarnet
commented
1 month ago Two questions:
daemon off option? (you should)S6_VERBOSITY? 1 isn't ideal for debugging 😉
jaydrogers
commented
1 month ago Thanks for chiming in!
Are you running nginx with the daemon off option? (you should)
My run script calls nginx but I do have daemon off; set in the NGINX conf.
What messages do you get when running your container with a higher value for S6_VERBOSITY? 1 isn't ideal for debugging 😉
Thanks for the tip! I forgot about that.
Running this:
docker run --rm -e S6_VERBOSITY=5 serversideup/php:beta-8.3-fpm-nginxThis is my output:
It still takes 5 seconds for the container to shutdown and it's showing a 111 exit code.
jaydrogers
commented
1 month ago I see my "one off" scripts running as services. I wonder if I need an explicit exit 0 in my script so S6 Overlay does not try to monitor my scripts?
docker run --rm --entrypoint='' serversideup/php:beta-8.3-fpm-nginx ls -al /etc/s6-overlay/scripts/These are scripts that run before the services start. For example, here is the 50-laravel-automations script:
#!/command/with-contenv sh
script_name="laravel-automations"
test_db_connection() {
php -r "
require '$APP_BASE_DIR/vendor/autoload.php';
use Illuminate\Support\Facades\DB;
\$app = require_once '$APP_BASE_DIR/bootstrap/app.php';
\$kernel = \$app->make(Illuminate\Contracts\Console\Kernel::class);
\$kernel->bootstrap();
\$driver = DB::getDriverName();
if( \$driver === 'sqlite' ){
echo 'SQLite detected';
exit(0); // Assume SQLite is always ready
}
try {
DB::connection()->getPdo(); // Attempt to get PDO instance
if (DB::connection()->getDatabaseName()) {
exit(0); // Database exists and can be connected to, exit with status 0 (success)
} else {
echo 'Database name not found.';
exit(1); // Database name not found, exit with status 1 (failure)
}
} catch (Exception \$e) {
echo 'Database connection error: ' . \$e->getMessage();
exit(1); // Connection error, exit with status 1 (failure)
}
"
}
# Set default values for Laravel automations
: "${AUTORUN_ENABLED:=false}"
: "${AUTORUN_LARAVEL_MIGRATION_TIMEOUT:=30}"
if [ "$DISABLE_DEFAULT_CONFIG" = "false" ]; then
# Check to see if an Artisan file exists and assume it means Laravel is configured.
if [ -f "$APP_BASE_DIR/artisan" ] && [ "$AUTORUN_ENABLED" = "true" ]; then
echo "Checking for Laravel automations..."
############################################################################
# artisan migrate
############################################################################
if [ "${AUTORUN_LARAVEL_MIGRATION:=true}" = "true" ]; then
count=0
timeout=$AUTORUN_LARAVEL_MIGRATION_TIMEOUT
echo "🚀 Clearing Laravel cache before attempting migrations..."
php "$APP_BASE_DIR/artisan" config:clear
while [ $count -lt "$timeout" ]; do
test_db_connection > /dev/null 2>&1
status=$?
if [ $status -eq 0 ]; then
echo "✅ Database connection successful."
break
else
echo "Waiting on database connection, retrying... $((timeout - count)) seconds left"
count=$((count + 1))
sleep 1
fi
done
if [ $count -eq "$timeout" ]; then
echo "Database connection failed after multiple attempts."
exit 1
fi
echo "🚀 Running migrations..."
if [ "${AUTORUN_LARAVEL_MIGRATION_ISOLATION:=false}" = "true" ]; then
php "$APP_BASE_DIR/artisan" migrate --force --isolated
else
php "$APP_BASE_DIR/artisan" migrate --force
fi
fi
############################################################################
# artisan storage:link
############################################################################
if [ "${AUTORUN_LARAVEL_STORAGE_LINK:=true}" = "true" ]; then
if [ -d "$APP_BASE_DIR/public/storage" ]; then
echo "✅ Storage already linked..."
else
echo "🔐 Linking the storage..."
php "$APP_BASE_DIR/artisan" storage:link
fi
fi
############################################################################
# artisan config:cache
############################################################################
if [ "${AUTORUN_LARAVEL_CONFIG_CACHE:=true}" = "true" ]; then
echo "🚀 Caching Laravel config..."
php "$APP_BASE_DIR/artisan" config:cache
fi
############################################################################
# artisan route:cache
############################################################################
if [ "${AUTORUN_LARAVEL_ROUTE_CACHE:=true}" = "true" ]; then
echo "🚀 Caching Laravel routes..."
php "$APP_BASE_DIR/artisan" route:cache
fi
############################################################################
# artisan view:cache
############################################################################
if [ "${AUTORUN_LARAVEL_VIEW_CACHE:=true}" = "true" ]; then
echo "🚀 Caching Laravel views..."
php "$APP_BASE_DIR/artisan" view:cache
fi
############################################################################
# artisan event:cache
############################################################################
if [ "${AUTORUN_LARAVEL_EVENT_CACHE:=true}" = "true" ]; then
echo "🚀 Caching Laravel events..."
php "$APP_BASE_DIR/artisan" event:cache
fi
fi
else
if [ "$LOG_OUTPUT_LEVEL" = "debug" ]; then
echo "👉 $script_name: DISABLE_DEFAULT_CONFIG does not equal 'false', so automations will NOT be performed."
fi
fiDo I need an exit 0 so I don't see this message anymore?
s6-rc: info: service 50-laravel-automations successfully stoppedI have a few additional questions to help orient my diagnosis:
ENV S6_KILL_GRACETIME=1?S6_KILL_GRACETIME, but with ENV S6_KILL_FINISH_MAXTIME=1 instead?/etc/s6-overlay/s6-rc.d, in particular the user bundle, constructed?I can see your nginx service all right but there's something more to it. The scripts you put in /etc/s6-overlay/scripts should not be run by s6-rc; the only way they'll be started and stopped like this is if you somehow make service definition directories out of them in /etc/s6-overlay/s6-rc.d and add them to the user bundle, which I can't see anywhere in your repository.
jaydrogers
commented
1 month ago Thanks for asking more! I greatly appreciate your help.
These are public images that I'm using, so feel free to run these same commands on your local machine if you'd like.
Command:
docker run --rm -e S6_KILL_GRACETIME=1 -e S6_VERBOSITY=5 serversideup/php:beta-8.3-fpm-nginxResults:
Output:
Command:
docker run --rm -e S6_KILL_FINISH_MAXTIME=1 -e S6_VERBOSITY=5 serversideup/php:beta-8.3-fpm-nginxResults:
Output:
Don't kill me on this one 😃
I had a very special use case for my project where I needed my entrypoint scripts to run in non-S6 variations AND s6-variations.
My solution was creating a script that I call at build time (docker-php-serversideup-s6-init) that copies my basic entrypoint scripts and puts them into the S6 Overlay structure as one-shot scripts. This allows me to keep a single source of truth in my repo.
If you're interested in exploring the structure, you can inspect it yourself if you'd like:
docker run --rm --entrypoint='' --workdir=/etc/s6-overlay --user root -it serversideup/php:beta-8.3-fpm-nginx bash(In the command above, I switch you to the root user since the containers are unprivileged by default. Since S6 wants to start FPM and there is a weird config FPM needs to run as root, I just disable S6 from running with setting the entrypoint to null, but you can browse around easily)
/etc/s6-overlay/etc/s6-overlay/
|-- s6-rc.d
| |-- 0-container-info
| | |-- type
| | `-- up
| |-- 1-debug-mode
| | |-- dependencies
| | |-- type
| | `-- up
| |-- 10-init-webserver-config
| | |-- dependencies
| | |-- type
| | `-- up
| |-- 50-laravel-automations
| | |-- dependencies
| | |-- type
| | `-- up
| |-- nginx
| | |-- data
| | | `-- check
| | |-- dependencies
| | |-- notification-fd
| | |-- run
| | `-- type
| |-- php-fpm
| | |-- data
| | | `-- check
| | |-- dependencies
| | |-- notification-fd
| | |-- run
| | `-- type
| |-- user
| | |-- contents.d
| | | |-- 0-container-info
| | | |-- 1-debug-mode
| | | |-- 10-init-webserver-config
| | | |-- 50-laravel-automations
| | | |-- nginx
| | | `-- php-fpm
| | `-- type
| `-- user2
| |-- contents.d
| `-- type
`-- scripts
|-- 0-container-info -> /etc/entrypoint.d/0-container-info.sh
|-- 1-debug-mode -> /etc/entrypoint.d/1-debug-mode.sh
|-- 10-init-webserver-config -> /etc/entrypoint.d/10-init-webserver-config.sh
`-- 50-laravel-automations -> /etc/entrypoint.d/50-laravel-automations.shOK, so here's what's happening:
S6_KILL_GRACETIME to 1 (or 0, it should work) to avoid an unnecessary wait.S6_KILL_GRACETIME is 3000, i.e. 3 seconds. Not 5 seconds. So one of the mysteries is: why are you experiencing a ~5 second wait instead of a 3 second one? Are you using the latest version of s6-overlay?docker stop instead of Ctrl-C?jaydrogers
commented
1 month ago No problem! I greatly appreciate your efforts to help with a limited toolset available.
Yes, I am using v3.2.0.0.
The S6 env settings I have set in the container are:
S6_BEHAVIOUR_IF_STAGE2_FAILS=2
S6_CMD_WAIT_FOR_SERVICES_MAXTIME=0
S6_KEEP_ENV=1
S6_VERBOSITY=1So this is where it gets more interesting. I was going to mention that most times it takes 10 seconds for the container to exit (specifically with Docker Compose).
docker stopdocker run --rm --name=s6-php serversideup/php:beta-8.3-fpm-nginxdocker stop s6-phpI am getting very weird results.
With my version of Docker - Docker version 27.0.3, build 7d4bcd863a4c863e650eed02a550dfeb98560b83 - when I ^C the docker run invocation, I get context canceled and I cannot see the outputs. I'll keep investigating what's happening there, but apparently they changed the behaviour of docker run. I can say, however, that s6-svscan correctly gets a SIGINT, so everything is working properly on that front and it's just the exit code that needs to be fixed (but the change of behaviour interferes with my ability to debug).
However, when I docker stop, s6-svscan is getting a SIGQUIT. Is there something in your scripts or various configurations that changes the STOPSIGNAL for Docker? I can see no STOPSIGNAL directive in the Dockerfile I run (the php-fpm one with s6-overlay), so what I should be seeing is a SIGTERM, which would work as expected. A SIGQUIT will not work, so please check that nothing is messing with the STOPSIGNAL.
jaydrogers
commented
1 month ago Thanks for researching this more!
Is there something in your scripts or various configurations that changes the STOPSIGNAL for Docker?
Since I base my images off of PHP's official images, they do have a line to set the STOPSIGNAL: https://github.com/docker-library/php/blob/f5a51b67da6ae457bcf89238f5cf0bf73444a760/8.3/bookworm/fpm/Dockerfile#L270-L272
Should I change this in my fpm-nginx (with S6 Overlay) image since I'm using a totally different init system compared to the official PHP image?
I can set STOPSIGNAL in the Dockerfile: https://docs.docker.com/reference/dockerfile/#stopsignal
skarnet
commented
1 month ago Yes. The PHP-FPM Dockerfile is meant for a container where the php-fpm process itself runs as process 1, alone in the container, so they adapt the stop signal to accommodate php-fpm. But when you're running a container with an init system (s6-overlay being only one of many), then you should use the default stop signal, i.e. SIGTERM. If you do so, you'll be able to get the expected results from docker stop.
jaydrogers
commented
1 month ago That worked! Wow, what a stack of code to dive into.
I greatly appreciate your help on this. I learned a lot. Thank you for putting together this amazing project and maintaining it 🙌
skarnet
commented
1 month ago np, glad you could make it work!
jaydrogers
commented
1 week ago I don't mean to wake up an old thread, but I figured keeping this organized on a single thread would be helpful.
STOPSIGNAL SIGTERM fixes the container taking a while to shut down, but this causes PHP-FPM to immediately shut down. SIGQUIT is required for a graceful shutdown of FPMI found this article explaining their solution:
The official PHP images even highlight this in their source code:
Thanks so much for your help! I hope the Olympics have been a good thing to have in town than a burden. 🇫🇷
skarnet
commented
1 week ago php-fpm as your CMD.down-signal file containing SIGQUIT in the php-fpm service definition directory.jaydrogers
commented
1 week ago But if you're running php-fpm as a regular supervised service, add a down-signal file containing SIGQUIT in the php-fpm service definition directory.
I think this is the ticket! Thanks for pointing this out, I was not aware of this option. I do have some issues trying to get this to work though.
Here is a link to my S6 service structure: https://github.com/serversideup/docker-php/tree/main/src/s6/etc/s6-overlay/s6-rc.d/php-fpm
When I did a Google search for "down-signal s6", it brought up this page for s6-svc:
https://skarnet.org/software/s6/s6-svc.html
I attempted the following steps:
.down-signal file at /etc/s6-overlay/s6-rc.d/php-fpm/down-signalInside of that file is:
SIGQUITrun script to use s6-svc#!/command/execlineb -P
with-contenv
s6-notifyoncheck -d
s6-svc -d /etc/s6-overlay/s6-rc.d/php-fpm
/usr/local/sbin/php-fpm --nodaemonizeThis fails with:
php-1 | s6-svc: warning: ignoring extra arguments
php-1 | s6-svc: fatal: unable to control /etc/s6-overlay/s6-rc.d/php-fpm/: supervisor not listenings6-notifyoncheck?My gut feeling is I can only use one option of s6-notifyoncheck or s6-svc? I think I previously used s6-notifyoncheck so I can use php-fpm as a dependency in my S6 configuration for NGINX.
Thanks a ton for your help!
skarnet
commented
1 week ago Please don't play sorcerer's apprentice.
down-signal, without an initial dot.s6-svc -d is a command used by the administrator to stop a service. Unless you aim to control supervised services manually inside of your container, forget you found the s6-svc page by accident.down-signal informs the supervisor of what the termination signal should be. The supervisor, s6-supervise, reads it, and when the time comes to stop the service, that's the signal it's going to send to the service process in order to kill it.
s6-svc -d command, telling the supervisor to stop the service; or when the supervisor receives a SIGTERM, which means that the whole supervision tree is stopping. The latter is what happens when s6-overlay runs its shutdown procedure.down-signal file is mentioned under the "longrun" section, and what it does is documented here.down-signal should probably be documented on the s6-overlay page, though, because php-fpm is certainly not the only service deviating from the SIGTERM default.down-signal file containing SIGQUIT to your php-fpm service definition directory. And nothing else - keep your run script as it was.
Problem
When I run images with S6 Overlay, it takes about 5-10 seconds for them to quit when I press
CTRL + C.Expected Result
Example
Here is a quick demo of my project that has two variations:
serversideup/php:beta-8.3-fpm) WITHOUT S6 Overlayserversideup/php:beta-8.3-fpm-nginx) with S6 OverlayYou will notice the FPM image (that does NOT run S6 Overlay) exits almost immediately and correctly. It's not until I introduce S6 Overlay is where the container changes behavior.
Video example
https://github.com/just-containers/s6-overlay/assets/3174134/c389622a-9bc3-4552-9467-6c687b8e9a3d
My code
Here's examples if this helps
Related issue (other projects)
What I'm looking for
I'm not expecting anyone to go fix anything in my code. I just wanted to ask the community if they have seen anything that I can do to learn from and improve.
If there is a better channel to ask this question on instead of issues, let me know. Thank you and I appreciate your help!