Closed deusebio closed 9 months ago
Per yesterday's discussion, this is by design as the "shutdown" action is specified, so Pebble is doing what it was told and hence using the zero exit code. We probably wouldn't wire the exit code through directly, as there are (or may be) multiple services involved.
We'll definitely consider adding a way to have Pebble exit with a nonzero exit code in this case, but we'll need to have a (very short) spec and discuss first. My initial suggestion is to have a new "action", shutdown-nonzero
(we can bikeshed on the naming).
In the meantime to fix the customer issue, you can use the technique we looked at yesterday, where the ROCK entrypoint script does kill -9 1
(send SIGKILL to Pebble) if the process being run exits with nonzero exit code. @cjdcordeiro will post the code for that here for the record.
The proposal for such a workaround (using kill
) is proposed in https://github.com/canonical/charmed-spark-rock/pull/53/files
Just a small update, as sending the SIGKILL to process with PID 1 won't work, so the advised workaround here would be to send another signal (e.g. SIGHUP), as done here
For reference, @deusebio created spec DA062 to brainstorm the solution for this. After much discussion, we've settled on updating on-failure: shutdown
to return a nonzero code (specifically 10 meaning "service failed"), and adding options for on-failure: success-shutdown
and on-success: failure-shutdown
when you want the opposite of the default meaning.
This is included in v1.6.0 which I just released: https://github.com/canonical/pebble/releases/tag/v1.6.0
Though note there seems to be some failures building the Snap for non-amd64 architectures @cjdcordeiro
When a Pebble service (that has
on-failure: shutdown
) fails, therefore leading also Pebble (and the container) to shut down, the exit code is0
.This does not allow to externally handle failure of processes managed by pebble or containers (e.g. k8s)
It would be nice that if pebble is shut down becuase of a service process failing, the exiting code gets propagated to pebble exit code as well, such that it can be externally handled and leveraged on.
To Reproduce
I have simply created a vanilla example, moving from the hello-world application found in the documentation:
After creating the image and importing this in docker using skopeo, I run the image: