Closed enricodeleo closed 6 years ago
What is the output of docker ps -a
and dokku redis:info SERVICE
?
root@main-paas:~# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
96ed391896c5 dokku/platform:latest "/start web" 21 hours ago Up 21 hours platform.web.2
af583148fd3d dokku/platform:latest "/start web" 21 hours ago Up 21 hours platform.web.1
2ef4582ed9f5 redis:3.2.8 "docker-entrypoint.s…" 21 hours ago Up 21 hours 6379/tcp dokku.redis.platformcache
1cd27949b5e5 dokku/appunlimited:latest "/start web" 12 days ago Up 12 days appunlimited:latest.web.1
7ebddb239dec dokku/blog:latest "/start web" 12 days ago Up 12 days blog:latest.web.1
63633fd63f6e dokku/publishing:latest "/start web" 12 days ago Up 12 days publishing.web.1
c7c4965f11ac 6f3624f0a9aa "/start web" 12 days ago Up 12 days platform.web.2.1518722804
677243f1ac46 6f3624f0a9aa "/start web" 12 days ago Up 12 days platform.web.1.1518722296
e1d0b758797e mongo:3.2.9 "/entrypoint.sh mong…" 12 days ago Up 12 days 27017/tcp dokku.mongo.appunlimited
69e9aa45ad8b mongo:3.2.9 "/entrypoint.sh mong…" 12 days ago Up 12 days 27017/tcp dokku.mongo.publishing
0b43bcafab32 redis:3.2.8 "docker-entrypoint.s…" 12 days ago Up 12 days 6379/tcp dokku.redis.publicache
3eb15bac5d91 redis:3.2.8 "docker-entrypoint.s…" 12 days ago Up 12 days 6379/tcp dokku.redis.defaultcache
27e86e0e8673 mongo:3.2.9 "/entrypoint.sh mong…" 12 days ago Up 12 days 27017/tcp dokku.mongo.blog
fd278168bbd7 d854edd99cfd "/start web" 12 days ago Created serene_bose
a182f8a6b187 6cf48f1219ed "/start web" 12 days ago Created brave_jones
b78fa5639b22 c95935c8d6da "/start web" 12 days ago Created fervent_borg
5c5c3475f89a 7a216d230d3b "/start web" 12 days ago Created priceless_stonebraker
3bb345321225 05501b7ccdda "/start web" 12 days ago Created gracious_borg
and
root@main-paas:~# dokku redis:info defaultcache
=====> Container Information
Config dir: /var/lib/dokku/services/redis/defaultcache/config
Data dir: /var/lib/dokku/services/redis/defaultcache/data
Dsn: redis://defaultcache:e5a848220e5095698d8726b5859950da347caacad48c11eaabf169aab0a8a101@dokku-redis-defaultcache:6379
Exposed ports: -
Id: 3eb15bac5d91f93ed235d6d18ebafae0e6ba09630543550c01b40cc20fbbd09d
Internal ip: 172.17.0.3
Links: -
Service root: /var/lib/dokku/services/redis/defaultcache
Status: running
Version: redis:3.2.8
For the redis that wont be destroyed, can you run:
dokku --trace --force redis:destroy SERVICE
And gist the output?
It still stays on stop
root@main-paas:~# dokku --trace --force redis:destroy defaultcache
+ export DOKKU_DISTRO
++ . /etc/os-release
++ echo ubuntu
+ DOKKU_DISTRO=ubuntu
+ export DOKKU_IMAGE=gliderlabs/herokuish
+ DOKKU_IMAGE=gliderlabs/herokuish
+ export DOKKU_LIB_ROOT=/var/lib/dokku
+ DOKKU_LIB_ROOT=/var/lib/dokku
+ export PLUGIN_PATH=/var/lib/dokku/plugins
+ PLUGIN_PATH=/var/lib/dokku/plugins
+ export PLUGIN_AVAILABLE_PATH=/var/lib/dokku/plugins/available
+ PLUGIN_AVAILABLE_PATH=/var/lib/dokku/plugins/available
+ export PLUGIN_ENABLED_PATH=/var/lib/dokku/plugins/enabled
+ PLUGIN_ENABLED_PATH=/var/lib/dokku/plugins/enabled
+ export PLUGIN_CORE_PATH=/var/lib/dokku/core-plugins
+ PLUGIN_CORE_PATH=/var/lib/dokku/core-plugins
+ export PLUGIN_CORE_AVAILABLE_PATH=/var/lib/dokku/core-plugins/available
+ PLUGIN_CORE_AVAILABLE_PATH=/var/lib/dokku/core-plugins/available
+ export PLUGIN_CORE_ENABLED_PATH=/var/lib/dokku/core-plugins/enabled
+ PLUGIN_CORE_ENABLED_PATH=/var/lib/dokku/core-plugins/enabled
+ export DOKKU_API_VERSION=1
+ DOKKU_API_VERSION=1
+ export DOKKU_NOT_IMPLEMENTED_EXIT=10
+ DOKKU_NOT_IMPLEMENTED_EXIT=10
+ export DOKKU_VALID_EXIT=0
+ DOKKU_VALID_EXIT=0
+ export DOKKU_LOGS_DIR=/var/log/dokku
+ DOKKU_LOGS_DIR=/var/log/dokku
+ export DOKKU_EVENTS_LOGFILE=/var/log/dokku/events.log
+ DOKKU_EVENTS_LOGFILE=/var/log/dokku/events.log
+ export DOKKU_CONTAINER_LABEL=dokku
+ DOKKU_CONTAINER_LABEL=dokku
+ export DOKKU_GLOBAL_RUN_ARGS=--label=dokku
+ DOKKU_GLOBAL_RUN_ARGS=--label=dokku
+ source /var/lib/dokku/core-plugins/available/common/functions
++ set -eo pipefail
++ [[ -n 1 ]]
++ set -x
+ parse_args redis:destroy defaultcache
+ declare 'desc=top-level cli arg parser'
+ local next_index=1
+ local skip=false
+ args=("$@")
+ local args
+ for arg in '"$@"'
+ [[ false == \t\r\u\e ]]
+ case "$arg" in
+ next_index=2
+ for arg in '"$@"'
+ [[ false == \t\r\u\e ]]
+ case "$arg" in
+ next_index=3
+ return 0
+ args=("$@")
+ skip_arg=false
+ [[ redis:destroy =~ ^--.* ]]
+ has_tty
+ declare 'desc=return 0 if we have a tty'
++ /usr/bin/tty
+ [[ /dev/pts/0 == \n\o\t\ \a\ \t\t\y ]]
+ return 0
++ id -un
+ [[ dokku != \d\o\k\k\u ]]
+ [[ redis:destroy =~ ^plugin:.* ]]
+ [[ redis:destroy == \s\s\h\-\k\e\y\s\:\a\d\d ]]
+ [[ -n '' ]]
+ dokku_auth redis:destroy defaultcache
+ declare 'desc=calls user-auth plugin trigger'
+ export SSH_USER=root
+ SSH_USER=root
+ export SSH_NAME=default
+ SSH_NAME=default
+ plugn trigger user-auth root default redis:destroy defaultcache
+ source /var/lib/dokku/core-plugins/available/common/functions
++ set -eo pipefail
++ [[ -n 1 ]]
++ set -x
+ [[ ! -n '' ]]
+ source /var/lib/dokku/core-plugins/available/common/functions
++ set -eo pipefail
++ [[ -n 1 ]]
++ set -x
+ [[ redis:destroy =~ (ssh-keys:add|ssh-keys:remove) ]]
+ return 0
+ case "$1" in
+ execute_dokku_cmd redis:destroy defaultcache
+ declare 'desc=executes dokku sub-commands'
+ local PLUGIN_NAME=redis:destroy
+ local PLUGIN_CMD=redis:destroy
+ local implemented=0
+ local script
+ argv=("$@")
+ local argv
+ case "$PLUGIN_NAME" in
++ readlink -f /var/lib/dokku/plugins/enabled/redis
+ [[ /var/lib/dokku/plugins/available/redis == *core-plugins* ]]
+ [[ -x /var/lib/dokku/plugins/enabled/redis:destroy/subcommands/default ]]
+ [[ -x /var/lib/dokku/plugins/enabled/redis:destroy/subcommands/redis:destroy ]]
+ [[ -x /var/lib/dokku/plugins/enabled/redis/subcommands/destroy ]]
+ [[ -n destroy ]]
+ /var/lib/dokku/plugins/enabled/redis/subcommands/destroy redis:destroy defaultcache
+ source /var/lib/dokku/plugins/enabled/common/functions
++ set -eo pipefail
++ [[ -n 1 ]]
++ set -x
++++ dirname /var/lib/dokku/plugins/enabled/redis/subcommands/destroy
+++ cd /var/lib/dokku/plugins/enabled/redis/subcommands
+++ pwd
++ dirname /var/lib/dokku/plugins/enabled/redis/subcommands
+ source /var/lib/dokku/plugins/enabled/redis/functions
++++ dirname /var/lib/dokku/plugins/enabled/redis/functions
+++ cd /var/lib/dokku/plugins/enabled/redis
+++ pwd
++ source /var/lib/dokku/plugins/enabled/redis/config
+++ export REDIS_IMAGE=redis
+++ REDIS_IMAGE=redis
+++ export REDIS_IMAGE_VERSION=3.2.8
+++ REDIS_IMAGE_VERSION=3.2.8
+++ export REDIS_ROOT=/var/lib/dokku/services/redis
+++ REDIS_ROOT=/var/lib/dokku/services/redis
+++ export PLUGIN_COMMAND_PREFIX=redis
+++ PLUGIN_COMMAND_PREFIX=redis
+++ export PLUGIN_CONFIG_ROOT=/var/lib/dokku/config/redis
+++ PLUGIN_CONFIG_ROOT=/var/lib/dokku/config/redis
+++ export PLUGIN_DATA_ROOT=/var/lib/dokku/services/redis
+++ PLUGIN_DATA_ROOT=/var/lib/dokku/services/redis
+++ PLUGIN_DATASTORE_PORTS=(6379)
+++ export PLUGIN_DATASTORE_PORTS
+++ export PLUGIN_DATASTORE_WAIT_PORT=6379
+++ PLUGIN_DATASTORE_WAIT_PORT=6379
+++ export PLUGIN_DEFAULT_ALIAS=REDIS
+++ PLUGIN_DEFAULT_ALIAS=REDIS
+++ export PLUGIN_ALT_ALIAS=DOKKU_REDIS
+++ PLUGIN_ALT_ALIAS=DOKKU_REDIS
+++ export PLUGIN_IMAGE=redis
+++ PLUGIN_IMAGE=redis
+++ export PLUGIN_IMAGE_VERSION=3.2.8
+++ PLUGIN_IMAGE_VERSION=3.2.8
+++ export PLUGIN_SCHEME=redis
+++ PLUGIN_SCHEME=redis
+++ export PLUGIN_SERVICE=Redis
+++ PLUGIN_SERVICE=Redis
+++ export PLUGIN_VARIABLE=REDIS
+++ PLUGIN_VARIABLE=REDIS
+++ export PLUGIN_BASE_PATH=/var/lib/dokku/plugins
+++ PLUGIN_BASE_PATH=/var/lib/dokku/plugins
+++ [[ -n 1 ]]
+++ export PLUGIN_BASE_PATH=/var/lib/dokku/plugins/enabled
+++ PLUGIN_BASE_PATH=/var/lib/dokku/plugins/enabled
+++ [[ -d /var/lib/dokku/services/redis/* ]]
++ set -eo pipefail
++ [[ -n 1 ]]
++ set -x
++++ dirname /var/lib/dokku/plugins/enabled/redis/functions
+++ cd /var/lib/dokku/plugins/enabled/redis
+++ pwd
++ source /var/lib/dokku/plugins/enabled/redis/common-functions
+++ set -eo pipefail
+++ [[ -n 1 ]]
+++ set -x
++ source /var/lib/dokku/plugins/enabled/common/functions
+++ set -eo pipefail
+++ [[ -n 1 ]]
+++ set -x
++ source /var/lib/dokku/plugins/available/config/functions
+++ set -eo pipefail
+++ [[ -n 1 ]]
+++ set -x
+++ source /var/lib/dokku/core-plugins/available/common/functions
++++ set -eo pipefail
++++ [[ -n 1 ]]
++++ set -x
++ [[ -f /var/lib/dokku/plugins/available/docker-options/functions ]]
++ source /var/lib/dokku/plugins/available/docker-options/functions
+++ set -eo pipefail
+++ [[ -n 1 ]]
+++ set -x
+++ source /var/lib/dokku/core-plugins/available/common/functions
++++ set -eo pipefail
++++ [[ -n 1 ]]
++++ set -x
+++ AVAILABLE_PHASES=(build deploy run)
+ service-destroy-cmd redis:destroy defaultcache
+ declare 'desc=delete the Redis service/data/container if there are no links left'
+ argv=("$@")
+ local cmd=redis:destroy argv
+ [[ redis:destroy == \r\e\d\i\s\:\d\e\s\t\r\o\y ]]
+ shift 1
+ declare SERVICE=defaultcache FORCE_FLAG=
+ [[ -z defaultcache ]]
+ verify_service_name defaultcache
+ declare 'desc=Verifies that a service exists'
+ declare SERVICE=defaultcache
+ [[ ! -n defaultcache ]]
+ [[ ! -d /var/lib/dokku/services/redis/defaultcache ]]
+ return 0
+ SERVICE_ROOT=/var/lib/dokku/services/redis/defaultcache
+ LINKS_FILE=/var/lib/dokku/services/redis/defaultcache/LINKS
++ get_service_name defaultcache
++ declare 'desc=Retrieves a docker service label'
++ declare SERVICE=defaultcache
++ echo dokku.redis.defaultcache
+ SERVICE_NAME=dokku.redis.defaultcache
+ [[ -s /var/lib/dokku/services/redis/defaultcache/LINKS ]]
+ [[ '' == \f\o\r\c\e ]]
+ [[ '' == \-\f ]]
+ [[ '' == \-\-\f\o\r\c\e ]]
+ [[ -z 1 ]]
+ dokku_log_info1 'Deleting defaultcache'
+ declare 'desc=log info1 formatter'
+ echo '-----> Deleting defaultcache'
-----> Deleting defaultcache
++ docker ps -aq -f name=dokku.redis.defaultcache
+ [[ -n 3eb15bac5d91 ]]
+ dokku_log_verbose_quiet 'Deleting container data'
+ declare 'desc=log verbose formatter (with quiet option)'
+ [[ -z '' ]]
+ echo ' Deleting container data'
Deleting container data
+ service_stop defaultcache
+ declare 'desc=Stops a running service'
+ declare SERVICE=defaultcache
+ local SERVICE_ROOT=/var/lib/dokku/services/redis/defaultcache
++ get_service_name defaultcache
++ declare 'desc=Retrieves a docker service label'
++ declare SERVICE=defaultcache
++ echo dokku.redis.defaultcache
+ local SERVICE_NAME=dokku.redis.defaultcache
++ docker ps -f status=running
++ grep -e 'dokku.redis.defaultcache$'
++ awk '{print $1}'
+ local ID=3eb15bac5d91
+ [[ -z 3eb15bac5d91 ]]
+ [[ -n 3eb15bac5d91 ]]
+ dokku_log_info1_quiet 'Stopping container'
+ declare 'desc=log info1 formatter (with quiet option)'
+ [[ -z '' ]]
+ echo '-----> Stopping container'
-----> Stopping container
+ docker stop dokku.redis.defaultcache
What is the output of
docker stop 3eb15bac5d91
This command has no output. Stays on
root@main-paas:~# docker stop 3eb15bac5d91
Right so this sounds like a docker issue.
What happens on docker rm -f 3eb15bac5d91
?
Still no output :(
Alright, I would file an issue in docker, not much I can do here about a stuck container. Maybe check docker logs?
Alternatively, you can restart your server. Feel free to log into our slack chat if you have any issues with that.
Sorry I wasn't much help otherwise :(
Thank you anyway. I'd rather prefer not to reboot the machine since the last time it happened (2 weeks ago apparently my host stopped replying to requests) I had to re-start each service manually leading to a service interruption.
I use just one redis container on my dokku machine, and resources in general seem okay but suddenly the service that was using redis stopped working. After a few minute I understood that redis was failing so I thought I could restart the service but it keeps on
Stopping container
. Even unlinking and destroying fails with the service stucked onStopping container
.Any ideas?
UPDATE I managed to create another redis service and connect the application to that, but the former container is still not responding and stays on