search

JaneliaSciComp / jacs-cm

JACS Configuration Management
Other
8 stars 3 forks source link

No such image: janeliascicomp/jacs-init:1.0.11 and starting container failed #6

Closed SupermeLC closed 4 years ago

SupermeLC commented 4 years ago

Hi, I tried to build the MouseLight tools and services.

Problem

Problem 1

When I initialize the filesystems: 

./manage.sh init-filesystems

The output is:

..............

+ docker service ps --no-trunc jacs_jacs-init
ID                          NAME                                       IMAGE                                                                                                     NODE                DESIRED STATE       CURRENT STATE             ERROR                                                                                                                      PORTS
vl06m8itxwt57wmj5survq98k   jacs_jacs-init.l2g1tu01ogjwwupjd56gzdh3e   janeliascicomp/jacs-init:1.0.11@sha256:b9faeeb1717d1bc74db6b0c82d91eb26023fadf736d1ad4e010ee2397138465a   mk-All-Series       Shutdown            Rejected 12 minutes ago   "No such image: janeliascicomp/jacs-init:1.0.11@sha256:b9faeeb1717d1bc74db6b0c82d91eb26023fadf736d1ad4e010ee2397138465a"   
yw2wtq46ti0zceqll0a5077xl   jacs_jacs-init.4p0fbtmpo59bwahbne34zl47h   janeliascicomp/jacs-init:1.0.11@sha256:b9faeeb1717d1bc74db6b0c82d91eb26023fadf736d1ad4e010ee2397138465a   csdl-neutu          Shutdown            Complete 12 minutes ago             
..............

Host1 is csdl-neutu and Host2 is mk-All-Series. The image cannot be found and I cannot found any files in the CONFIG_DIR.

Problem 2

Then I run the command:

docker service rm jacs_jacs-init

and initialize the filesystems:

./manage.sh init-filesystems

This time, the error changes:

..............

+ docker service ps --no-trunc jacs_jacs-init
ID                          NAME                                       IMAGE                                                                                                     NODE                DESIRED STATE       CURRENT STATE            ERROR                                                                  PORTS
tgw6uh7myelunyuv6m3vqjyd6   jacs_jacs-init.svoc7rbv0bf52fexcbxp6vudg   janeliascicomp/jacs-init:1.0.11@sha256:b9faeeb1717d1bc74db6b0c82d91eb26023fadf736d1ad4e010ee2397138465a   mk-All-Series       Shutdown            Failed 5 seconds ago     "starting container failed: no status provided on response: unknown"   
ne0p1k97hvj7baaak7cu6bxtc   jacs_jacs-init.h2vzbu36y4ys0aahlnxzjgrm2   janeliascicomp/jacs-init:1.0.11@sha256:b9faeeb1717d1bc74db6b0c82d91eb26023fadf736d1ad4e010ee2397138465a   csdl-neutu          Shutdown            Complete 8 seconds ago         
..............

and I still connot found any files in the CONFIG_DIR.

Steps to reproduce

Follow the instruction "MouseLight Deployment".

The ".env.config" contains:

    DEPLOYMENT=mouselight

    UNAME=docker-nobody
    GNAME=docker-nobody

    HOST1=csdl-neutu
    HOST2=mk-All-Series

    REDUNDANT_STORAGE=/opt/project/$COMPOSE_PROJECT_NAME
    NON_REDUNDANT_STORAGE=/$REDUNDANT_STORAGE/data
    CONFIG_DIR=$REDUNDANT_STORAGE/config

Full Output

Bug 1:

(base) [csdl-neutu@csdl-neutu jacs-cm]$ ./manage.sh init-filesystems
Generating .env from .env.config
Parsing .env
Using deployment mouselight defined by /opt/deploy/jacs-cm/deployments/mouselight
Initializing swarm file systems...
+ DOCKER_USER=4444:4444
+ docker-compose -f /opt/deploy/jacs-cm/deployments/mouselight/swarm-init.yml config
WARNING: Some services (jacs-init) use the 'deploy' key, which will be ignored. Compose does not support 'deploy' configuration - use `docker stack deploy` to deploy to a swarm.
+ docker stack deploy --prune -c .tmp.swarm.yml jacs
Creating service jacs_jacs-init
+ sleep 10
+ docker service logs --no-task-ids --no-trunc jacs_jacs-init
jacs_jacs-init.0@csdl-neutu    | Verified CONFIG_DIR exists: /opt/project/jacs/config
jacs_jacs-init.0@csdl-neutu    | Verified DB_DIR exists: /opt/project/jacs/db
jacs_jacs-init.0@csdl-neutu    | Verified DATA_DIR exists: //opt/project/jacs/data
jacs_jacs-init.0@csdl-neutu    | Verified BACKUPS_DIR exists: //opt/project/jacs/data/backups
jacs_jacs-init.0@csdl-neutu    | Verified MongoDB data directories: /opt/project/jacs/db/mongo/jacs
jacs_jacs-init.0@csdl-neutu    | Verified MySQL data directory: /opt/project/jacs/db/mysql/jacs
jacs_jacs-init.0@csdl-neutu    | Verified MySQL config directory: /opt/project/jacs/config/mysql
jacs_jacs-init.0@csdl-neutu    | Verified SOLR indexes directory: /opt/project/jacs/db/solr
jacs_jacs-init.0@csdl-neutu    | Verified RabbitMQ data directory: /opt/project/jacs/db/rabbitmq/jacs
jacs_jacs-init.0@csdl-neutu    | Verified certificates directory: /opt/project/jacs/config/certs
jacs_jacs-init.0@csdl-neutu    | Verified JACS Async Services config directory: /opt/project/jacs/config/jacs-async
jacs_jacs-init.0@csdl-neutu    | Verified JACS Sync Services config directory: /opt/project/jacs/config/jacs-sync
jacs_jacs-init.0@csdl-neutu    | Verified JACS Dashboard config directory: /opt/project/jacs/config/jacs-dashboard
jacs_jacs-init.0@csdl-neutu    | Verified Jade config directory: /opt/project/jacs/config/jade
jacs_jacs-init.0@csdl-neutu    | Verified Jade storage: //opt/project/jacs/data/jacsstorage
jacs_jacs-init.0@csdl-neutu    | Verified jacs-messaging config directory: /opt/project/jacs/config/messaging
jacs_jacs-init.0@csdl-neutu    | Verified IPP config directory: /opt/project/jacs/config/ipp
jacs_jacs-init.0@csdl-neutu    | Verified API Gateway config directory: /opt/project/jacs/config/api-gateway
jacs_jacs-init.0@csdl-neutu    | Verified Mongo backups directory: //opt/project/jacs/data/backups/mongo
jacs_jacs-init.0@csdl-neutu    | Verified MySQL backups directory: //opt/project/jacs/data/backups/mysql
jacs_jacs-init.0@csdl-neutu    | Verified jacs-messaging backups directory: //opt/project/jacs/data/backups/messaging
+ docker service ps --no-trunc jacs_jacs-init
ID                          NAME                                       IMAGE                                                                                                     NODE                DESIRED STATE       CURRENT STATE             ERROR                                                                                                                      PORTS
vl06m8itxwt57wmj5survq98k   jacs_jacs-init.l2g1tu01ogjwwupjd56gzdh3e   janeliascicomp/jacs-init:1.0.11@sha256:b9faeeb1717d1bc74db6b0c82d91eb26023fadf736d1ad4e010ee2397138465a   mk-All-Series       Shutdown            Rejected 12 minutes ago   "No such image: janeliascicomp/jacs-init:1.0.11@sha256:b9faeeb1717d1bc74db6b0c82d91eb26023fadf736d1ad4e010ee2397138465a"   
yw2wtq46ti0zceqll0a5077xl   jacs_jacs-init.4p0fbtmpo59bwahbne34zl47h   janeliascicomp/jacs-init:1.0.11@sha256:b9faeeb1717d1bc74db6b0c82d91eb26023fadf736d1ad4e010ee2397138465a   csdl-neutu          Shutdown            Complete 12 minutes ago                                                                       
+ set +x
Filesystem initializing is running. When it's finished, all the tasks above should be in Shutdown state.
To clean up, run this command: docker service rm jacs_jacs-init

Bug 2:

(base) [csdl-neutu@csdl-neutu jacs-cm]$ ./manage.sh init-filesystems
Generating .env from .env.config
Parsing .env
Using deployment mouselight defined by /opt/deploy/jacs-cm/deployments/mouselight
Initializing swarm file systems...
+ DOCKER_USER=4444:4444
+ docker-compose -f /opt/deploy/jacs-cm/deployments/mouselight/swarm-init.yml config
WARNING: Some services (jacs-init) use the 'deploy' key, which will be ignored. Compose does not support 'deploy' configuration - use `docker stack deploy` to deploy to a swarm.
+ docker stack deploy --prune -c .tmp.swarm.yml jacs
Creating service jacs_jacs-init
+ sleep 10
+ docker service logs --no-task-ids --no-trunc jacs_jacs-init
jacs_jacs-init.0@csdl-neutu    | Verified CONFIG_DIR exists: /opt/project/jacs/config
jacs_jacs-init.0@csdl-neutu    | Verified DB_DIR exists: /opt/project/jacs/db
jacs_jacs-init.0@csdl-neutu    | Verified DATA_DIR exists: //opt/project/jacs/data
jacs_jacs-init.0@csdl-neutu    | Verified BACKUPS_DIR exists: //opt/project/jacs/data/backups
jacs_jacs-init.0@csdl-neutu    | Verified MongoDB data directories: /opt/project/jacs/db/mongo/jacs
jacs_jacs-init.0@csdl-neutu    | Verified MySQL data directory: /opt/project/jacs/db/mysql/jacs
jacs_jacs-init.0@csdl-neutu    | Verified MySQL config directory: /opt/project/jacs/config/mysql
jacs_jacs-init.0@csdl-neutu    | Verified SOLR indexes directory: /opt/project/jacs/db/solr
jacs_jacs-init.0@csdl-neutu    | Verified RabbitMQ data directory: /opt/project/jacs/db/rabbitmq/jacs
jacs_jacs-init.0@csdl-neutu    | Verified certificates directory: /opt/project/jacs/config/certs
jacs_jacs-init.0@csdl-neutu    | Verified JACS Async Services config directory: /opt/project/jacs/config/jacs-async
jacs_jacs-init.0@csdl-neutu    | Verified JACS Sync Services config directory: /opt/project/jacs/config/jacs-sync
jacs_jacs-init.0@csdl-neutu    | Verified JACS Dashboard config directory: /opt/project/jacs/config/jacs-dashboard
jacs_jacs-init.0@csdl-neutu    | Verified Jade config directory: /opt/project/jacs/config/jade
jacs_jacs-init.0@csdl-neutu    | Verified Jade storage: //opt/project/jacs/data/jacsstorage
jacs_jacs-init.0@csdl-neutu    | Verified jacs-messaging config directory: /opt/project/jacs/config/messaging
jacs_jacs-init.0@csdl-neutu    | Verified IPP config directory: /opt/project/jacs/config/ipp
jacs_jacs-init.0@csdl-neutu    | Verified API Gateway config directory: /opt/project/jacs/config/api-gateway
jacs_jacs-init.0@csdl-neutu    | Verified Mongo backups directory: //opt/project/jacs/data/backups/mongo
jacs_jacs-init.0@csdl-neutu    | Verified MySQL backups directory: //opt/project/jacs/data/backups/mysql
jacs_jacs-init.0@csdl-neutu    | Verified jacs-messaging backups directory: //opt/project/jacs/data/backups/messaging
+ docker service ps --no-trunc jacs_jacs-init
ID                          NAME                                       IMAGE                                                                                                     NODE                DESIRED STATE       CURRENT STATE            ERROR                                                                  PORTS
vqu4yqkvnhor4hos7x29gn8ye   jacs_jacs-init.svoc7rbv0bf52fexcbxp6vudg   janeliascicomp/jacs-init:1.0.11@sha256:b9faeeb1717d1bc74db6b0c82d91eb26023fadf736d1ad4e010ee2397138465a   mk-All-Series       Shutdown            Failed 5 seconds ago     "starting container failed: no status provided on response: unknown"   
m9gueajj22smymnuecr6o00qg   jacs_jacs-init.h2vzbu36y4ys0aahlnxzjgrm2   janeliascicomp/jacs-init:1.0.11@sha256:b9faeeb1717d1bc74db6b0c82d91eb26023fadf736d1ad4e010ee2397138465a   csdl-neutu          Shutdown            Complete 8 seconds ago                                                                          
+ set +x
Filesystem initializing is running. When it's finished, all the tasks above should be in Shutdown state.
To clean up, run this command: docker service rm jacs_jacs-init

Looking forward to your reply! Thank you!

krokicki commented 4 years ago

Looks like one of your systems ("mk-All-Series") is not downloading the Docker image correctly. The image is there on Docker Hub, as you can see because it worked on the other system. So I would guess this is either a Docker installation issue, or a networking issue. You might start by making sure you can connect to DockerHub from both systems. And then make sure you can run the Hello World image on both systems: https://hub.docker.com/_/hello-world

krokicki commented 4 years ago

I've been made aware that Docker Hub had some downtime recently, so that might be why you had trouble downloading the image. You might want to just try it again. I will close this, since it's not an issue with jacs-cm.