Run of specific service should run dependent services

shayrybak commented 6 years ago

ISSUE TYPE

Bug Report

container.yml

version: "2"
settings:
  conductor:
    # The Conductor container does the heavy lifting, and provides a portable
    # Python runtime for building your target containers. It should be derived
    # from the same distribution as you're building your target containers with.
    base: centos:7
    roles_path: 
      - ../../roles

    # environment:  # List or mapping of environment variables
  project_name: waf

services: 
  # kibana:
  #   from: centos:7
  #   ports:
  #     - 5601:5601
  #   expose:
  #     - 5601
  #   links:
  #     - elastic
  #   entrypoint: "/usr/bin/dumb-init"
  #   command: /usr/share/kibana/bin/kibana
  #   user: kibana
  #   roles:
  #     - dumb_init
  #     - kibana
  #     - cleanup
  lime:
    from: centos:7
    links:
      - mongodb
      - redis
      - rabbitmq
      - sumologic
      - mysql
    roles:
      - { role: common, app_name: lime }
      - dumb_init
      - lua
      - lime
    entrypoint: "/usr/bin/dumb-init"
    command: <some command>
    depends_on:
      - mongodb
      - redis
      - rabbitmq
      - sumologic
      - mysql
  mongodb:
    from: centos:7
    ports:
      - 27017:27017
    expose:
      - 27017
    entrypoint: "/usr/bin/dumb-init"
    command: mongod --config /etc/mongod.conf
    roles:
      - dumb_init
      - mongodb

OS / ENVIRONMENT

{
  "ContainersPaused": 0,
  "Labels": [],
  "CgroupDriver": "cgroupfs",
  "ContainersRunning": 2,
  "ContainerdCommit": {
    "Expected": "89623f28b87a6004d4b785663257362d1658a729",
    "ID": "89623f28b87a6004d4b785663257362d1658a729"
  },
  "InitBinary": "docker-init",
  "NGoroutines": 95,
  "Swarm": {
    "ControlAvailable": false,
    "NodeID": "",
    "Error": "",
    "RemoteManagers": null,
    "LocalNodeState": "inactive",
    "NodeAddr": ""
  },
  "LoggingDriver": "json-file",
  "OSType": "linux",
  "HttpProxy": "docker.for.mac.http.internal:3128",
  "Runtimes": {
    "runc": {
      "path": "docker-runc"
    }
  },
  "DriverStatus": [
    [
      "Backing Filesystem",
      "extfs"
    ],
    [
      "Supports d_type",
      "true"
    ],
    [
      "Native Overlay Diff",
      "true"
    ]
  ],
  "OperatingSystem": "Docker for Mac",
  "Containers": 26,
  "HttpsProxy": "docker.for.mac.http.internal:3129",
  "BridgeNfIp6tables": true,
  "MemTotal": 12575907840,
  "SecurityOptions": [
    "name=seccomp,profile=default"
  ],
  "Driver": "overlay2",
  "IndexServerAddress": "https://index.docker.io/v1/",
  "ClusterStore": "",
  "InitCommit": {
    "Expected": "949e6fa",
    "ID": "949e6fa"
  },
  "GenericResources": null,
  "Isolation": "",
  "SystemStatus": null,
  "OomKillDisable": true,
  "ClusterAdvertise": "",
  "SystemTime": "2018-01-25T11:52:59.630909673Z",
  "Name": "linuxkit-025000000001",
  "CPUSet": true,
  "RegistryConfig": {
    "AllowNondistributableArtifactsCIDRs": [],
    "Mirrors": [],
    "IndexConfigs": {
      "docker.io": {
        "Official": true,
        "Name": "docker.io",
        "Secure": true,
        "Mirrors": []
      }
    },
    "AllowNondistributableArtifactsHostnames": [],
    "InsecureRegistryCIDRs": [
      "127.0.0.0/8"
    ]
  },
  "DefaultRuntime": "runc",
  "ContainersStopped": 24,
  "NCPU": 4,
  "NFd": 76,
  "Architecture": "x86_64",
  "KernelMemory": true,
  "CpuCfsQuota": true,
  "Debug": true,
  "ID": "CGIZ:QNJJ:A2XO:CO5J:G6WS:6ZEQ:M54A:ANHW:M7KA:TE7W:HVMI:OPGV",
  "IPv4Forwarding": true,
  "KernelVersion": "4.9.60-linuxkit-aufs",
  "BridgeNfIptables": true,
  "NoProxy": "",
  "LiveRestoreEnabled": false,
  "ServerVersion": "17.12.0-ce",
  "CpuCfsPeriod": true,
  "ExperimentalBuild": true,
  "MemoryLimit": true,
  "SwapLimit": true,
  "Plugins": {
    "Volume": [
      "local"
    ],
    "Network": [
      "bridge",
      "host",
      "ipvlan",
      "macvlan",
      "null",
      "overlay"
    ],
    "Authorization": null,
    "Log": [
      "awslogs",
      "fluentd",
      "gcplogs",
      "gelf",
      "journald",
      "json-file",
      "logentries",
      "splunk",
      "syslog"
    ]
  },
  "Images": 107,
  "DockerRootDir": "/var/lib/docker",
  "NEventsListener": 2,
  "CPUShares": true,
  "RuncCommit": {
    "Expected": "b2567b37d7b75eb4cf325b77297b140ea686ce8f",
    "ID": "b2567b37d7b75eb4cf325b77297b140ea686ce8f"
  }
}
{
  "KernelVersion": "4.9.60-linuxkit-aufs",
  "Components": [
    {
      "Version": "17.12.0-ce",
      "Name": "Engine",
      "Details": {
        "KernelVersion": "4.9.60-linuxkit-aufs",
        "Os": "linux",
        "BuildTime": "2017-12-27T20:12:29.000000000+00:00",
        "ApiVersion": "1.35",
        "MinAPIVersion": "1.12",
        "GitCommit": "c97c6d6",
        "Arch": "amd64",
        "Experimental": "true",
        "GoVersion": "go1.9.2"
      }
    }
  ],
  "Arch": "amd64",
  "BuildTime": "2017-12-27T20:12:29.000000000+00:00",
  "ApiVersion": "1.35",
  "Platform": {
    "Name": ""
  },
  "Version": "17.12.0-ce",
  "MinAPIVersion": "1.12",
  "GitCommit": "c97c6d6",
  "Os": "linux",
  "Experimental": true,
  "GoVersion": "go1.9.2"
}

SUMMARY

When running a specific service which has dependencies the run comm

STEPS TO REPRODUCE

Create a container.yml file with 2 services where one has a depends_on attribute for the other, build the services and run ansible-container run <service1>

EXPECTED RESULTS

Expected a-c to run the dependent service as well as the service requested in correct order

ACTUAL RESULTS

Got the following error:

Parsing conductor CLI args.
Engine integration loaded. Preparing run.   engine=Docker™ daemon
Verifying service image service=lime

PLAY [Deploy waf] **************************************************************

TASK [docker_service] **********************************************************
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "Configuration error - Service 'lime' depends on service 'mongodb' which is undefined."}
    to retry, use: --limit @/Users/shay/projects/ansible-container/projects/waf/ansible-deployment/playbook.retry

PLAY RECAP *********************************************************************
localhost                  : ok=0    changed=0    unreachable=0    failed=1

Traceback (most recent call last):
  File "/usr/bin/conductor", line 11, in <module>
    load_entry_point('ansible-container', 'console_scripts', 'conductor')()
  File "/_ansible/container/__init__.py", line 19, in __wrapped__
    return fn(*args, **kwargs)
  File "/_ansible/container/cli.py", line 399, in conductor_commandline
    **params)
  File "/_ansible/container/__init__.py", line 19, in __wrapped__
    return fn(*args, **kwargs)
  File "/_ansible/container/core.py", line 854, in conductorcmd_run
    'Error executing the run command. Not all containers may be running.'
container.exceptions.AnsibleContainerException: Error executing the run command. Not all containers may be running.
Conductor terminated. Cleaning up.  command_rc=1 conductor_id=226cb7b8e84e87538d10409bd4ef25860416eb27b611e52df417026bbb3a2fa6 save_container=False
ERROR   Conductor exited with status 1

j00bar commented 6 years ago

Using "depends on" is very much an anti-pattern and really should be avoided. If my word isn't enough, maybe his will be: https://medium.com/@kelseyhightower/12-fractured-apps-1080c73d481c (Search in the article for "Eliminate the need to deploy services in a specific order").

Even still this would be a low priority for me. The depends_on is highly Docker specific, but I'll accept it as something we can address.

shayrybak commented 6 years ago

I care less about the specific order in this case, although some applications that I use, like nginx, won't start up if the upstream isn't resolvable, in our use case the upstream would be another container.

But this was more for convenience, if I want to run a specific service out of the full project, I want it to run with its dependencies.

In any case, thank you for accepting.

ansible / ansible-container