search

ansible / ansible-container

DEPRECATED -- Ansible Container was a tool to build Docker images and orchestrate containers using only Ansible playbooks.
GNU Lesser General Public License v3.0
2.19k stars 394 forks source link

ansible-container 0.9.0.0 - BuildError #447

Open ghost opened 7 years ago

ghost commented 7 years ago
ISSUE TYPE
container.yml
version: "2"
settings:
  conductor_base: centos:7

services:
  web:
    from: centos:7
    roles:
      - apache-container
    ports:
      - "80:80"
    command: ['/usr/bin/dumb-init', '/usr/sbin/apache2ctl', '-D', 'FOREGROUND']
    dev_overrides:
      environment:
        - "DEBUG=1"
registries: {}
main.yml
NO MAIN.YML file
OS / ENVIRONMENT
Ansible Container, version 0.9.0.0
Linux, node1, 4.10.10-200.fc25.x86_64, #1 SMP Thu Apr 13 01:11:51 UTC 2017, x86_64
2.7.13 (default, Jan 12 2017, 17:59:37)
[GCC 6.3.1 20161221 (Red Hat 6.3.1-1)] /usr/bin/python
{
  "ContainersPaused": 0,
  "Labels": null,
  "CgroupDriver": "cgroupfs",
  "ContainersRunning": 0,
  "ContainerdCommit": {
    "Expected": "4ab9917febca54791c5f071a9d1f404867857fcc",
    "ID": "4ab9917febca54791c5f071a9d1f404867857fcc"
  },
  "InitBinary": "docker-init",
  "NGoroutines": 23,
  "Swarm": {
    "Managers": 0,
    "ControlAvailable": false,
    "NodeID": "",
    "Cluster": {
      "Spec": {
        "TaskDefaults": {},
        "Orchestration": {},
        "EncryptionConfig": {
          "AutoLockManagers": false
        },
        "Raft": {
          "HeartbeatTick": 0,
          "ElectionTick": 0
        },
        "CAConfig": {},
        "Dispatcher": {}
      },
      "Version": {},
      "ID": "",
      "CreatedAt": "0001-01-01T00:00:00Z",
      "UpdatedAt": "0001-01-01T00:00:00Z"
    },
    "Nodes": 0,
    "Error": "",
    "RemoteManagers": null,
    "LocalNodeState": "inactive",
    "NodeAddr": ""
  },
  "LoggingDriver": "json-file",
  "OSType": "linux",
  "HttpProxy": "",
  "Runtimes": {
    "runc": {
      "path": "docker-runc"
    }
  },
  "DriverStatus": [
    [
      "Pool Name",
      "docker-253:0-1049581-pool"
    ],
    [
      "Pool Blocksize",
      "65.54 kB"
    ],
    [
      "Base Device Size",
      "107.4 GB"
    ],
    [
      "Backing Filesystem",
      "ext4"
    ],
    [
      "Data file",
      "/dev/loop0"
    ],
    [
      "Metadata file",
      "/dev/loop1"
    ],
    [
      "Data Space Used",
      "4.018 GB"
    ],
    [
      "Data Space Total",
      "107.4 GB"
    ],
    [
      "Data Space Available",
      "28.64 GB"
    ],
    [
      "Metadata Space Used",
      "5.038 MB"
    ],
    [
      "Metadata Space Total",
      "2.147 GB"
    ],
    [
      "Metadata Space Available",
      "2.142 GB"
    ],
    [
      "Thin Pool Minimum Free Space",
      "10.74 GB"
    ],
    [
      "Udev Sync Supported",
      "true"
    ],
    [
      "Deferred Removal Enabled",
      "false"
    ],
    [
      "Deferred Deletion Enabled",
      "false"
    ],
    [
      "Deferred Deleted Device Count",
      "0"
    ],
    [
      "Data loop file",
      "/var/lib/docker/devicemapper/devicemapper/data"
    ],
    [
      "Metadata loop file",
      "/var/lib/docker/devicemapper/devicemapper/metadata"
    ],
    [
      "Library Version",
      "1.02.136 (2016-11-05)"
    ]
  ],
  "OperatingSystem": "Fedora 25 (Twenty Five)",
  "Containers": 8,
  "HttpsProxy": "",
  "BridgeNfIp6tables": true,
  "MemTotal": 1542840320,
  "SecurityOptions": [
    "name=seccomp,profile=default"
  ],
  "Driver": "devicemapper",
  "IndexServerAddress": "https://index.docker.io/v1/",
  "ClusterStore": "",
  "InitCommit": {
    "Expected": "949e6fa",
    "ID": "949e6fa"
  },
  "Isolation": "",
  "SystemStatus": null,
  "OomKillDisable": true,
  "ClusterAdvertise": "",
  "SystemTime": "2017-04-20T20:08:53.743331778Z",
  "Name": "node1",
  "CPUSet": true,
  "RegistryConfig": {
    "InsecureRegistryCIDRs": [
      "127.0.0.0/8"
    ],
    "IndexConfigs": {
      "docker.io": {
        "Official": true,
        "Name": "docker.io",
        "Secure": true,
        "Mirrors": null
      }
    },
    "Mirrors": []
  },
  "DefaultRuntime": "runc",
  "ContainersStopped": 8,
  "NCPU": 1,
  "NFd": 16,
  "Architecture": "x86_64",
  "KernelMemory": true,
  "CpuCfsQuota": true,
  "Debug": false,
  "ID": "SKUR:BA5G:52G2:4G5Z:SYU2:2YL2:7Y5N:75LA:3WYN:OQS3:SVIL:JARF",
  "IPv4Forwarding": true,
  "KernelVersion": "4.10.10-200.fc25.x86_64",
  "BridgeNfIptables": true,
  "NoProxy": "",
  "LiveRestoreEnabled": false,
  "ServerVersion": "17.03.1-ce",
  "CpuCfsPeriod": true,
  "ExperimentalBuild": false,
  "MemoryLimit": true,
  "SwapLimit": true,
  "Plugins": {
    "Volume": [
      "local"
    ],
    "Network": [
      "bridge",
      "host",
      "macvlan",
      "null",
      "overlay"
    ],
    "Authorization": null
  },
  "Images": 46,
  "DockerRootDir": "/var/lib/docker",
  "NEventsListener": 0,
  "CPUShares": true,
  "RuncCommit": {
    "Expected": "54296cf40ad8143b62dbcaa1d90e520a2136ddfe",
    "ID": "54296cf40ad8143b62dbcaa1d90e520a2136ddfe"
  }
}
{
  "KernelVersion": "4.10.10-200.fc25.x86_64",
  "Arch": "amd64",
  "BuildTime": "2017-03-27T17:14:43.435239605+00:00",
  "ApiVersion": "1.27",
  "Version": "17.03.1-ce",
  "MinAPIVersion": "1.12",
  "GitCommit": "c6d412e",
  "Os": "linux",
  "GoVersion": "go1.7.5"
}
SUMMARY

The build is failing.

STEPS TO REPRODUCE
[vagrant@node1:/home/vagrant/projects/ansible-container-webstore]$ sudo ansible-container build
EXPECTED RESULTS

The build to succeed.

ACTUAL RESULTS
Building Docker Engine context...
Starting Docker build of Ansible Container Conductor image (please be patient)...
ERROR   Unknown exception
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/container/cli.py", line 259, in __call__
    getattr(core, u'hostcmd_{}'.format(args.subcommand))(**vars(args))
  File "/usr/lib/python2.7/site-packages/container/__init__.py", line 28, in __wrapped__
    return fn(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/container/core.py", line 144, in hostcmd_build
    cache=kwargs['cache']
  File "/usr/lib/python2.7/site-packages/container/docker/engine.py", line 81, in __wrapped__
    return fn(self, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/container/__init__.py", line 28, in __wrapped__
    return fn(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/container/docker/engine.py", line 706, in build_conductor_image
    nocache=not cache)
  File "/usr/lib/python2.7/site-packages/docker/models/images.py", line 172, in build
    raise BuildError(event.get('error') or event)
BuildError: The command '/bin/sh -c yum update -y &&     yum install -y epel-release   &&     yum install -y gcc git python-devel rsync libffi-devel openssl-devel &&     yum clean all' returned a non-zero code: 1
ghost commented 7 years ago

Where are those commands coming from yum update -y && yum install -y epel-release? Is this trying to build a new conductor docker image and within that docker container, it's now trying to run those yum commands? If that's the case, I think it could be failing because it can't reach the internet? The reason I say that is because my host uses a proxy to get out to the internet to bypass some ssl man in the middle stuff.

chouseknecht commented 7 years ago

@sarlindo

That's correct, when you run ansible-container build it builds the conductor image for the project. The Dockerfile for building the conductor is here, and that's where those commands come from.

Try setting HTTP_PROXY using --with-variables. That will set it in the conductor container. You will likely need to add it as an environment variable for the web service as well.

ghost commented 7 years ago

@chouseknecht thanks.

Just tried this but same issues. I will have to dig a bit further.

ansible-container build -e HTTP_PROXY=http://IP:PORT

ghost commented 7 years ago

@chouseknecht actually may not be the issue. Here is what I did.

I created and attached to the centos 7 image

centos              7                   a8493f5f50ff        2 weeks ago         192 MB
docker run -i -t a8493f5f50ff /bin/bash

I then ran these commands in the container.

yum install -y epel-release   &&     yum install -y gcc git python-devel rsync libffi-devel openssl-devel &&     yum clean all

and it's failed on this, looks like issue with EPEL repo

 One of the configured repositories failed (Unknown),
 and yum doesn't have enough cached data to continue. At this point the only
 safe thing yum can do is fail. There are a few ways to work "fix" this:

     1. Contact the upstream for the repository and get them to fix the problem.

     2. Reconfigure the baseurl/etc. for the repository, to point to a working
        upstream. This is most often useful if you are using a newer
        distribution release than is supported by the repository (and the
        packages for the previous distribution release still work).

     3. Run the command with the repository temporarily disabled
            yum --disablerepo=<repoid> ...

     4. Disable the repository permanently, so yum won't use it by default. Yum
        will then just ignore the repository until you permanently enable it
        again or use --enablerepo for temporary usage:

            yum-config-manager --disable <repoid>
        or
            subscription-manager repos --disable=<repoid>

     5. Configure the failing repository to be skipped, if it is unavailable.
        Note that yum will try to contact the repo. when it runs most commands,
        so will have to try and fail each time (and thus. yum will be be much
        slower). If it is a very temporary problem though, this is often a nice
        compromise:

            yum-config-manager --save --setopt=<repoid>.skip_if_unavailable=true

Cannot retrieve metalink for repository: epel/x86_64. Please verify its path and try again
chouseknecht commented 7 years ago

@sarlindo you may need to first run yum update prior to yum install.

As for getting the proxy to work, you might try the variable in lowercase, as suggested in this thread: https://github.com/moby/moby/issues/24697

ghost commented 7 years ago

@chouseknecht sorry I did run yum update, just didn't copy that into the previous message

yum update -y &&     yum install -y epel-release   &&     yum install -y gcc git python-devel rsync libffi-devel openssl-devel &&     yum clean all

I got those commands to work in my centos container after I set the proxy variable "export https_proxy=xxxx".

However still not working with passing the variable to the build, whether it's upper case or lower case.

ansible-container build --with-variables HTTP_PROXY=http://IP:PORT ansible-container build --with-variables http_proxy=http://IP:PORT

Where can I see the actual log with the exact errors from the ansible-container build?

j00bar commented 7 years ago

@sarlindo To see the complete log, add --debug, as in ansible-container --debug build.

ghost commented 7 years ago

@j00bar @chouseknecht Thanks the debug flag did show the same "epel" issue which means it's proxy related. I got this to work, but directly modifying the dockerfile for the conductor container because this was not working passing in variables with --with-variables (shown below). I also had to add these proxy statements under environment: to the container.yml file or my build would not work. Having said that, I only need the proxy for the build and not the run, how can I control this granularity? 

ENV http_proxy=XXX
ENV https_proxy=XXX

Also, a couple of observations (may not be related to this issue, but let me know if I should instead open a separate one).

  1. For ansible roles, the "meta" folder must exist with at the very least in main.yml "dependencies: []" or build fails, is this expected behavior?
  2. For existing roles that work fine for VM's or baremetal may not work with containers because some depend on service module (systemd) to restart services because of the good old battle with docker pid 1, systemd etc... What is the recommendation around this issue?
  3. Ansible groupvars are they supported by ansible-container?
  4. I have roles that use AWS to pull things from S3 using "local action" which runs on the controller machine instead of the remote VM being targeted. How do I accomplish this with ansible-container? I don't want to have to run this from within the targeted container because I would then have to setup boto, awscli etc... aws keys.
chouseknecht commented 7 years ago

@sarlindo

  1. has been fixed. A role does not need to have meta/main.yml
  2. For roles expecting an init system (i.e. systemd), the author will need to add a conditional check for ANSIBLE_CONTAINER=1 in the environment. If true, then the role is operating in a container, and there is likely no init system.
  3. There is not presently support for group vars.
  4. I think you could add boto to the requirements.txt file for your project. This would install it into the conductor. And then, set the AWS credentials as environment vars in the conductor using --with-variables.
ghost commented 7 years ago

@chouseknecht thanks for the reply and correcting the meta folder requirement.

A few more questions and that's it from me, I promise :^)

  1. Are there plans for group var support, I think this is a big one for reuse of existing ansible projects.
  2. In regards to proxy, I only need it for the build and not the run, how can I control this granularity?
  3. So any role with "local action" will actually be running on the conductor images, is that correct?
  4. I am having a hard time understanding this project VS ansible docker_service that can read compose files, is the biggest advantage the addition of the wrapper that can allow for existing ansible roles to be used?
  5. can the conductor image also be controlled through roles? or must I modify the dockerfile for the conductor?
ghost commented 7 years ago

I'm seeing the same problem. Building the Ansible Container Conductor image fails behind proxy because the proxy settings are not passed to docker build as build arguments. The following hack fixes the problem:

Index: container/docker/engine.py
===================================================================
--- container/docker/engine.py  (revision 8a24cb4ff10485e2cf5ba8e2e35bf1d018675281)
+++ container/docker/engine.py  (revision )
@@ -689,12 +689,17 @@
             tarball_file.close()
             tarball_file = open(tarball_path, 'rb')
             logger.info('Starting Docker build of Ansible Container Conductor image (please be patient)...')
+            # pass proxy settings to Docker
+            buildargs = {var: os.environ[var]
+                         for var in ('http_proxy', 'https_proxy', 'no_proxy')
+                         if var in os.environ}
             # FIXME: Error out properly if build of conductor fails.
             if self.debug:
                 for line_json in self.client.api.build(fileobj=tarball_file,
                                                   decode=True,
                                                   custom_context=True,
                                                   tag=self.image_name_for_service('conductor'),
+                                                  buildargs=buildargs,
                                                   rm=True,
                                                   nocache=not cache):
                     try:
@@ -718,6 +723,7 @@
                 image = self.client.images.build(fileobj=tarball_file,
                                                  custom_context=True,
                                                  tag=self.image_name_for_service('conductor'),
+                                                 buildargs=buildargs,
                                                  rm=True,
                                                  nocache=not cache)
                 return image.id
davidebelloni commented 7 years ago

Hi, I've the same problem and the proxy variables is also not inherited by 'docker for mac' settings!

Here my 'ansible-container --debug version' output:

Ansible Container, version 0.9.0.0
Darwin, FGRDDavide-Belloni.local, 16.5.0, Darwin Kernel Version 16.5.0: Fri Mar  3 16:52:33 PST 2017; root:xnu-3789.51.2~3/RELEASE_X86_64, x86_64
2.7.13 (default, May 12 2017, 11:09:30) 
[GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.42)] /Users/davidebelloni/Software/ansible-devel/bin/python2.7
{
  "ContainersPaused": 0, 
  "Labels": null, 
  "CgroupDriver": "cgroupfs", 
  "ContainersRunning": 0, 
  "ContainerdCommit": {
    "Expected": "4ab9917febca54791c5f071a9d1f404867857fcc", 
    "ID": "4ab9917febca54791c5f071a9d1f404867857fcc"
  }, 
  "InitBinary": "docker-init", 
  "NGoroutines": 27, 
  "Swarm": {
    "Managers": 0, 
    "ControlAvailable": false, 
    "NodeID": "", 
    "Cluster": {
      "Spec": {
        "TaskDefaults": {}, 
        "Orchestration": {}, 
        "EncryptionConfig": {
          "AutoLockManagers": false
        }, 
        "Raft": {
          "HeartbeatTick": 0, 
          "ElectionTick": 0
        }, 
        "CAConfig": {}, 
        "Dispatcher": {}
      }, 
      "Version": {}, 
      "ID": "", 
      "CreatedAt": "0001-01-01T00:00:00Z", 
      "UpdatedAt": "0001-01-01T00:00:00Z"
    }, 
    "Nodes": 0, 
    "Error": "", 
    "RemoteManagers": null, 
    "LocalNodeState": "inactive", 
    "NodeAddr": ""
  }, 
  "LoggingDriver": "json-file", 
  "OSType": "linux", 
  "HttpProxy": "http://proxy.org.com:8080", 
  "Runtimes": {
    "runc": {
      "path": "docker-runc"
    }
  }, 
  "DriverStatus": [
    [
      "Backing Filesystem", 
      "extfs"
    ], 
    [
      "Supports d_type", 
      "true"
    ], 
    [
      "Native Overlay Diff", 
      "true"
    ]
  ], 
  "OperatingSystem": "Alpine Linux v3.5", 
  "Containers": 13, 
  "HttpsProxy": "http://proxy.org.com:8080", 
  "BridgeNfIp6tables": true, 
  "MemTotal": 2096185344, 
  "SecurityOptions": [
    "name=seccomp,profile=default"
  ], 
  "Driver": "overlay2", 
  "IndexServerAddress": "https://index.docker.io/v1/", 
  "ClusterStore": "", 
  "InitCommit": {
    "Expected": "949e6facb77383876aeff8a6944dde66b3089574", 
    "ID": "N/A"
  }, 
  "Isolation": "", 
  "SystemStatus": null, 
  "OomKillDisable": true, 
  "ClusterAdvertise": "", 
  "SystemTime": "2017-05-12T10:16:13.078057488Z", 
  "Name": "moby", 
  "CPUSet": true, 
  "RegistryConfig": {
    "InsecureRegistryCIDRs": [
      "127.0.0.0/8"
    ], 
    "IndexConfigs": {
      "docker.io": {
        "Official": true, 
        "Name": "docker.io", 
        "Secure": true, 
        "Mirrors": null
      }
    }, 
    "Mirrors": []
  }, 
  "DefaultRuntime": "runc", 
  "ContainersStopped": 13, 
  "NCPU": 2, 
  "NFd": 16, 
  "Architecture": "x86_64", 
  "KernelMemory": true, 
  "CpuCfsQuota": true, 
  "Debug": true, 
  "ID": "ZHSS:WJN6:67LK:63TO:HZX4:ILQ5:Z6WT:K6FQ:RW2O:D7T2:7LTK:HUPI", 
  "IPv4Forwarding": true, 
  "KernelVersion": "4.9.13-moby", 
  "BridgeNfIptables": true, 
  "NoProxy": "*.local,169.254/16", 
  "LiveRestoreEnabled": false, 
  "ServerVersion": "17.03.1-ce", 
  "CpuCfsPeriod": true, 
  "ExperimentalBuild": true, 
  "MemoryLimit": true, 
  "SwapLimit": true, 
  "Plugins": {
    "Volume": [
      "local"
    ], 
    "Network": [
      "bridge", 
      "host", 
      "ipvlan", 
      "macvlan", 
      "null", 
      "overlay"
    ], 
    "Authorization": null
  }, 
  "Images": 16, 
  "DockerRootDir": "/var/lib/docker", 
  "NEventsListener": 1, 
  "CPUShares": true, 
  "RuncCommit": {
    "Expected": "54296cf40ad8143b62dbcaa1d90e520a2136ddfe", 
    "ID": "54296cf40ad8143b62dbcaa1d90e520a2136ddfe"
  }
}
{
  "KernelVersion": "4.9.13-moby", 
  "Arch": "amd64", 
  "BuildTime": "2017-03-24T00:00:50.070226199+00:00", 
  "ApiVersion": "1.27", 
  "Version": "17.03.1-ce", 
  "MinAPIVersion": "1.12", 
  "GitCommit": "c6d412e", 
  "Os": "linux", 
  "Experimental": true, 
  "GoVersion": "go1.7.5"
}

Here 'ansible-container --debug build' output:

2017-05-12T12:18:22.221074 The default type is            [container.config] caller_file=/Users/xxx/Software/ansible-devel/lib/python2.7/site-packages/container/config.py caller_func=_resolve_defaults caller_line=124 config=<class 'ruamel.yaml.comments.CommentedMap'> defaults=<type '_ordereddict.ordereddict'>
2017-05-12T12:18:22.224271 Getting environment variables... [container.config] caller_file=/Users/xxx/Software/ansible-devel/lib/python2.7/site-packages/container/config.py caller_func=_get_environment_variables caller_line=140
2017-05-12T12:18:22.226759 Read environment variables     [container.config] caller_file=/Users/xxx/Software/ansible-devel/lib/python2.7/site-packages/container/config.py caller_func=_get_environment_variables caller_line=145 env_vars={}
2017-05-12T12:18:22.229373 Resolved template variables    [container.config] caller_file=/Users/xxx/Software/ansible-devel/lib/python2.7/site-packages/container/config.py caller_func=_resolve_defaults caller_line=129 template_vars={}
2017-05-12T12:18:22.232269 Parsed config                  [container.config] caller_file=/Users/xxx/Software/ansible-devel/lib/python2.7/site-packages/container/config.py caller_func=set_env caller_line=110 config={"version": "2", "settings": {"conductor_base": "debian:jessie"}, "services": {"runtime": {"roles": [{"role": "app", "CONTAINER_USER": "rnd", "SOURCE_HOME": "source"}, {"role": "cleanup", "tags": ["cleanup"]}], "from": "base-images-runtime:latest", "labels": {"com.org.githash": "$BUILDCOMMIT", "com.org.image": "$BUILDIMAGE-runtime"}, "environment": ["LANG=C.UTF-8", "PYTHONUNBUFFERED=1", "PATH=/env/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "VIRTUAL_ENV=/env"], "expose": ["8000"], "working_dir": "/source", "command": ["/usr/local/bin/proxychains_entry.sh", "gunicorn", "--log-config", "logging.conf", "-b", "0.0.0.0:8000", "-w", "4", "myapp:app"], "entrypoint": ["/usr/local/bin/dumb-init", "--"], "user": "rnd", "links": ["squid", "memcached"], "options": {"kube": {"runAsUser": 1000}}}, "apache": {"roles": [{"role": "httpd_files", "CONTAINER_USER": "httpd"}, {"role": "cleanup", "tags": ["cleanup"]}], "from": "base-images-apache:latest", "ports": ["80:8080", "443:44443"], "working_dir": "/usr/local/apache2", "command": ["/usr/local/apache2/bin/httpd", "-DFOREGROUND"], "entrypoint": ["/usr/local/bin/dumb-init", "--"], "volumes": ["/usr/local/apache2/logs"], "user": "httpd", "options": {"kube": {"runAsUser": 1000}}}, "squid": {"from": "base-images-squid:latest", "roles": [{"role": "squid_files", "CONTAINER_USER": "squidd"}, {"role": "cleanup", "tags": ["cleanup"]}], "expose": ["3128"], "working_dir": "/usr/local/squid", "entrypoint": ["/usr/local/squid/sbin/auto-reload.sh"], "volumes": ["/usr/local/squid/var/logs"], "user": "squidd", "options": {"kube": {"runAsUser": 1000}}}, "memcached": {"from": "memcached:1.4.34", "ports": ["11211"], "command": ["/usr/local/bin/memcached", "-v", "-p", "11211", "-U", "0", "-u", "memcached", "-l", "0.0.0.0", "-c", "1024", "-m", "1024"], "user": "999", "options": {"kube": {"runAsUser": 999}}}, "dnsmasq": {"from": "andyshinn/dnsmasq", "ports": ["53535:53/tcp", "53535:53/udp"], "entrypoint": ["/dnsmasq_entrypoint.sh", "$NODNSMASQ"], "volumes": ["$PWD/ansible/dnsmasq_entrypoint.sh:/dnsmasq_entrypoint.sh"]}}, "registries": {}, "defaults": {}}
2017-05-12T12:18:22.235680 Loading engine capabilities    [container.utils.loader] caller_file=/Users/xxx/Software/ansible-devel/lib/python2.7/site-packages/container/utils/loader.py caller_func=load_engine caller_line=14 capabilities=['BUILD', 'RUN'] engine=docker
2017-05-12T12:18:22.283569 Could not find container for conductor [container.docker.engine] all_containers=[] caller_file=/Users/xxx/Software/ansible-devel/lib/python2.7/site-packages/container/docker/engine.py caller_func=get_container_id_for_service caller_line=351 container=u'ansible_conductor'
2017-05-12T12:18:22.296154 Could not find the latest image for service, searching for other tags with same image name [container.docker.engine] caller_file=/Users/xxx/Software/ansible-devel/lib/python2.7/site-packages/container/docker/engine.py caller_func=get_latest_image_for_service caller_line=390 image_name=u'ansible-conductor' service=conductor
2017-05-12T12:18:22.302898 Call: Engine.build_conductor_image [container.docker.engine] args=('/Users/xxx/git/progetto-0/ansible', 'debian:jessie') caller_file=/Users/xxx/Software/ansible-devel/lib/python2.7/site-packages/container/docker/engine.py caller_func=Engine.build_conductor_image caller_line=23 kwargs={'cache': True}
2017-05-12T12:18:22.309667 Using temporary directory      [container.utils.temp] caller_file=/Users/xxx/Software/ansible-devel/lib/python2.7/site-packages/container/utils/temp.py caller_func=__enter__ caller_line=17 path=/var/folders/hq/jbnj8s1s5751mgtx8j5t4qq80000gn/T/tmpe7AZ8y
2017-05-12T12:18:22.313518 Building Docker Engine context... [container.docker.engine] caller_file=/Users/xxx/Software/ansible-devel/lib/python2.7/site-packages/container/docker/engine.py caller_func=build_conductor_image caller_line=611
2017-05-12T12:18:22.354920 Rendered Jinja Template:       [container.utils] body=FROM debian:jessie
ENV ANSIBLE_CONTAINER=1

RUN apt-get update -y && \
    apt-get install -y python2.7 git python-dev rsync libffi-dev libssl-dev python-apt && \
    cd /usr/bin && \
    ln -fs python2.7 python && \
    apt-get clean

ADD https://get.docker.com/builds/Linux/x86_64/docker-17.04.0-ce.tgz /tmp/docker.tgz

COPY /contrib/get-pip.py /get-pip.py
RUN python /get-pip.py && \
    mkdir -p /etc/ansible/roles /_ansible/src && \
    cd /usr/local/bin && \
    tar -xz --strip-components=1 -f /tmp/docker.tgz

# The COPY here will break cache if the version of conductor changed
COPY /container-src /_ansible/container
RUN cd /_ansible && \
    pip install -r container/conductor-build/conductor-requirements.txt && \
    PYTHONPATH=. LC_ALL="en_US.UTF-8" python container/conductor-build/setup.py develop -v && \
    ansible-galaxy install -p /etc/ansible/roles -r container/conductor-build/conductor-requirements.yml

# The COPY here will break cache if the requirements or ansible.cfg has changed
COPY /build-src /_ansible/build
RUN ( test -f /_ansible/build/ansible-requirements.txt && pip install --no-cache-dir -r /_ansible/build/ansible-requirements.txt || true ) && \
    ( test -f /_ansible/build/requirements.yml && ansible-galaxy install -p /etc/ansible/roles -r /_ansible/build/requirements.yml || true ) && \
    ( test -f /_ansible/build/ansible.cfg && cp /_ansible/build/ansible.cfg /etc/ansible/ansible.cfg || true)

VOLUME /usr

 caller_file=/Users/xxx/Software/ansible-devel/lib/python2.7/site-packages/container/utils/__init__.py caller_func=jinja_render_to_temp caller_line=76
2017-05-12T12:18:22.359810 Context manifest:              [container.docker.engine] caller_file=/Users/xxx/Software/ansible-devel/lib/python2.7/site-packages/container/docker/engine.py caller_func=build_conductor_image caller_line=667
2017-05-12T12:18:22.361481 tarball item: build-src/ansible.cfg (190 bytes) [container.docker.engine] bytes=190 file=build-src/ansible.cfg
2017-05-12T12:18:22.362696 tarball item: build-src/ansible-requirements.txt (146 bytes) [container.docker.engine] bytes=146 file=build-src/ansible-requirements.txt
2017-05-12T12:18:22.363894 tarball item: build-src/requirements.yml (294 bytes) [container.docker.engine] bytes=294 file=build-src/requirements.yml
2017-05-12T12:18:22.365214 tarball item: build-src/.touch (0 bytes) [container.docker.engine] bytes=0 file=build-src/.touch
2017-05-12T12:18:22.366795 tarball item: contrib/get-pip.py (1595409 bytes) [container.docker.engine] bytes=1595409 file=contrib/get-pip.py
2017-05-12T12:18:22.368514 tarball item: container-src (0 bytes) [container.docker.engine] bytes=0L file=container-src
2017-05-12T12:18:22.369752 tarball item: container-src/__init__.py (949 bytes) [container.docker.engine] bytes=949 file=container-src/__init__.py
2017-05-12T12:18:22.371351 tarball item: container-src/__init__.pyc (1688 bytes) [container.docker.engine] bytes=1688 file=container-src/__init__.pyc
2017-05-12T12:18:22.372616 tarball item: container-src/cli.py (19017 bytes) [container.docker.engine] bytes=19017 file=container-src/cli.py
2017-05-12T12:18:22.373835 tarball item: container-src/cli.pyc (16343 bytes) [container.docker.engine] bytes=16343 file=container-src/cli.pyc
2017-05-12T12:18:22.375285 tarball item: container-src/config.py (12751 bytes) [container.docker.engine] bytes=12751 file=container-src/config.py
2017-05-12T12:18:22.377259 tarball item: container-src/config.pyc (13251 bytes) [container.docker.engine] bytes=13251 file=container-src/config.pyc
2017-05-12T12:18:22.378535 tarball item: container-src/core.py (35686 bytes) [container.docker.engine] bytes=35686 file=container-src/core.py
2017-05-12T12:18:22.379763 tarball item: container-src/core.pyc (27537 bytes) [container.docker.engine] bytes=27537 file=container-src/core.pyc
2017-05-12T12:18:22.380979 tarball item: container-src/docker (0 bytes) [container.docker.engine] bytes=0L file=container-src/docker
2017-05-12T12:18:22.382197 tarball item: container-src/docker/__init__.py (205 bytes) [container.docker.engine] bytes=205 file=container-src/docker/__init__.py
2017-05-12T12:18:22.383406 tarball item: container-src/docker/__init__.pyc (386 bytes) [container.docker.engine] bytes=386 file=container-src/docker/__init__.pyc
2017-05-12T12:18:22.384605 tarball item: container-src/docker/deploy.py (248 bytes) [container.docker.engine] bytes=248 file=container-src/docker/deploy.py
2017-05-12T12:18:22.385797 tarball item: container-src/docker/deploy.pyc (384 bytes) [container.docker.engine] bytes=384 file=container-src/docker/deploy.pyc
2017-05-12T12:18:22.387382 tarball item: container-src/docker/engine.py (32737 bytes) [container.docker.engine] bytes=32737 file=container-src/docker/engine.py
2017-05-12T12:18:22.388817 tarball item: container-src/docker/engine.pyc (30357 bytes) [container.docker.engine] bytes=30357 file=container-src/docker/engine.pyc
2017-05-12T12:18:22.390177 tarball item: container-src/docker/files (0 bytes) [container.docker.engine] bytes=0L file=container-src/docker/files
2017-05-12T12:18:22.391870 tarball item: container-src/docker/files/conductor-requirements.txt (278 bytes) [container.docker.engine] bytes=278 file=container-src/docker/files/conductor-requirements.txt
2017-05-12T12:18:22.394698 tarball item: container-src/docker/files/conductor-requirements.yml (60 bytes) [container.docker.engine] bytes=60 file=container-src/docker/files/conductor-requirements.yml
2017-05-12T12:18:22.397002 tarball item: container-src/docker/files/get-pip.py (1595409 bytes) [container.docker.engine] bytes=1595409 file=container-src/docker/files/get-pip.py
2017-05-12T12:18:22.399303 tarball item: container-src/docker/files/get-pip.pyc (1593830 bytes) [container.docker.engine] bytes=1593830 file=container-src/docker/files/get-pip.pyc
2017-05-12T12:18:22.401586 tarball item: container-src/docker/files/setup.py (3325 bytes) [container.docker.engine] bytes=3325 file=container-src/docker/files/setup.py
2017-05-12T12:18:22.404001 tarball item: container-src/docker/files/setup.pyc (4636 bytes) [container.docker.engine] bytes=4636 file=container-src/docker/files/setup.pyc
2017-05-12T12:18:22.408521 tarball item: container-src/docker/importer.py (22882 bytes) [container.docker.engine] bytes=22882 file=container-src/docker/importer.py
2017-05-12T12:18:22.411019 tarball item: container-src/docker/importer.pyc (21455 bytes) [container.docker.engine] bytes=21455 file=container-src/docker/importer.pyc
2017-05-12T12:18:22.414777 tarball item: container-src/docker/templates (0 bytes) [container.docker.engine] bytes=0L file=container-src/docker/templates
2017-05-12T12:18:22.416304 tarball item: container-src/docker/templates/conductor-dockerfile.j2 (1902 bytes) [container.docker.engine] bytes=1902 file=container-src/docker/templates/conductor-dockerfile.j2
2017-05-12T12:18:22.417597 tarball item: container-src/engine.py (6057 bytes) [container.docker.engine] bytes=6057 file=container-src/engine.py
2017-05-12T12:18:22.418889 tarball item: container-src/engine.pyc (11741 bytes) [container.docker.engine] bytes=11741 file=container-src/engine.pyc
2017-05-12T12:18:22.420278 tarball item: container-src/exceptions.py (1991 bytes) [container.docker.engine] bytes=1991 file=container-src/exceptions.py
2017-05-12T12:18:22.421800 tarball item: container-src/exceptions.pyc (6641 bytes) [container.docker.engine] bytes=6641 file=container-src/exceptions.pyc
2017-05-12T12:18:22.423053 tarball item: container-src/k8s (0 bytes) [container.docker.engine] bytes=0L file=container-src/k8s
2017-05-12T12:18:22.424325 tarball item: container-src/k8s/__init__.py (196 bytes) [container.docker.engine] bytes=196 file=container-src/k8s/__init__.py
2017-05-12T12:18:22.425536 tarball item: container-src/k8s/__init__.pyc (383 bytes) [container.docker.engine] bytes=383 file=container-src/k8s/__init__.pyc
2017-05-12T12:18:22.426731 tarball item: container-src/k8s/base_deploy.py (27642 bytes) [container.docker.engine] bytes=27642 file=container-src/k8s/base_deploy.py
2017-05-12T12:18:22.427971 tarball item: container-src/k8s/base_deploy.pyc (21612 bytes) [container.docker.engine] bytes=21612 file=container-src/k8s/base_deploy.pyc
2017-05-12T12:18:22.429164 tarball item: container-src/k8s/base_engine.py (4983 bytes) [container.docker.engine] bytes=4983 file=container-src/k8s/base_engine.py
2017-05-12T12:18:22.430354 tarball item: container-src/k8s/base_engine.pyc (5895 bytes) [container.docker.engine] bytes=5895 file=container-src/k8s/base_engine.pyc
2017-05-12T12:18:22.431545 tarball item: container-src/k8s/deploy.py (1720 bytes) [container.docker.engine] bytes=1720 file=container-src/k8s/deploy.py
2017-05-12T12:18:22.432730 tarball item: container-src/k8s/deploy.pyc (2256 bytes) [container.docker.engine] bytes=2256 file=container-src/k8s/deploy.pyc
2017-05-12T12:18:22.433930 tarball item: container-src/k8s/engine.py (2389 bytes) [container.docker.engine] bytes=2389 file=container-src/k8s/engine.py
2017-05-12T12:18:22.435122 tarball item: container-src/k8s/engine.pyc (2188 bytes) [container.docker.engine] bytes=2188 file=container-src/k8s/engine.pyc
2017-05-12T12:18:22.436311 tarball item: container-src/openshift (0 bytes) [container.docker.engine] bytes=0L file=container-src/openshift
2017-05-12T12:18:22.437930 tarball item: container-src/openshift/__init__.py (196 bytes) [container.docker.engine] bytes=196 file=container-src/openshift/__init__.py
2017-05-12T12:18:22.439202 tarball item: container-src/openshift/__init__.pyc (389 bytes) [container.docker.engine] bytes=389 file=container-src/openshift/__init__.pyc
2017-05-12T12:18:22.440430 tarball item: container-src/openshift/deploy.py (6203 bytes) [container.docker.engine] bytes=6203 file=container-src/openshift/deploy.py
2017-05-12T12:18:22.442121 tarball item: container-src/openshift/deploy.pyc (5658 bytes) [container.docker.engine] bytes=5658 file=container-src/openshift/deploy.pyc
2017-05-12T12:18:22.444932 tarball item: container-src/openshift/engine.py (2151 bytes) [container.docker.engine] bytes=2151 file=container-src/openshift/engine.py
2017-05-12T12:18:22.447762 tarball item: container-src/openshift/engine.pyc (2960 bytes) [container.docker.engine] bytes=2960 file=container-src/openshift/engine.pyc
2017-05-12T12:18:22.450059 tarball item: container-src/templates (0 bytes) [container.docker.engine] bytes=0L file=container-src/templates
2017-05-12T12:18:22.452289 tarball item: container-src/templates/ansible-container-inventory.py (1230 bytes) [container.docker.engine] bytes=1230 file=container-src/templates/ansible-container-inventory.py
2017-05-12T12:18:22.455021 tarball item: container-src/templates/ansible-container-inventory.pyc (2327 bytes) [container.docker.engine] bytes=2327 file=container-src/templates/ansible-container-inventory.pyc
2017-05-12T12:18:22.457373 tarball item: container-src/templates/ansible-dockerfile.j2 (1194 bytes) [container.docker.engine] bytes=1194 file=container-src/templates/ansible-dockerfile.j2
2017-05-12T12:18:22.458861 tarball item: container-src/templates/ansible.cfg (41 bytes) [container.docker.engine] bytes=41 file=container-src/templates/ansible.cfg
2017-05-12T12:18:22.460159 tarball item: container-src/templates/build-docker-compose.j2.yml (1635 bytes) [container.docker.engine] bytes=1635 file=container-src/templates/build-docker-compose.j2.yml
2017-05-12T12:18:22.462795 tarball item: container-src/templates/builder.sh (620 bytes) [container.docker.engine] bytes=620 file=container-src/templates/builder.sh
2017-05-12T12:18:22.464049 tarball item: container-src/templates/compose_versioned.j2.yml (237 bytes) [container.docker.engine] bytes=237 file=container-src/templates/compose_versioned.j2.yml
2017-05-12T12:18:22.465299 tarball item: container-src/templates/hosts.j2 (90 bytes) [container.docker.engine] bytes=90 file=container-src/templates/hosts.j2
2017-05-12T12:18:22.466599 tarball item: container-src/templates/init (0 bytes) [container.docker.engine] bytes=0L file=container-src/templates/init
2017-05-12T12:18:22.467964 tarball item: container-src/templates/init/ansible-requirements.j2.txt (130 bytes) [container.docker.engine] bytes=130 file=container-src/templates/init/ansible-requirements.j2.txt
2017-05-12T12:18:22.469401 tarball item: container-src/templates/init/ansible.j2.cfg (145 bytes) [container.docker.engine] bytes=145 file=container-src/templates/init/ansible.j2.cfg
2017-05-12T12:18:22.470818 tarball item: container-src/templates/init/container.j2.yml (2026 bytes) [container.docker.engine] bytes=2026 file=container-src/templates/init/container.j2.yml
2017-05-12T12:18:22.472327 tarball item: container-src/templates/init/meta.j2.yml (1231 bytes) [container.docker.engine] bytes=1231 file=container-src/templates/init/meta.j2.yml
2017-05-12T12:18:22.473599 tarball item: container-src/templates/init/requirements.j2.yml (298 bytes) [container.docker.engine] bytes=298 file=container-src/templates/init/requirements.j2.yml
2017-05-12T12:18:22.474974 tarball item: container-src/templates/install-docker-compose.j2.yml (712 bytes) [container.docker.engine] bytes=712 file=container-src/templates/install-docker-compose.j2.yml
2017-05-12T12:18:22.477793 tarball item: container-src/templates/listhosts-docker-compose.j2.yml (1640 bytes) [container.docker.engine] bytes=1640 file=container-src/templates/listhosts-docker-compose.j2.yml
2017-05-12T12:18:22.480747 tarball item: container-src/templates/restart-docker-compose.j2.yml (75 bytes) [container.docker.engine] bytes=75 file=container-src/templates/restart-docker-compose.j2.yml
2017-05-12T12:18:22.482213 tarball item: container-src/templates/role (0 bytes) [container.docker.engine] bytes=0L file=container-src/templates/role
2017-05-12T12:18:22.483446 tarball item: container-src/templates/role/.travis.j2.yml (540 bytes) [container.docker.engine] bytes=540 file=container-src/templates/role/.travis.j2.yml
2017-05-12T12:18:22.484657 tarball item: container-src/templates/role/defaults (0 bytes) [container.docker.engine] bytes=0L file=container-src/templates/role/defaults
2017-05-12T12:18:22.485851 tarball item: container-src/templates/role/defaults/main.j2.yml (55 bytes) [container.docker.engine] bytes=55 file=container-src/templates/role/defaults/main.j2.yml
2017-05-12T12:18:22.487050 tarball item: container-src/templates/role/meta (0 bytes) [container.docker.engine] bytes=0L file=container-src/templates/role/meta
2017-05-12T12:18:22.488750 tarball item: container-src/templates/role/meta/main.j2.yml (3220 bytes) [container.docker.engine] bytes=3220 file=container-src/templates/role/meta/main.j2.yml
2017-05-12T12:18:22.490076 tarball item: container-src/templates/role/README.j2.md (1287 bytes) [container.docker.engine] bytes=1287 file=container-src/templates/role/README.j2.md
2017-05-12T12:18:22.491347 tarball item: container-src/templates/role/test (0 bytes) [container.docker.engine] bytes=0L file=container-src/templates/role/test
2017-05-12T12:18:22.492552 tarball item: container-src/templates/role/test/test.j2.yml (81 bytes) [container.docker.engine] bytes=81 file=container-src/templates/role/test/test.j2.yml
2017-05-12T12:18:22.493748 tarball item: container-src/templates/run-docker-compose.j2.yml (75 bytes) [container.docker.engine] bytes=75 file=container-src/templates/run-docker-compose.j2.yml
2017-05-12T12:18:22.494948 tarball item: container-src/templates/stop-docker-compose.j2.yml (75 bytes) [container.docker.engine] bytes=75 file=container-src/templates/stop-docker-compose.j2.yml
2017-05-12T12:18:22.496218 tarball item: container-src/templates/wait_on_host.py (2365 bytes) [container.docker.engine] bytes=2365 file=container-src/templates/wait_on_host.py
2017-05-12T12:18:22.497805 tarball item: container-src/templates/wait_on_host.pyc (2601 bytes) [container.docker.engine] bytes=2601 file=container-src/templates/wait_on_host.pyc
2017-05-12T12:18:22.500636 tarball item: container-src/utils (0 bytes) [container.docker.engine] bytes=0L file=container-src/utils
2017-05-12T12:18:22.502857 tarball item: container-src/utils/__init__.py (9585 bytes) [container.docker.engine] bytes=9585 file=container-src/utils/__init__.py
2017-05-12T12:18:22.506061 tarball item: container-src/utils/__init__.pyc (12114 bytes) [container.docker.engine] bytes=12114 file=container-src/utils/__init__.pyc
2017-05-12T12:18:22.508428 tarball item: container-src/utils/_text.py (12304 bytes) [container.docker.engine] bytes=12304 file=container-src/utils/_text.py
2017-05-12T12:18:22.510749 tarball item: container-src/utils/_text.pyc (8932 bytes) [container.docker.engine] bytes=8932 file=container-src/utils/_text.pyc
2017-05-12T12:18:22.513117 tarball item: container-src/utils/galaxy.py (8206 bytes) [container.docker.engine] bytes=8206 file=container-src/utils/galaxy.py
2017-05-12T12:18:22.514535 tarball item: container-src/utils/galaxy.pyc (8664 bytes) [container.docker.engine] bytes=8664 file=container-src/utils/galaxy.pyc
2017-05-12T12:18:22.517225 tarball item: container-src/utils/loader.py (906 bytes) [container.docker.engine] bytes=906 file=container-src/utils/loader.py
2017-05-12T12:18:22.518760 tarball item: container-src/utils/loader.pyc (1288 bytes) [container.docker.engine] bytes=1288 file=container-src/utils/loader.pyc
2017-05-12T12:18:22.520102 tarball item: container-src/utils/logmux.py (1313 bytes) [container.docker.engine] bytes=1313 file=container-src/utils/logmux.py
2017-05-12T12:18:22.521804 tarball item: container-src/utils/logmux.pyc (3090 bytes) [container.docker.engine] bytes=3090 file=container-src/utils/logmux.pyc
2017-05-12T12:18:22.523135 tarball item: container-src/utils/temp.py (698 bytes) [container.docker.engine] bytes=698 file=container-src/utils/temp.py
2017-05-12T12:18:22.524370 tarball item: container-src/utils/temp.pyc (1594 bytes) [container.docker.engine] bytes=1594 file=container-src/utils/temp.pyc
2017-05-12T12:18:22.525867 tarball item: container-src/utils/visibility.py (3329 bytes) [container.docker.engine] bytes=3329 file=container-src/utils/visibility.py
2017-05-12T12:18:22.528772 tarball item: container-src/utils/visibility.pyc (4650 bytes) [container.docker.engine] bytes=4650 file=container-src/utils/visibility.pyc
2017-05-12T12:18:22.531323 tarball item: container-src/conductor-build/setup.py (3325 bytes) [container.docker.engine] bytes=3325 file=container-src/conductor-build/setup.py
2017-05-12T12:18:22.532582 tarball item: container-src/conductor-build/conductor-requirements.txt (278 bytes) [container.docker.engine] bytes=278 file=container-src/conductor-build/conductor-requirements.txt
2017-05-12T12:18:22.533805 tarball item: container-src/conductor-build/conductor-requirements.yml (60 bytes) [container.docker.engine] bytes=60 file=container-src/conductor-build/conductor-requirements.yml
2017-05-12T12:18:22.535018 tarball item: Dockerfile (1456 bytes) [container.docker.engine] bytes=1456 file=Dockerfile
2017-05-12T12:18:22.540474 Starting Docker build of Ansible Container Conductor image (please be patient)... [container.docker.engine] caller_file=/Users/xxx/Software/ansible-devel/lib/python2.7/site-packages/container/docker/engine.py caller_func=build_conductor_image caller_line=675
Step 1/11 : FROM debian:jessie
 ---> e5599115b6a6
Step 2/11 : ENV ANSIBLE_CONTAINER 1
 ---> Using cache
 ---> 79a84459d18d
Step 3/11 : RUN apt-get update -y &&     apt-get install -y python2.7 git python-dev rsync libffi-dev libssl-dev python-apt &&     cd /usr/bin &&     ln -fs python2.7 python &&     apt-get clean
 ---> Running in a78bab6df4f0
chouseknecht commented 7 years ago

@davidebelloni, I don't see an error. Looks like you may not have pasted the entire output.

As far as setting http_proxy, the option --with-variables was not working. It was fixed yesterday by PR #516. If you run from source, you can pick up the change.

You may also need to set the http_proxy variable in the service, or target, container. Add it to the environment directive within container.yml. For example:

version: 2
settings:
   ...
defaults:
   proxy_server: "https://my.proxy.server"
services:
   web:
       environment:
           http_proxy: "{{ proxy_server }}"

Of course you could also set within a role, at the task level. There's example of doing that here.

geerlingguy commented 7 years ago

I am not using a proxy, but it seems I'm running into the same (general/vague) error in the OP during the tar task (monitored using --debug):

tar (child): /tmp/docker.tgz: Cannot read: Is a directory
tar (child): At beginning of tape, quitting now
tar (child): Error is not recoverable: exiting now

gzip: stdin: unexpected end of file
tar: Child returned status 2
tar: Error is not recoverable: exiting now

Edit: It looks like I'm actually running into https://github.com/ansible/ansible-container/issues/592 — though the resulting message displayed in the console is the same.