search

ansible / ansible-container

DEPRECATED -- Ansible Container was a tool to build Docker images and orchestrate containers using only Ansible playbooks.
GNU Lesser General Public License v3.0
2.19k stars 394 forks source link

Running ansible-container with --var-file seems to fail when yml file has non ascii characters in some cases #625

Closed pmarrone closed 7 years ago

pmarrone commented 7 years ago
ISSUE TYPE
container.yml
version: "2"
settings:

  conductor:
    # The Conductor container does the heavy lifting, and provides a portable
    # Python runtime for building your target containers. It should be derived
    # from the same distribution as you're building your target containers with.
    base: ubuntu:xenial
    # roles_path:   # Specify a local path containing Ansible roles
    # volumes:      # Provide a list of volumes to mount
    # environment:  # List or mapping of environment variables

  # Set the name of the project. Defaults to basename of the project directory.
  # For built services, concatenated with service name to form the built image name.
  project_name: uvfind-container

  # The deployment_output_path is mounted to the Conductor container, and the 
  # `run` and `deployment` commands then write generated Ansible playbooks to it.
  # deployment_output_path: ./ansible-deployment

  # Add your containers here, specifying the base image you want to build from.
  # To use this example, uncomment it and delete the curly braces after services key.
  # You may need to run `docker pull ubuntu:trusty` for this to work.
services:
  mysql:
    environment:
      MYSQL_DATABASE: vufind
      MYSQL_USER: vufind 
      MYSQL_PASSWORD: vufind
      MYSQL_ROOT_PASSWORD: root
    volumes:
      - ${PWD}/mysql-entrypoint.d:/docker-entrypoint-initdb.d
    from: mysql
    ports:
      - 3306:3306
  vu-find:
    from: base-vufind
    ports:
      - 8080:80
      - 8081:8080
    links:
      - mysql
    volumes:
      - ${PWD}/mongosocket:/tmp/mongosocket
    command: bash -c "/usr/local/vu-find/solr.sh start; /usr/sbin/apache2ctl -D FOREGROUND"
    roles:
      - role: preparecontainer
        gather_facts: false
      - vu-find

registries: {}
  # Add optional registries used for deployment. For example:
  #  google:
  #    url: https://gcr.io
  #    namespace: my-cool-project-xxxxxx
OS / ENVIRONMENT
Ansible Container, version 0.9.2rc0
Linux, bibliotecas-2, 4.4.0-78-generic, #99-Ubuntu SMP Thu Apr 27 15:29:09 UTC 2017, x86_64
2.7.12 (default, Nov 19 2016, 06:48:10) 
[GCC 5.4.0 20160609] /home/pmarrone/Proyectos/ansible-container/venv/bin/python
{
  "ContainersPaused": 0, 
  "Labels": null, 
  "CgroupDriver": "cgroupfs", 
  "ContainersRunning": 1, 
  "ContainerdCommit": {
    "Expected": "4ab9917febca54791c5f071a9d1f404867857fcc", 
    "ID": "4ab9917febca54791c5f071a9d1f404867857fcc"
  }, 
  "InitBinary": "docker-init", 
  "NGoroutines": 27, 
  "Swarm": {
    "Managers": 0, 
    "ControlAvailable": false, 
    "NodeID": "", 
    "Cluster": {
      "Spec": {
        "TaskDefaults": {}, 
        "Orchestration": {}, 
        "EncryptionConfig": {
          "AutoLockManagers": false
        }, 
        "Raft": {
          "HeartbeatTick": 0, 
          "ElectionTick": 0
        }, 
        "CAConfig": {}, 
        "Dispatcher": {}
      }, 
      "Version": {}, 
      "ID": "", 
      "CreatedAt": "0001-01-01T00:00:00Z", 
      "UpdatedAt": "0001-01-01T00:00:00Z"
    }, 
    "Nodes": 0, 
    "Error": "", 
    "RemoteManagers": null, 
    "LocalNodeState": "inactive", 
    "NodeAddr": ""
  }, 
  "LoggingDriver": "json-file", 
  "OSType": "linux", 
  "HttpProxy": "", 
  "Runtimes": {
    "runc": {
      "path": "docker-runc"
    }
  }, 
  "DriverStatus": [
    [
      "Root Dir", 
      "/var/lib/docker/aufs"
    ], 
    [
      "Backing Filesystem", 
      "extfs"
    ], 
    [
      "Dirs", 
      "526"
    ], 
    [
      "Dirperm1 Supported", 
      "true"
    ]
  ], 
  "OperatingSystem": "Ubuntu 16.04.2 LTS", 
  "Containers": 32, 
  "HttpsProxy": "", 
  "BridgeNfIp6tables": true, 
  "MemTotal": 8267018240, 
  "SecurityOptions": [
    "name=apparmor", 
    "name=seccomp,profile=default"
  ], 
  "Driver": "aufs", 
  "IndexServerAddress": "https://index.docker.io/v1/", 
  "ClusterStore": "", 
  "InitCommit": {
    "Expected": "949e6fa", 
    "ID": "949e6fa"
  }, 
  "Isolation": "", 
  "SystemStatus": null, 
  "OomKillDisable": true, 
  "ClusterAdvertise": "", 
  "SystemTime": "2017-06-28T17:47:08.367891059-03:00", 
  "Name": "bibliotecas-2", 
  "CPUSet": true, 
  "RegistryConfig": {
    "InsecureRegistryCIDRs": [
      "127.0.0.0/8"
    ], 
    "IndexConfigs": {
      "docker.io": {
        "Official": true, 
        "Name": "docker.io", 
        "Secure": true, 
        "Mirrors": null
      }
    }, 
    "Mirrors": []
  }, 
  "DefaultRuntime": "runc", 
  "ContainersStopped": 31, 
  "NCPU": 4, 
  "NFd": 21, 
  "Architecture": "x86_64", 
  "KernelMemory": true, 
  "CpuCfsQuota": true, 
  "Debug": false, 
  "ID": "WX7D:SXZU:UU4G:INRK:P4EI:BY2F:YZYQ:Z4VM:SEBU:LUPV:T3NK:GVFO", 
  "IPv4Forwarding": true, 
  "KernelVersion": "4.4.0-78-generic", 
  "BridgeNfIptables": true, 
  "NoProxy": "", 
  "LiveRestoreEnabled": false, 
  "ServerVersion": "17.03.1-ce", 
  "CpuCfsPeriod": true, 
  "ExperimentalBuild": false, 
  "MemoryLimit": true, 
  "SwapLimit": false, 
  "Plugins": {
    "Volume": [
      "local"
    ], 
    "Network": [
      "bridge", 
      "host", 
      "macvlan", 
      "null", 
      "overlay"
    ], 
    "Authorization": null
  }, 
  "Images": 347, 
  "DockerRootDir": "/var/lib/docker", 
  "NEventsListener": 0, 
  "CPUShares": true, 
  "RuncCommit": {
    "Expected": "54296cf40ad8143b62dbcaa1d90e520a2136ddfe", 
    "ID": "54296cf40ad8143b62dbcaa1d90e520a2136ddfe"
  }
}
{
  "KernelVersion": "4.4.0-78-generic", 
  "Arch": "amd64", 
  "BuildTime": "2017-03-27T17:14:09.765618756+00:00", 
  "ApiVersion": "1.27", 
  "Version": "17.03.1-ce", 
  "MinAPIVersion": "1.12", 
  "GitCommit": "c6d412e", 
  "Os": "linux", 
  "GoVersion": "go1.7.5"
}
SUMMARY

While migrating a playbook to ansible-container 0.9.1, I had to call my var files manually since there's no more main.yml file to reference them anymore. I used to call them like this in main.yml:

  vars_files:
    - vars/user.yml

Now, following the docs, I'm tyring to run

ansible-container --var-file vars/user.yml build

Running ansible-container with --var-fileseems to fail when yml file has non ascii characters in some cases. I get this: UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 369: ordinal not in range(128)

This is not always the case, but it happened when there were non-ascii characters inside arrays, mixed with template variables. These are some of the smallest test var-files I could come up with I created an empty project with ansible-container init, created a vars.yml file and ran the following tests

This vars.yml file fails

sources:
  - 'ñ'
  - '{{ 1 }}'

This one also fails sources: [ 'ñ', '{{ 1 }}']

This one also fails

sources:
  - 'ñ {{ 1 }}'

But this works

sources:
  - 'n {{ 1 }}'

And this works also

sources:
  - 'ñ'
  - 'test'

This one also works

sources: 'ñ'
alist:
  - '{{ 1 }}'

The complete the error trace:

Parsing conductor CLI args.
Traceback (most recent call last):
  File "/usr/local/bin/conductor", line 11, in <module>
    load_entry_point('ansible-container', 'console_scripts', 'conductor')()
  File "/_ansible/container/__init__.py", line 19, in __wrapped__
    return fn(*args, **kwargs)
  File "/_ansible/container/cli.py", line 369, in conductor_commandline
    conductor_config = AnsibleContainerConductorConfig(list_to_ordereddict(containers_config))
  File "/_ansible/container/__init__.py", line 19, in __wrapped__
    return fn(*args, **kwargs)
  File "/_ansible/container/config.py", line 256, in __init__
    self._process_defaults()
  File "/_ansible/container/config.py", line 289, in _process_defaults
    callback=lambda processed: self._templar.set_available_variables(
  File "/_ansible/container/config.py", line 276, in _process_section
    templar.template(buffer.getvalue())
  File "/usr/local/lib/python2.7/dist-packages/ansible/template/__init__.py", line 431, in template
    variable_hash = sha1(text_type(variable).encode('utf-8'))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 369: ordinal not in range(128)
Conductor terminated. Preserving as requested.  command_rc=1 conductor_id=5253b7600ab5411bf131c7e9ec6d5b01f36e84beb5a2cb5165406716c4df4d3e save_container=True
STEPS TO REPRODUCE
  1. Create a project with one of the mentioned var files
  2. Run ansible-container with the--var-file parameter pointing to the created file 
    mkdir -p /tmp/test-ascii-vars && \
cd /tmp/test-ascii-vars && \
ansible-container init;  \
echo "sources: [ 'ñ', '{{ 1 }}']"  > vars.yml && \
ansible-container --var-file vars.yml build
EXPECTED RESULTS

Ansible-container build process should launch normally

ACTUAL RESULTS
$ ansible-container --var-file vars.yml build
Building Docker Engine context...
Starting Docker build of Ansible Container Conductor image (please be patient)...
Parsing conductor CLI args.
Traceback (most recent call last):
  File "/usr/bin/conductor", line 11, in <module>
    load_entry_point('ansible-container', 'console_scripts', 'conductor')()
  File "/_ansible/container/__init__.py", line 19, in __wrapped__
    return fn(*args, **kwargs)
Conductor terminated. Cleaning up.      command_rc=1 conductor_id=ef8f9f6533be145848f601f152c93a0cbe1d81be4f7c8f78616bf9b04b1bdf6f save_container=False
  File "/_ansible/container/cli.py", line 369, in conductor_commandline
    conductor_config = AnsibleContainerConductorConfig(list_to_ordereddict(containers_config))
  File "/_ansible/container/__init__.py", line 19, in __wrapped__
    return fn(*args, **kwargs)
  File "/_ansible/container/config.py", line 256, in __init__
    self._process_defaults()
  File "/_ansible/container/config.py", line 289, in _process_defaults
    callback=lambda processed: self._templar.set_available_variables(
  File "/_ansible/container/config.py", line 276, in _process_section
    templar.template(buffer.getvalue())
  File "/usr/lib/python2.7/site-packages/ansible/template/__init__.py", line 431, in template
    variable_hash = sha1(text_type(variable).encode('utf-8'))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 2: ordinal not in range(128)
ERROR   Conductor exited with status 1

Debug output:

$ ansible-container --debug --var-file vars.yml build
2017-06-28T18:51:28.820401 Use variable file: /tmp/test-ascii-vars/vars.yml [container.config] caller_file=/home/pmarrone/Proyectos/ansible-container/container/config.py caller_func=_get_variables_from_file caller_line=197 file=/tmp/test-ascii-vars/vars.yml
2017-06-28T18:51:28.821809 The default type is            [container.config] caller_file=/home/pmarrone/Proyectos/ansible-container/container/config.py caller_func=_resolve_defaults caller_line=160 config=<class 'ruamel.yaml.comments.CommentedMap'> defaults=<type '_ordereddict.ordereddict'>
2017-06-28T18:51:28.822382 Getting environment variables... [container.config] caller_file=/home/pmarrone/Proyectos/ansible-container/container/config.py caller_func=_get_environment_variables caller_line=176
2017-06-28T18:51:28.822932 Read environment variables     [container.config] caller_file=/home/pmarrone/Proyectos/ansible-container/container/config.py caller_func=_get_environment_variables caller_line=181 env_vars={}
2017-06-28T18:51:28.823434 Resolved template variables    [container.config] caller_file=/home/pmarrone/Proyectos/ansible-container/container/config.py caller_func=_resolve_defaults caller_line=165 template_vars={"sources": ["\u00f1", "{{ 1 }}"]}
2017-06-28T18:51:28.824629 Parsed config                  [container.config] caller_file=/home/pmarrone/Proyectos/ansible-container/container/config.py caller_func=set_env caller_line=141 config={"version": "2", "settings": {"conductor": {"base": "centos:7"}, "project_name": "test-ascii-vars", "pwd": "/tmp/test-ascii-vars"}, "services": {}, "registries": {}, "defaults": {"sources": ["\u00f1", "{{ 1 }}"]}}
2017-06-28T18:51:28.825082 Loading engine capabilities    [container.utils.loader] caller_file=/home/pmarrone/Proyectos/ansible-container/container/utils/loader.py caller_func=load_engine caller_line=14 capabilities=['BUILD', 'RUN'] engine=docker
2017-06-28T18:51:28.843232 Could not find container for conductor [container.docker.engine] all_containers=[<Container: 004450999c>] caller_file=/home/pmarrone/Proyectos/ansible-container/container/docker/engine.py caller_func=get_container_id_for_service caller_line=426 container=u'test-ascii-vars_conductor'
2017-06-28T18:51:28.846614 Call: Engine.build_conductor_image [container.docker.engine] args=('/tmp/test-ascii-vars', 'centos:7') caller_file=/home/pmarrone/Proyectos/ansible-container/container/docker/engine.py caller_func=Engine.build_conductor_image caller_line=23 kwargs={'cache': True}
2017-06-28T18:51:28.848373 Using temporary directory      [container.utils.temp] caller_file=/home/pmarrone/Proyectos/ansible-container/container/utils/temp.py caller_func=__enter__ caller_line=17 path=/tmp/tmpl6dlJI
2017-06-28T18:51:28.848820 Building Docker Engine context... [container.docker.engine] caller_file=/home/pmarrone/Proyectos/ansible-container/container/docker/engine.py caller_func=build_conductor_image caller_line=706
2017-06-28T18:51:28.871103 Rendered Jinja Template:       [container.utils] body=FROM centos:7
ENV ANSIBLE_CONTAINER=1

RUN yum update -y && \
    yum install -y epel-release && \
    yum install -y make gcc git python-devel curl rsync libffi-devel openssl-devel && \
    yum clean all

COPY /contrib/get-pip.py /get-pip.py
RUN python /get-pip.py && \
    mkdir -p /etc/ansible/roles /_ansible/src && \                              
    curl https://get.docker.com/builds/Linux/x86_64/docker-17.04.0-ce.tgz \     
    | tar -zxC /usr/local/bin/ --strip-components=1                             

# The COPY here will break cache if the version of conductor changed            
COPY /container-src /_ansible/container                                         
# Fix: 'pip install six' is a temporary fix for setuptools==36.0.0 bug          
RUN cd /_ansible && \                                                           
    pip install six && \                                                        
    pip install -r container/conductor-build/conductor-requirements.txt && \    
    PYTHONPATH=. LC_ALL="en_US.UTF-8" python container/conductor-build/setup.py develop -v && \                                                                 
    ansible-galaxy install -p /etc/ansible/roles -r container/conductor-build/conductor-requirements.yml                                                        

# The COPY here will break cache if the requirements or ansible.cfg has changed 
COPY /build-src /_ansible/build                                                 
RUN ( test -f /_ansible/build/ansible-requirements.txt && pip install --no-cache-dir -r /_ansible/build/ansible-requirements.txt || true ) && \                 
    ( test -f /_ansible/build/requirements.yml && ansible-galaxy install -p /etc/ansible/roles -r /_ansible/build/requirements.yml || true ) && \               
    ( test -f /_ansible/build/ansible.cfg && cp /_ansible/build/ansible.cfg /etc/ansible/ansible.cfg || true)                                                   

VOLUME /usr                                                                     

 caller_file=/home/pmarrone/Proyectos/ansible-container/container/utils/__init__.py caller_func=jinja_render_to_temp caller_line=82                             
2017-06-28T18:51:28.871948 Context manifest:              [container.docker.engine] caller_file=/home/pmarrone/Proyectos/ansible-container/container/docker/engine.py caller_func=build_conductor_image caller_line=762                         
2017-06-28T18:51:28.872116 tarball item: build-src/ansible.cfg (144 bytes) [container.docker.engine] bytes=144 file=build-src/ansible.cfg
2017-06-28T18:51:28.872232 tarball item: build-src/ansible-requirements.txt (130 bytes) [container.docker.engine] bytes=130 file=build-src/ansible-requirements.txt                                                                             
2017-06-28T18:51:28.872339 tarball item: build-src/requirements.yml (298 bytes) [container.docker.engine] bytes=298 file=build-src/requirements.yml
2017-06-28T18:51:28.872443 tarball item: build-src/.touch (0 bytes) [container.docker.engine] bytes=0 file=build-src/.touch
2017-06-28T18:51:28.872544 tarball item: contrib/get-pip.py (1595409 bytes) [container.docker.engine] bytes=1595409 file=contrib/get-pip.py
2017-06-28T18:51:28.872644 tarball item: container-src (0 bytes) [container.docker.engine] bytes=0L file=container-src
2017-06-28T18:51:28.872744 tarball item: container-src/engine.py (6544 bytes) [container.docker.engine] bytes=6544 file=container-src/engine.py
2017-06-28T18:51:28.872845 tarball item: container-src/engine.pyc (10643 bytes) [container.docker.engine] bytes=10643 file=container-src/engine.pyc
2017-06-28T18:51:28.872976 tarball item: container-src/utils (0 bytes) [container.docker.engine] bytes=0L file=container-src/utils
2017-06-28T18:51:28.873120 tarball item: container-src/utils/_text.py (12304 bytes) [container.docker.engine] bytes=12304 file=container-src/utils/_text.py
2017-06-28T18:51:28.873250 tarball item: container-src/utils/galaxy.py (8120 bytes) [container.docker.engine] bytes=8120 file=container-src/utils/galaxy.py
2017-06-28T18:51:28.873369 tarball item: container-src/utils/temp.py (698 bytes) [container.docker.engine] bytes=698 file=container-src/utils/temp.py
2017-06-28T18:51:28.873488 tarball item: container-src/utils/_text.pyc (8826 bytes) [container.docker.engine] bytes=8826 file=container-src/utils/_text.pyc
2017-06-28T18:51:28.873588 tarball item: container-src/utils/logmux.pyc (2649 bytes) [container.docker.engine] bytes=2649 file=container-src/utils/logmux.pyc
2017-06-28T18:51:28.873682 tarball item: container-src/utils/logmux.py (1313 bytes) [container.docker.engine] bytes=1313 file=container-src/utils/logmux.py
2017-06-28T18:51:28.873777 tarball item: container-src/utils/temp.pyc (1398 bytes) [container.docker.engine] bytes=1398 file=container-src/utils/temp.pyc
2017-06-28T18:51:28.873871 tarball item: container-src/utils/__init__.py (11013 bytes) [container.docker.engine] bytes=11013 file=container-src/utils/__init__.py                                                                               
2017-06-28T18:51:28.873967 tarball item: container-src/utils/visibility.py (3329 bytes) [container.docker.engine] bytes=3329 file=container-src/utils/visibility.py                                                                             
2017-06-28T18:51:28.874062 tarball item: container-src/utils/__init__.pyc (11643 bytes) [container.docker.engine] bytes=11643 file=container-src/utils/__init__.pyc                                                                             
2017-06-28T18:51:28.874157 tarball item: container-src/utils/loader.py (906 bytes) [container.docker.engine] bytes=906 file=container-src/utils/loader.py
2017-06-28T18:51:28.874250 tarball item: container-src/utils/visibility.pyc (4238 bytes) [container.docker.engine] bytes=4238 file=container-src/utils/visibility.pyc                                                                           
2017-06-28T18:51:28.874345 tarball item: container-src/utils/loader.pyc (1190 bytes) [container.docker.engine] bytes=1190 file=container-src/utils/loader.pyc
2017-06-28T18:51:28.874439 tarball item: container-src/utils/galaxy.pyc (7481 bytes) [container.docker.engine] bytes=7481 file=container-src/utils/galaxy.pyc
2017-06-28T18:51:28.874535 tarball item: container-src/k8s (0 bytes) [container.docker.engine] bytes=0L file=container-src/k8s
2017-06-28T18:51:28.874630 tarball item: container-src/k8s/engine.py (1254 bytes) [container.docker.engine] bytes=1254 file=container-src/k8s/engine.py
2017-06-28T18:51:28.874725 tarball item: container-src/k8s/__init__.py (196 bytes) [container.docker.engine] bytes=196 file=container-src/k8s/__init__.py
2017-06-28T18:51:28.874820 tarball item: container-src/k8s/config.py (422 bytes) [container.docker.engine] bytes=422 file=container-src/k8s/config.py
2017-06-28T18:51:28.874918 tarball item: container-src/k8s/deploy.py (1720 bytes) [container.docker.engine] bytes=1720 file=container-src/k8s/deploy.py
2017-06-28T18:51:28.875013 tarball item: container-src/k8s/base_deploy.py (27717 bytes) [container.docker.engine] bytes=27717 file=container-src/k8s/base_deploy.py                                                                             
2017-06-28T18:51:28.875109 tarball item: container-src/k8s/base_config.py (932 bytes) [container.docker.engine] bytes=932 file=container-src/k8s/base_config.py
2017-06-28T18:51:28.875204 tarball item: container-src/k8s/base_engine.py (9281 bytes) [container.docker.engine] bytes=9281 file=container-src/k8s/base_engine.py                                                                               
2017-06-28T18:51:28.875327 tarball item: container-src/__init__.py (950 bytes) [container.docker.engine] bytes=950 file=container-src/__init__.py
2017-06-28T18:51:28.875448 tarball item: container-src/cli.py (21145 bytes) [container.docker.engine] bytes=21145 file=container-src/cli.py
2017-06-28T18:51:28.875569 tarball item: container-src/cli.pyc (16935 bytes) [container.docker.engine] bytes=16935 file=container-src/cli.pyc
2017-06-28T18:51:28.875681 tarball item: container-src/config.py (14321 bytes) [container.docker.engine] bytes=14321 file=container-src/config.py
2017-06-28T18:51:28.875791 tarball item: container-src/templates (0 bytes) [container.docker.engine] bytes=0L file=container-src/templates
2017-06-28T18:51:28.875904 tarball item: container-src/templates/hosts.j2 (90 bytes) [container.docker.engine] bytes=90 file=container-src/templates/hosts.j2
2017-06-28T18:51:28.876015 tarball item: container-src/templates/builder.sh (620 bytes) [container.docker.engine] bytes=620 file=container-src/templates/builder.sh                                                                             
2017-06-28T18:51:28.876126 tarball item: container-src/templates/restart-docker-compose.j2.yml (75 bytes) [container.docker.engine] bytes=75 file=container-src/templates/restart-docker-compose.j2.yml                                         
2017-06-28T18:51:28.876238 tarball item: container-src/templates/build-docker-compose.j2.yml (1635 bytes) [container.docker.engine] bytes=1635 file=container-src/templates/build-docker-compose.j2.yml                                         
2017-06-28T18:51:28.876349 tarball item: container-src/templates/listhosts-docker-compose.j2.yml (1640 bytes) [container.docker.engine] bytes=1640 file=container-src/templates/listhosts-docker-compose.j2.yml                                 
2017-06-28T18:51:28.876480 tarball item: container-src/templates/compose_versioned.j2.yml (237 bytes) [container.docker.engine] bytes=237 file=container-src/templates/compose_versioned.j2.yml                                                 
2017-06-28T18:51:28.876594 tarball item: container-src/templates/ansible-dockerfile.j2 (1194 bytes) [container.docker.engine] bytes=1194 file=container-src/templates/ansible-dockerfile.j2                                                     
2017-06-28T18:51:28.876716 tarball item: container-src/templates/ansible.cfg (41 bytes) [container.docker.engine] bytes=41 file=container-src/templates/ansible.cfg                                                                             
2017-06-28T18:51:28.876810 tarball item: container-src/templates/role (0 bytes) [container.docker.engine] bytes=0L file=container-src/templates/role
2017-06-28T18:51:28.876904 tarball item: container-src/templates/role/defaults (0 bytes) [container.docker.engine] bytes=0L file=container-src/templates/role/defaults                                                                          
2017-06-28T18:51:28.876997 tarball item: container-src/templates/role/defaults/main.j2.yml (55 bytes) [container.docker.engine] bytes=55 file=container-src/templates/role/defaults/main.j2.yml                                                 
2017-06-28T18:51:28.877091 tarball item: container-src/templates/role/.travis.j2.yml (540 bytes) [container.docker.engine] bytes=540 file=container-src/templates/role/.travis.j2.yml                                                           
2017-06-28T18:51:28.877218 tarball item: container-src/templates/role/README.j2.md (1287 bytes) [container.docker.engine] bytes=1287 file=container-src/templates/role/README.j2.md                                                             
2017-06-28T18:51:28.877314 tarball item: container-src/templates/role/meta (0 bytes) [container.docker.engine] bytes=0L file=container-src/templates/role/meta
2017-06-28T18:51:28.877409 tarball item: container-src/templates/role/meta/main.j2.yml (3220 bytes) [container.docker.engine] bytes=3220 file=container-src/templates/role/meta/main.j2.yml                                                     
2017-06-28T18:51:28.877503 tarball item: container-src/templates/role/test (0 bytes) [container.docker.engine] bytes=0L file=container-src/templates/role/test
2017-06-28T18:51:28.877599 tarball item: container-src/templates/role/test/test.j2.yml (81 bytes) [container.docker.engine] bytes=81 file=container-src/templates/role/test/test.j2.yml                                                         
2017-06-28T18:51:28.877692 tarball item: container-src/templates/wait_on_host.py (2365 bytes) [container.docker.engine] bytes=2365 file=container-src/templates/wait_on_host.py                                                                 
2017-06-28T18:51:28.877787 tarball item: container-src/templates/init (0 bytes) [container.docker.engine] bytes=0L file=container-src/templates/init
2017-06-28T18:51:28.877879 tarball item: container-src/templates/init/requirements.j2.yml (299 bytes) [container.docker.engine] bytes=299 file=container-src/templates/init/requirements.j2.yml                                                 
2017-06-28T18:51:28.877973 tarball item: container-src/templates/init/container.j2.yml (2514 bytes) [container.docker.engine] bytes=2514 file=container-src/templates/init/container.j2.yml                                                     
2017-06-28T18:51:28.878066 tarball item: container-src/templates/init/ansible-requirements.j2.txt (131 bytes) [container.docker.engine] bytes=131 file=container-src/templates/init/ansible-requirements.j2.txt                                 
2017-06-28T18:51:28.878160 tarball item: container-src/templates/init/meta.j2.yml (1231 bytes) [container.docker.engine] bytes=1231 file=container-src/templates/init/meta.j2.yml                                                               
2017-06-28T18:51:28.878255 tarball item: container-src/templates/init/ansible.j2.cfg (145 bytes) [container.docker.engine] bytes=145 file=container-src/templates/init/ansible.j2.cfg                                                           
2017-06-28T18:51:28.878348 tarball item: container-src/templates/install-docker-compose.j2.yml (712 bytes) [container.docker.engine] bytes=712 file=container-src/templates/install-docker-compose.j2.yml                                       
2017-06-28T18:51:28.878442 tarball item: container-src/templates/ansible-container-inventory.py (1230 bytes) [container.docker.engine] bytes=1230 file=container-src/templates/ansible-container-inventory.py                                   
2017-06-28T18:51:28.878535 tarball item: container-src/templates/stop-docker-compose.j2.yml (75 bytes) [container.docker.engine] bytes=75 file=container-src/templates/stop-docker-compose.j2.yml                                               
2017-06-28T18:51:28.878627 tarball item: container-src/templates/run-docker-compose.j2.yml (75 bytes) [container.docker.engine] bytes=75 file=container-src/templates/run-docker-compose.j2.yml                                                 
2017-06-28T18:51:28.878719 tarball item: container-src/openshift (0 bytes) [container.docker.engine] bytes=0L file=container-src/openshift
2017-06-28T18:51:28.878812 tarball item: container-src/openshift/engine.py (2151 bytes) [container.docker.engine] bytes=2151 file=container-src/openshift/engine.py                                                                             
2017-06-28T18:51:28.878904 tarball item: container-src/openshift/__init__.py (196 bytes) [container.docker.engine] bytes=196 file=container-src/openshift/__init__.py                                                                           
2017-06-28T18:51:28.878997 tarball item: container-src/openshift/config.py (427 bytes) [container.docker.engine] bytes=427 file=container-src/openshift/config.py                                                                               
2017-06-28T18:51:28.879090 tarball item: container-src/openshift/deploy.py (6241 bytes) [container.docker.engine] bytes=6241 file=container-src/openshift/deploy.py                                                                             
2017-06-28T18:51:28.879203 tarball item: container-src/__init__.pyc (1229 bytes) [container.docker.engine] bytes=1229 file=container-src/__init__.pyc
2017-06-28T18:51:28.879297 tarball item: container-src/core.pyc (28502 bytes) [container.docker.engine] bytes=28502 file=container-src/core.pyc
2017-06-28T18:51:28.879389 tarball item: container-src/docker (0 bytes) [container.docker.engine] bytes=0L file=container-src/docker
2017-06-28T18:51:28.879483 tarball item: container-src/docker/engine.py (37499 bytes) [container.docker.engine] bytes=37499 file=container-src/docker/engine.py
2017-06-28T18:51:28.879576 tarball item: container-src/docker/engine.pyc (31959 bytes) [container.docker.engine] bytes=31959 file=container-src/docker/engine.pyc                                                                               
2017-06-28T18:51:28.879668 tarball item: container-src/docker/files (0 bytes) [container.docker.engine] bytes=0L file=container-src/docker/files
2017-06-28T18:51:28.879761 tarball item: container-src/docker/files/get-pip.py (1595409 bytes) [container.docker.engine] bytes=1595409 file=container-src/docker/files/get-pip.py                                                               
2017-06-28T18:51:28.879855 tarball item: container-src/docker/__init__.py (205 bytes) [container.docker.engine] bytes=205 file=container-src/docker/__init__.py
2017-06-28T18:51:28.879949 tarball item: container-src/docker/tests (0 bytes) [container.docker.engine] bytes=0L file=container-src/docker/tests
2017-06-28T18:51:28.880041 tarball item: container-src/docker/tests/import (0 bytes) [container.docker.engine] bytes=0L file=container-src/docker/tests/import
2017-06-28T18:51:28.880134 tarball item: container-src/docker/tests/import/Dockerfile (1151 bytes) [container.docker.engine] bytes=1151 file=container-src/docker/tests/import/Dockerfile                                                       
2017-06-28T18:51:28.880228 tarball item: container-src/docker/tests/import/lulz.tgz (59072 bytes) [container.docker.engine] bytes=59072 file=container-src/docker/tests/import/lulz.tgz                                                         
2017-06-28T18:51:28.880323 tarball item: container-src/docker/tests/import/two-fish (0 bytes) [container.docker.engine] bytes=0L file=container-src/docker/tests/import/two-fish                                                                
2017-06-28T18:51:28.880418 tarball item: container-src/docker/tests/import/two-fish/who.txt (7 bytes) [container.docker.engine] bytes=7 file=container-src/docker/tests/import/two-fish/who.txt                                                 
2017-06-28T18:51:28.880513 tarball item: container-src/docker/tests/import/two-fish/green-eggs (0 bytes) [container.docker.engine] bytes=0L file=container-src/docker/tests/import/two-fish/green-eggs                                          
2017-06-28T18:51:28.880607 tarball item: container-src/docker/tests/import/two-fish/green-eggs/ham.txt (5 bytes) [container.docker.engine] bytes=5 file=container-src/docker/tests/import/two-fish/green-eggs/ham.txt                           
2017-06-28T18:51:28.880702 tarball item: container-src/docker/tests/import/red-fish (0 bytes) [container.docker.engine] bytes=0L file=container-src/docker/tests/import/red-fish                                                                
2017-06-28T18:51:28.880797 tarball item: container-src/docker/tests/import/red-fish/lulz.txt (10 bytes) [container.docker.engine] bytes=10 file=container-src/docker/tests/import/red-fish/lulz.txt                                             
2017-06-28T18:51:28.880892 tarball item: container-src/docker/tests/import/one-fish.txt (6 bytes) [container.docker.engine] bytes=6 file=container-src/docker/tests/import/one-fish.txt                                                         
2017-06-28T18:51:28.880986 tarball item: container-src/docker/tests/import/blue-fish.txt (10 bytes) [container.docker.engine] bytes=10 file=container-src/docker/tests/import/blue-fish.txt                                                     
2017-06-28T18:51:28.881080 tarball item: container-src/docker/config.py (1052 bytes) [container.docker.engine] bytes=1052 file=container-src/docker/config.py
2017-06-28T18:51:28.881191 tarball item: container-src/docker/templates (0 bytes) [container.docker.engine] bytes=0L file=container-src/docker/templates
2017-06-28T18:51:28.881289 tarball item: container-src/docker/templates/conductor-dockerfile.j2 (3010 bytes) [container.docker.engine] bytes=3010 file=container-src/docker/templates/conductor-dockerfile.j2                                   
2017-06-28T18:51:28.881384 tarball item: container-src/docker/deploy.py (248 bytes) [container.docker.engine] bytes=248 file=container-src/docker/deploy.py
2017-06-28T18:51:28.881477 tarball item: container-src/docker/__init__.pyc (337 bytes) [container.docker.engine] bytes=337 file=container-src/docker/__init__.pyc                                                                               
2017-06-28T18:51:28.881570 tarball item: container-src/docker/importer.py (24838 bytes) [container.docker.engine] bytes=24838 file=container-src/docker/importer.py                                                                             
2017-06-28T18:51:28.881664 tarball item: container-src/docker/config.pyc (1458 bytes) [container.docker.engine] bytes=1458 file=container-src/docker/config.pyc
2017-06-28T18:51:28.881774 tarball item: container-src/core.py (39627 bytes) [container.docker.engine] bytes=39627 file=container-src/core.py
2017-06-28T18:51:28.881883 tarball item: container-src/config.pyc (13757 bytes) [container.docker.engine] bytes=13757 file=container-src/config.pyc
2017-06-28T18:51:28.881983 tarball item: container-src/exceptions.pyc (5914 bytes) [container.docker.engine] bytes=5914 file=container-src/exceptions.pyc
2017-06-28T18:51:28.882079 tarball item: container-src/exceptions.py (2158 bytes) [container.docker.engine] bytes=2158 file=container-src/exceptions.py
2017-06-28T18:51:28.882179 tarball item: container-src/conductor-build/setup.py (3215 bytes) [container.docker.engine] bytes=3215 file=container-src/conductor-build/setup.py                                                                   
2017-06-28T18:51:28.882274 tarball item: container-src/conductor-build/conductor-requirements.txt (272 bytes) [container.docker.engine] bytes=272 file=container-src/conductor-build/conductor-requirements.txt                                 
2017-06-28T18:51:28.882390 tarball item: container-src/conductor-build/conductor-requirements.yml (60 bytes) [container.docker.engine] bytes=60 file=container-src/conductor-build/conductor-requirements.yml                                   
2017-06-28T18:51:28.882513 tarball item: Dockerfile (1486 bytes) [container.docker.engine] bytes=1486 file=Dockerfile
2017-06-28T18:51:28.883837 Starting Docker build of Ansible Container Conductor image (please be patient)... [container.docker.engine] caller_file=/home/pmarrone/Proyectos/ansible-container/container/docker/engine.py caller_func=build_conductor_image caller_line=770                                                      
Step 1/10 : FROM centos:7
 ---> 8140d0c64310
Step 2/10 : ENV ANSIBLE_CONTAINER 1
 ---> Using cache
 ---> ca543c10b674
Step 3/10 : RUN yum update -y &&     yum install -y epel-release &&     yum install -y make gcc git python-devel curl rsync libffi-devel openssl-devel &&     yum clean all
 ---> Using cache
 ---> ff6a750dbbc2
Step 4/10 : COPY /contrib/get-pip.py /get-pip.py
 ---> Using cache
 ---> 0a2b0af60efe
Step 5/10 : RUN python /get-pip.py &&     mkdir -p /etc/ansible/roles /_ansible/src &&     curl https://get.docker.com/builds/Linux/x86_64/docker-17.04.0-ce.tgz     | tar -zxC /usr/local/bin/ --strip-components=1
 ---> Using cache
 ---> 6a55b5174c0c
Step 6/10 : COPY /container-src /_ansible/container
 ---> Using cache
 ---> 40810df871b4
Step 7/10 : RUN cd /_ansible &&     pip install six &&     pip install -r container/conductor-build/conductor-requirements.txt &&     PYTHONPATH=. LC_ALL="en_US.UTF-8" python container/conductor-build/setup.py develop -v &&     ansible-galaxy install -p /etc/ansible/roles -r container/conductor-build/conductor-requirements.yml
 ---> Using cache
 ---> 6c6f440dc984
Step 8/10 : COPY /build-src /_ansible/build
 ---> Using cache
 ---> fa18cdcd8c27
Step 9/10 : RUN ( test -f /_ansible/build/ansible-requirements.txt && pip install --no-cache-dir -r /_ansible/build/ansible-requirements.txt || true ) &&     ( test -f /_ansible/build/requirements.yml && ansible-galaxy install -p /etc/ansible/roles -r /_ansible/build/requirements.yml || true ) &&     ( test -f /_ansible/build/ansible.cfg && cp /_ansible/build/ansible.cfg /etc/ansible/ansible.cfg || true)
 ---> Using cache
 ---> 6ab9f014a114
Step 10/10 : VOLUME /usr
 ---> Using cache
 ---> dc1c68f9accf
Successfully built dc1c68f9accf
2017-06-28T18:51:28.996701 Cleaning up temporary directory [container.utils.temp] caller_file=/home/pmarrone/Proyectos/ansible-container/container/utils/temp.py caller_func=__exit__ caller_line=22 path=/tmp/tmpl6dlJI
2017-06-28T18:51:28.997795 Config settings                [container.core] caller_file=/home/pmarrone/Proyectos/ansible-container/container/core.py caller_func=hostcmd_build caller_line=156 conf=<class 'container.docker.config.AnsibleContainerConfig'> config=<container.docker.config.AnsibleContainerConfig object at 0x7f521701a210> rawsettings={"conductor": {"base": "centos:7"}, "project_name": "test-ascii-vars", "pwd": "/tmp/test-ascii-vars"} settings={"conductor": {"base": "centos:7"}, "project_name": "test-ascii-vars", "pwd": "/tmp/test-ascii-vars"}   
2017-06-28T18:51:28.998433 Call: Engine.run_conductor     [container.docker.engine] args=('build', {'services': ordereddict([]), 'version': '2', 'registries': ordereddict([]), 'defaults': ordereddict([('sources', [u'\xf1', '{{ 1 }}'])]), 'settings': ordereddict([('conductor', ordereddict([('base', 'centos:7')])), ('project_name', 'test-ascii-vars'), ('pwd', '/tmp/test-ascii-vars')])}, '/tmp/test-ascii-vars', {'with_volumes': [], 'cache': True, 'local_python': False, 'with_variables': [], 'selinux': True, 'purge_last': True, 'subcommand': 'build', 'devel': False, 'config_vars': ordereddict([('sources', [u'\xf1', '{{ 1 }}'])]), 'roles_path': None, 'conductor_cache': True, 'ansible_options': u'', 'flatten': False, 'container_cache': True, 'debug': True, 'services_to_build': None, 'save_conductor_container': False}) caller_file=/home/pmarrone/Proyectos/ansible-container/container/docker/engine.py caller_func=Engine.run_conductor caller_line=23 kwargs={}
2017-06-28T18:51:29.000387 Docker run:                    [container.docker.engine] caller_file=/home/pmarrone/Proyectos/ansible-container/container/docker/engine.py caller_func=run_conductor caller_line=330 image=u'sha256:dc1c68f9accf9bcfaba151f8f5d020948dddacb149dda90e5ee7022315e47680' params={'name': u'test-ascii-vars_conductor', 'cap_add': ['SYS_ADMIN'], 'environment': {'DOCKER_HOST': 'unix:///var/run/docker.sock', 'ANSIBLE_ROLES_PATH': '/src/roles:/etc/ansible/roles'}, 'working_dir': '/src', 'command': ['conductor', 'build', '--project-name', 'test-ascii-vars', '--engine', 'docker', '--params', u'eyJ3aXRoX3ZvbHVtZXMiOiBbXSwgImNhY2hlIjogdHJ1ZSwgImxvY2FsX3B5dGhvbiI6IGZhbHNlLCAid2l0aF92YXJpYWJsZXMiOiBbXSwgInNlbGludXgiOiB0cnVlLCAicHVyZ2VfbGFzdCI6IHRydWUsICJzdWJjb21tYW5kIjogImJ1aWxkIiwgImRldmVsIjogZmFsc2UsICJjb25maWdfdmFycyI6IHsic291cmNlcyI6IFsiXHUwMGYxIiwgInt7IDEgfX0iXX0sICJyb2xlc19wYXRoIjogbnVsbCwgImNvbmR1Y3Rvcl9jYWNoZSI6IHRydWUsICJhbnNpYmxlX29wdGlvbnMiOiAiIiwgImZsYXR0ZW4iOiBmYWxzZSwgImNvbnRhaW5lcl9jYWNoZSI6IHRydWUsICJkZWJ1ZyI6IHRydWUsICJzZXJ2aWNlc190b19idWlsZCI6IG51bGwsICJzYXZlX2NvbmR1Y3Rvcl9jb250YWluZXIiOiBmYWxzZX0=', '--config', u'eyJzZXJ2aWNlcyI6IFtdLCAic2V0dGluZ3MiOiBbWyJjb25kdWN0b3IiLCB7ImJhc2UiOiAiY2VudG9zOjcifV0sIFsicHJvamVjdF9uYW1lIiwgInRlc3QtYXNjaWktdmFycyJdLCBbInB3ZCIsICIvdG1wL3Rlc3QtYXNjaWktdmFycyJdXSwgInZlcnNpb24iOiAiMiIsICJkZWZhdWx0cyI6IFtbInNvdXJjZXMiLCBbIlx1MDBmMSIsICJ7eyAxIH19Il1dXSwgInJlZ2lzdHJpZXMiOiBbXX0=', '--encoding', 'b64json'], 'user': 'root', 'volumes': {'/var/run/docker.sock': {'bind': '/var/run/docker.sock', 'mode': 'rw'}, '/tmp/test-ascii-vars': {'bind': '/src', 'mode': 'ro'}}, 'detach': True, 'privileged': True}
Parsing conductor CLI args.
2017-06-28T21:51:34.325996 Processing defaults section... [container.config] caller_file=/_ansible/container/config.py caller_func=_process_defaults caller_line=286
Traceback (most recent call last):
  File "/usr/bin/conductor", line 11, in <module>
    load_entry_point('ansible-container', 'console_scripts', 'conductor')()
  File "/_ansible/container/__init__.py", line 19, in __wrapped__
    return fn(*args, **kwargs)
  File "/_ansible/container/cli.py", line 369, in conductor_commandline
    conductor_config = AnsibleContainerConductorConfig(list_to_ordereddict(containers_config))
  File "/_ansible/container/__init__.py", line 19, in __wrapped__
    return fn(*args, **kwargs)
  File "/_ansible/container/config.py", line 256, in __init__
    self._process_defaults()
  File "/_ansible/container/config.py", line 289, in _process_defaults
    callback=lambda processed: self._templar.set_available_variables(
  File "/_ansible/container/config.py", line 276, in _process_section
    templar.template(buffer.getvalue())
2017-06-28T18:51:35.595237 Conductor terminated. Cleaning up. [container.docker.engine] caller_file=/home/pmarrone/Proyectos/ansible-container/container/docker/engine.py caller_func=await_conductor_command caller_line=357 command_rc=1 conductor_id=u'17ca5d0f7600b201871a8dd5d07dd27266d766c71734db4e4b6b67a35731ec1b' save_container=False
  File "/usr/lib/python2.7/site-packages/ansible/template/__init__.py", line 431, in template
    variable_hash = sha1(text_type(variable).encode('utf-8'))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 2: ordinal not in range(128)
2017-06-28T18:51:35.960543 Conductor exited with status 1 [container.cli] caller_file=/home/pmarrone/Proyectos/ansible-container/container/cli.py caller_func=__call__ caller_line=290
chouseknecht commented 7 years ago

@pmarrone

Thanks for using Ansible Container, and for taking the time to open an issue and share your feedback.

Agree that this looks like a bug. Will dig into it.

In the meantime, a possible workaround might be to use include_vars within your role. Check out the Ansible docs here.

j00bar commented 7 years ago

Might be related to this change: 497b7794

chouseknecht commented 7 years ago

@pmarrone

Did some testing on this. I think the issue is the placement of template variables within the variable file. We don't perform interpolation on the variable file itself, and so Ansible Container's template resolution is not quite the same as that of Ansible Playbook.

It also seems that passing unicode literals as variable values works. Using unicode values does not seem to be an issue for Ansible Container, nor Ansible Playbook.

Closing for now. If you disagree, please comment, and we'll re-open.

Thanks!