ansible-container unable to use docker local registry image to build under "settings: conductor: base: ubuntulocal:16.04"

[infojg9]

Dear ansible-container team,

Despite successfully imported docker image into docker local registry and succeeded to perform local pull from local registry, unable to use "conductor: base: ubuntulocal:16.04" docker local registry image inside container/container.yml of Ansible Container, version 0.9.2. Tried with and without sudo access without any positive result, error remains same in both the cases. When we reverted back 'base:' and 'from:' to ubuntu:16.04, everything works as expected.

Any tips/comments would be highly appreciated, looking forward for your kind expert's advice.

Thanks in advance ! Kind Regards, infojg9

====

SUMMARY

Expectation is to build the ansible-container capability to consume docker local registry image for ubuntu:16.04 official image and while we are trying to build it , its failing with the error(with and without root) as inline:

====

STEPS TO REPRODUCE

1. Ensure docker local registry is running correctly, for example, we have hosted the local registry under "localhost:5000/ubuntu" image:

$ docker images REPOSITORY TAG IMAGE ID CREATED SIZE ubuntu 16.04 f975c5035748 6 days ago 112MB ubuntulocal 16.04 f975c5035748 6 days ago 112MB localhost:5000/ubuntu 16.04 f975c5035748 6 days ago 112MB

2. Verify docker local registry pull is working:

$ docker pull localhost:5000/ubuntu:16.04 16.04: Pulling from ubuntu Digest: sha256:52286464db54577a128fa1b1aa3c115bd86721b490ff4cbd0cd14d190b66c570 Status: Image is up to date for localhost:5000/ubuntu:16.04

3. Run the inline container/container.yml

   $ ansible-container --debug build

Solution under investigation:

<...snip...>

container/container.yml: version: "2"

settings: conductor: base: localhost:5000/ubuntu # not working

base: ubuntulocal:16.04 # not working

#base: ubuntu:16.04 # official internet repo, works as expected

project_name: proj_name

services: proj_name: from: localhost:5000/ubuntu # not working

from: ubuntulocal:16.04 # not working

#from: ubuntu:16.04  # official internet repo, works as expected

roles:
  - role: role_test

command: ["/bin/bash"]

<...snip...>

====

====

EXPECTED RESULTS

Ansible container image should be build using local docker registry (either localhost:5000/ubuntu:16.04 or ubuntulocal:16.04) successfully. However only docker official repository is successfully processed by ansible-container (ubuntu:16.04) and docker local registry image is not accepted by teh current ansible-container.

====

ACTUAL RESULTS

Error:

    "stderr_lines": [
        "Building Docker Engine context...\t", 
        "Starting Docker build of Ansible Container Conductor image (please be patient)...\t", 
        "ERROR\tUnknown exception\t", 
        "Traceback (most recent call last):", 
        "  File \"/usr/local/lib/python2.7/dist-packages/container/cli.py\", line 299, in __call__", 
        "    getattr(core, u'hostcmd_{}'.format(args.subcommand))(**vars(args))", 
        "  File \"/usr/local/lib/python2.7/dist-packages/container/__init__.py\", line 28, in __wrapped__", 
        "    return fn(*args, **kwargs)", 
        "  File \"/usr/local/lib/python2.7/dist-packages/container/core.py\", line 181, in hostcmd_build", 
        "    environment=env_vars", 
        "  File \"/usr/local/lib/python2.7/dist-packages/container/docker/engine.py\", line 105, in __wrapped__", 
        "    return fn(self, *args, **kwargs)", 
        "  File \"/usr/local/lib/python2.7/dist-packages/container/__init__.py\", line 28, in __wrapped__", 
        "    return fn(*args, **kwargs)", 
        "  File \"/usr/local/lib/python2.7/dist-packages/container/docker/engine.py\", line 1069, in build_conductor_image", 
        "    nocache=not cache)", 
        "  File \"/usr/local/lib/python2.7/dist-packages/docker/models/images.py\", line 179, in build", 
        "    raise BuildError(chunk['error'])", 
        "BuildError: pull access denied for container-conductor-ubuntulocal-16.04, repository does not exist or may require 'docker login'"
    ], 
    "stdout": "", 
    "stdout_lines": []

====

Configuration data:

OS: Ubutnu 16.04.3 LTS amd64 official
Ansible: ansible-playbook 2.4.3.0
Ansible Container, version 0.9.2
Python 2.7.12

$ docker images
REPOSITORY                                  TAG                 IMAGE ID            CREATED             SIZE
ubuntu                                      16.04               f975c5035748        6 days ago          112MB
ubuntulocal                                 16.04               f975c5035748        6 days ago          112MB
localhost:5000/ubuntu                       16.04               f975c5035748        6 days ago          112MB
registry                                    2                   d1fd7d86a825        2 months ago        33.3MB
ansible/container-conductor-ubuntu-xenial   0.9.2               35cf9762959e        5 months ago        640MB

$ docker pull localhost:5000/ubuntu:16.04
16.04: Pulling from ubuntu
Digest: sha256:52286464db54577a128fa1b1aa3c115bd86721b490ff4cbd0cd14d190b66c570
Status: Image is up to date for localhost:5000/ubuntu:16.04

$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                      NAMES
eee7d299b88f        registry:2          "/entrypoint.sh /etc…"   24 minutes ago      Up 24 minutes       127.0.0.1:5000->5000/tcp   local_registry

$ ansible-container --debug version
Ansible Container, version 0.9.2
Linux, user-u16amd64, 4.13.0-32-generic, #35~16.04.1-Ubuntu SMP Thu Jan 25 10:13:43 UTC 2018, x86_64
2.7.12 (default, Dec  4 2017, 14:50:18) 
[GCC 5.4.0 20160609] /usr/bin/python

$ ansible-playbook --version
ansible-playbook 2.4.3.0
  config file = /home/user/.ansible.cfg
  configured module search path = [u'/home/user/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python2.7/dist-packages/ansible
  executable location = /usr/local/bin/ansible-playbook
  python version = 2.7.12 (default, Dec  4 2017, 14:50:18) [GCC 5.4.0 20160609]

$ python --version
Python 2.7.12

====

[freehck]

Disclaimer: I've just started to learn what ansible-container is.

@infojg9 according to documentation [1] there's a convention about conductor container names. They must be in form "container-conductor-$DISTRO-$TAG". So you needed to make a copy of container-conductor-ubuntu-xenial, not just ubuntu:16.04.

Common ubuntu image is not a suitable conductor, because ansible needs to have some additional software installed in the container in order to provision it. Python for example.

[1] http://docs.ansible.com/ansible-container/conductor.html

[Voronenko]

@infojg9 Is your question still actual? If question is resolved do you have any objections for that issue to be closed?

[infojg9]

Clarified the doubts, kindly close it, anyway next najor release (0.9.3 or 1.0 ?) and commits rates also slowed down: https://github.com/ansible/ansible-container/commit/2fa778a7c8d1699672314ac0b89c53554f435cb7.

[adpoe]

Hi all, I'm seeing this same behavior with what looks like a valid-named local container. I don't mind opening a new issue, but mentioning here, for context -- in case it's simpler to track.

On my local machine, I have both:

$ docker images
REPOSITORY                                  TAG                 IMAGE ID            CREATED             SIZE
ansible/container-conductor-ubuntu-bionic   0.9.3rc0            eb33b5aac7ee        21 hours ago        640MB
container-conductor-ubuntu-bionic           latest              eb33b5aac7ee        21 hours ago        640MB

In my container.yml, I'm trying to run:

version: "2"
settings:
  conductor:
    base: ubuntu:bionic 
  project_name: hello-world
            [.... etc ....]

Running the build step yields:

$ ansible-container build
Building Docker Engine context...   
Starting Docker build of Ansible Container Conductor image (please be patient)...   
ERROR   Unknown exception   
Traceback (most recent call last):
  File "/Users/tony/.pyenv/versions/3.5.2/lib/python3.5/site-packages/container/cli.py", line 299, in __call__
    getattr(core, u'hostcmd_{}'.format(args.subcommand))(**vars(args))
  File "/Users/tony/.pyenv/versions/3.5.2/lib/python3.5/site-packages/container/__init__.py", line 28, in __wrapped__
    return fn(*args, **kwargs)
  File "/Users/tony/.pyenv/versions/3.5.2/lib/python3.5/site-packages/container/core.py", line 181, in hostcmd_build
    environment=env_vars
  File "/Users/tony/.pyenv/versions/3.5.2/lib/python3.5/site-packages/container/docker/engine.py", line 105, in __wrapped__
    return fn(self, *args, **kwargs)
  File "/Users/tony/.pyenv/versions/3.5.2/lib/python3.5/site-packages/container/__init__.py", line 28, in __wrapped__
    return fn(*args, **kwargs)
  File "/Users/tony/.pyenv/versions/3.5.2/lib/python3.5/site-packages/container/docker/engine.py", line 1069, in build_conductor_image
    nocache=not cache)
  File "/Users/tony/.pyenv/versions/3.5.2/lib/python3.5/site-packages/docker/models/images.py", line 179, in build
    raise BuildError(chunk['error'])
docker.errors.BuildError: pull access denied for container-conductor-ubuntu-bionic, repository does not exist or may require 'docker login'

I can provide further details if you are interested, but I don't think this is fixed, based on what I'm seeing.

Version info

$ ansible-container --debug version
Ansible Container, version 0.9.2
Darwin, Tonys-MacBook-Pro.local, 17.5.0, Darwin Kernel Version 17.5.0: Mon Mar  5 22:24:32 PST 2018; root:xnu-4570.51.1~1/RELEASE_X86_64, x86_64
3.5.2 (default, Sep 21 2018, 01:10:39) 
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] /Users/tony/.pyenv/versions/3.5.2/bin/python

$ ansible-playbook --version 
ansible-playbook 2.6.4

If I'm not following proper protocol for reporting, let me know, I don't mind doing it the preferred way.

[Voronenko]

I think, 0.9.2 does not have container for bionic published. However, you can build conductor from source by running python bakery.py

Does changing base to xenial allow you to go further ? To exclude situation with local docker configuration itself?

[adpoe]

Yes, I can use the official containers, besides bionic.

I was just doing the tutorial from:

• https://docs.ansible.com/ansible-container/getting_started.html#hello-world

And this works for other versions, on my setup.

What's interesting is the point about bionic not being published on 0.9.2. Is there another version I could use, with access to bionic?

If I can avoid rolling my own solution, I'm happy to do that. But still would like to get to the bottom of this particular problem, if I can help.

[Voronenko]

So until we reach @j00bar or any other person with both access to docker hub organization for ansible and pypi - we are quite blocked.

For internal purposes I am supporting unofficial build https://pypi.org/project/sa-ansible-container/ but I think, I haven't pushed prebaked conductors yet.

https://github.com/ansible/ansible-container/pull/966

At least, that unblocks organizations

[adpoe]

Thank you, that makes sense.

I'll try your 0.9.3rc2 build and see how it goes, in the meantime.

I see more clearly what the tag signifies now, at:

REPOSITORY                                  TAG                 IMAGE ID            CREATED             SIZE
ansible/container-conductor-ubuntu-bionic   0.9.3rc0            eb33b5aac7ee        21 hours ago        640MB

It sounds like this means bionic is first supported with the 0.9.3 release. Which is great to know.

I appreciate the quick response, too.

[adpoe]

Docker image tag must == ansible-container version

It looks like we can get this working by changing the docker image's tag to match the exact version of ansible-container being used.

I downloaded your most recent release, 0.9.3r2, and got the following results.

$ ansible-container build
Building Docker Engine context...   
Starting Docker build of Ansible Container Conductor image (please be patient)...   
ERROR   Unknown exception   
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/container/cli.py", line 302, in __call__
    getattr(core, u'hostcmd_{}'.format(args.subcommand))(**vars(args))
  File "/usr/local/lib/python2.7/site-packages/container/__init__.py", line 28, in __wrapped__
    return fn(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/container/core.py", line 187, in hostcmd_build
    conductor_provider=config_conductor_provider
  File "/usr/local/lib/python2.7/site-packages/container/docker/engine.py", line 107, in __wrapped__
    return fn(self, *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/container/__init__.py", line 28, in __wrapped__
    return fn(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/container/docker/engine.py", line 1150, in build_conductor_image
    nocache=not cache)
  File "/usr/local/lib/python2.7/site-packages/docker/models/images.py", line 179, in build
    raise BuildError(chunk['error'])
BuildError: manifest for ansible/container-conductor-ubuntu-xenial:0.9.3rc2 not found

Even though I had the following docker images:

$ docker images
ansible/container-conductor-ubuntu-bionic   0.9.3rc0            eb33b5aac7ee        22 hours ago        640MB
container-conductor-ubuntu-bionic           latest              eb33b5aac7ee        22 hours ago        640MB

   ....

Then, I noticed that I didn't have any tags matching 0.9.3rc2, the version I'm running.

So I did: $ docker tag eb33b5aac7ee container-conductor-ubuntu-bionic:0.9.3rc2

And now, it works.

$ ansible-container build
Building Docker Engine context...   
Starting Docker build of Ansible Container Conductor image (please be patient)...   
Parsing conductor CLI args.
Copying build context into Conductor container.
Docker™ daemon integration engine loaded. Build starting.   project=hello-world
Building service... project=hello-world service=flask
Fingerprint for this layer: 77dace2ce02812d995e1c299d59eb96fce8dd2fafedcb7a28b3cef37f9b79ad4    parent_fingerprint=cfbbb1d4ada1309d1bf455f33fbed82f07b0b9b6369083e2bf3a2e8d9a9847fe parent_image_id=sha256:5182e96772bf11f4b912658e265dfe0db8bd314475443b6434ea708784192892 role=flask service=flask
Cached layer for for role flask not found or invalid.   cur_image_id=sha256:5182e96772bf11f4b912658e265dfe0db8bd314475443b6434ea708784192892 fingerprint=77dace2ce02812d995e1c299d59eb96fce8dd2fafedcb7a28b3cef37f9b79ad4 service=flask
Could not locate intermediate build container to reapply role flask. Applying role on image sha256:5182e96772bf11f4b912658e265dfe0db8bd314475443b6434ea708784192892 as container hello-world_flask-cfbbb1d4-flask.  cur_image_fingerprint=cfbbb1d4ada1309d1bf455f33fbed82f07b0b9b6369083e2bf3a2e8d9a9847fe service=flask

PLAY [flask] *******************************************************************

TASK [Gathering Facts] *********************************************************
ok: [flask]

TASK [flask : Install dumb init] ***********************************************
changed: [flask]

    [.... etc ....]

I'm not sure if that's the exact intended behavior, or not. But that looks like the source of the problem I was having.

[Voronenko]

PR moved versioning slightly forward, yep. But glad if it has unblocked you