epic: no python in the target image

TomasTomecek commented 5 years ago

There is a demand to be build container images with bender without the need of python in the base image.

There are multiple ways of doing this:

ansible-container mounts the interpreter inside the target environment (prone to errors, kinda hacky)
Barak Korren suggested to use the raw module to install python beforehand and uninstall it once the build is done (waste of resources, had to be only a single layer)
support builds from scratch (buildah from scratch) and then utilize the chroot connection plugin or a similar workflow, where we would use python on the host and populate a certain path (might require changes in playbooks, dunno if this is possible now)

Edit: adding more details

For the from scratch part, we should likely start with a research how feasible it is.

If we want to utilize chroot connection, we need python interpreter inside the container. The interpreter can be bind-mounted from the host system, but that's pretty nasty.

If we want to do local connection, the playbook needs to look in a certain way: e.g. we want to do dnf install --instalroot /path/to/the/container where ansible-bender would likely provided the /path/to/the/container variable. Then we might even need a second playbook which would be executed inside the just-provisioned environment (using connection buildah or local). Now that I just wrote it, I kinda like the approach. We could keep the current interface and just add a new option which would be a path to playbook to do the from scratch work. Example:

$ ansible-bender build --init-playbook init.yaml -- playbook.yaml my-fancy-image

With this command, ab would ran the init.yaml playbook using connection local and provisioned the scratch container:

$ cat init.yaml
--
- hosts: all
  tasks:
  - command: dnf install --installroot {{ container_root }} bash glibc

And then we would continue as we do now: we would use the playbook.yaml and using the buildah connection we would run against the {{ container_root }}. Hm, but that still requires python inside the container.

The easiest short-term solution I can think of is to provide a variable with the scratch mount and let people structure their playbooks around it. So instead of doing:

- hosts: all
  tasks:
  - dnf
      name:
      - nginx
      state: presemt
  - copy
      src: my-fancy-page.html
      dest: /var/www/html

do

- hosts: all
  tasks:
  - dnf
      name:
      - nginx
      state: presemt
      installroot: '{{ container_root }}'
  - copy
      src: my-fancy-page.html
      dest: '{{ container_root }}/var/www/html'

and bender would provide execute the playbook like this:

$ ansible-playbook -c local -e container_root=/path/to/container playbook.yaml

scratch is not a real image, it's a "keyword" for buildah:

$ buildah inspect scratch                                                                                                                                        
error reading build object "scratch": error reading image: error locating image "scratch" for importing settings: error locating image with name "scratch": image not known
ERRO[0000] exit status 1                                

$ buildah from scratch
working-container

TomasTomecek commented 5 years ago

@ipbabble hi, I opened this issue to track the work on the from scratch build method, please let me know what are your thoughts. We may also have a call and discuss if you want.

ipbabble commented 5 years ago

@TomasTomecek I'd like to review the ansible-bender work and see where we can make modifications to allow for from scratch. Yes a call would be useful in helping me zero in on the appropriate code.

jordemort commented 5 years ago

Stumbled across this - I have this sort-of-working, insofar as using ansible-bender with base images that don't have Python. I'm using the first approach, of mounting a Python interpreter into the container (into a nonstandard path that nothing else is using) with --build-volumes and then pointing at it via --python-interpreter. I'm producing the Python interpreter using a Dockerfile similar to this one - essentially I'm using pyenv to build a Python in a non-standard location, and then copying all the libraries to that location as well, and then using patchelf to cause Python and its extension modules to seek their libraries in the non-standard location. Prone to errors, kind of hacky, indeed. But it seems to work!

I'm using this with a base image that I create by using podman import on a tarball I built with debootstrap - it'd be pretty convenient if I could skip the podman import, do base_image: scratch, and use Ansible to copy the tarball in as well, but currently this seems to choke when ansible-bender attempts a buildah pull scratch. I plan to try out a patch to skip the buildah pull if the image is scratch and see if that is enough to make it work.

jordemort commented 5 years ago

Update: tried hacking the buildah builder to always aver that scratch exists but then the sanity check for starting the base_image fails, and if I hack that out it fails further down the line. I might dig into this more later but I'm going to stick with starting by importing a tarball with podman for my base_image for now.

TomasTomecek commented 5 years ago

@jordemort thanks for the update, that's really interesting! I agree that we should make bender work with the scratch image - people should still be able to run raw commands; let me create a new issue for that and reference your super-helpful comments.

Edit: doesn't make sense to create a new one, my original post has it all.

@jordemort thank you for your effort! Please let me know if there is anything I can help with.

jdoss commented 5 years ago

Having a {{ container_root }} var to consume from ab would make converting my bash that I use now:

CONTAINER=$(buildah from scratch)

MOUNTPOINT=$(buildah mount ${CONTAINER})

dnf install --disablerepo=* --enablerepo=fedora,updates \
            --installroot ${MOUNTPOINT} \
            --releasever ${FEDORA_VERSION} \
              glibc-minimal-langpack bash coreutils procps-ng vi \
            --nodocs \
            --setopt install_weak_deps=false -y

into Ansible much easier. I would rather not have to use a tarball to create scratch containers when DNF can install the packages I need on the fly.

miquecg commented 5 years ago

# ansible-bender --debug build --extra-ansible-args='-e foo=bar' ./simple-playbook.yaml
20:57:11.321 db.py             DEBUG  search for runtime dir
20:57:11.321 db.py             DEBUG  trying ~/.cache
20:57:11.321 db.py             DEBUG  runtime dir is /root/.cache
20:57:11.322 core.py           DEBUG  this system is not using selinux, /sys/fs/selinux/enforce is not present
20:57:11.327 core.py           DEBUG  ab vars key = ab_vars_20190718205711327579
20:57:11.328 utils.py          INFO   running command: "['ansible-playbook', '--version']"
20:57:11.328 utils.py          DEBUG  ansible-playbook --version
20:57:11.727 utils.py          DEBUG  ansible-playbook 2.8.2
20:57:11.728 utils.py          DEBUG    config file = /etc/ansible/ansible.cfg
20:57:11.728 utils.py          DEBUG    configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
20:57:11.728 utils.py          DEBUG    ansible python module location = /usr/lib/python3.7/site-packages/ansible
20:57:11.728 utils.py          DEBUG    executable location = /usr/bin/ansible-playbook
20:57:11.728 utils.py          DEBUG    python version = 3.7.3 (default, Jun 24 2019, 04:54:02) [GCC 9.1.0]
20:57:11.771 utils.py          DEBUG  it seems that ansible-playbook is not using python 2
20:57:11.771 core.py           DEBUG  ansible-playbook -c local -i /tmp/absoijujb6/i -e ansible_python_interpreter=/usr/bin/python ./.simple-playbook-20190718205711327579-tycexensav.yaml
20:57:11.771 utils.py          INFO   running command: "['ansible-playbook', '-c', 'local', '-i', '/tmp/absoijujb6/i', '-e', 'ansible_python_interpreter=/usr/bin/python', './.simple-playbook-20190718205711327579-tycexensav.yaml']"

ansible-playbook -c local -i /tmp/absoijujb6/i -e ansible_python_interpreter=/usr/bin/python ./.simple-playbook-20190718205711327579-tycexensav.yaml

I was expecting to see in the output the environment var passed in through --extra-ansible-args.

@TomasTomecek I'm asking this to see if it's currently possible what you suggest as an easy solution to build containers from scratch.

and bender would provide execute the playbook like this:

$ ansible-playbook -c local -e container_root=/path/to/container playbook.yaml

TomasTomecek commented 5 years ago

I was expecting to see in the output the environment var passed in through --extra-ansible-args.

That sounds like a bug, could you please open a new issue for that?

@TomasTomecek I'm asking this to see if it's currently possible what you suggest as an easy solution to build containers from scratch.

and bender would provide execute the playbook like this: $ ansible-playbook -c local -e container_root=/path/to/container playbook.yaml

That's really just an idea, honestly. You'd need to prepare the $container_root yourself, and then probably use the chroot connection to execute the playbook, and finally - perform all the buildah commands to get the final container image. This would deserve a nice blog post.

FFY00 commented 4 years ago

Meanwhile, would it be possible to provide an option to run a startup command when we start the container? This way I am able to install python.

TomasTomecek commented 4 years ago

Meanwhile, would it be possible to provide an option to run a startup command when we start the container? This way I am able to install python.

Could you please describe the use case in more detail in a separate issue? I recall that doing such a thing in raw should work.

micheelengronne commented 3 years ago

I am very interested by this. I can't use the layer mechanism right now as I raw install/uninstall python. My builds take 5 minutes to run and it is too long for my Twitch viewers :)

Trashmee commented 3 years ago

Hi,

I'd like to understand potential workarounds, I didn't fully understand the proposals above.

The only thing that works for me (up to now) is to use a simple Dockerfile to build a local image with python3 on top - and use that as baseline for ansible-bender.

That's somehow a bit ugly?

Something like --init-playbook or raw pretask sound cleaner (besides they still would install python3 into the container, but hey..)

# cat Dockerfile
FROM docker.io/gitlab/gitlab-ce:latest
RUN apt-get -y update && apt-get install -y python3

Hopeing there is cleaner, managable approach that i oversaw.

Thanks!

amilcarlucas commented 2 years ago

@Trashmee how exactly are you feeding that locally created docker into ansible-bender? Can you share the command line?

ansible-community / ansible-bender

epic: no python in the target image #49