Open nodje opened 11 months ago
In most cases it should be comparable or faster than 1.x, so I'm curious as well... I'd guess something's going on with your build cache or package management, because we actually reduced several redundant package installation steps from 1.x. Can you share your EE definition?
Keep in mind that processing of ARG
and RUN
commands is performed by the container system (docker or podman), and not builder itself. Builder is mostly a front end to building the instruction file that will be fed to either the podman build
or docker build
command.
Yeah, if even basic no-op things like ARG
have a noticeable delay, I'd suspect there's a problem with your container runtime's storage driver (common issues: antivirus, building to a network FS, using a storage driver that assumes cheap snapshots with a FS that doesn't, busted build cache). Obviously building from scratch can take a little longer than 1.x did if you're installing Python/core/runner, since those were previously baked into our base images. You can make your own base image for your EEs so you don't have to do that on every build if that's a problem, but the build cache usually makes that moot anyway, unless you're doing things that drastically change the early stages as you iterate.
... as a data point, the following EE def builds for me via ansible-builder build
in:
version: 3
images: base_image:
name: registry.access.redhat.com/ubi9/ubi:latest
dependencies:
python_interpreter:
package_system: python3.11
# (optional) provide the path of the Python interpreter to use for everything else.
# defaults to /usr/bin/python3; we'll do some sanity checking with this later...
python_path: /usr/bin/python3.11
ansible_core:
pip install
- a git ref, whateverpackage_pip: ansible-core<2.16
ansible_runner: package_pip: ansible-runner galaxy: collections:
thanks for your answers. Here the exec env I use:
---
version: 3
build_arg_defaults:
ANSIBLE_GALAXY_CLI_COLLECTION_OPTS: '--pre -c'
dependencies:
ansible_core:
package_pip: ansible-core==2.15.4
ansible_runner:
package_pip: ansible-runner==2.3.4
galaxy: requirements.yml
python: requirements.txt
system: bindep.txt
python_interpreter:
package_system: "python311"
python_path: "/usr/bin/python3.11"
images:
base_image:
name: automationhub.tld/ansible-automation-platform-24/ee-minimal-rhel9:latest
# Custom package manager path for the RHEL based images
options:
package_manager_path: /usr/bin/microdnf
additional_build_files:
- src: ansible.cfg
dest: configs
- src: ubi9.repo
dest: configs
- src: pip.conf
dest: configs
- src: rootCA.crt
dest: configs
additional_build_steps:
prepend_base:
- RUN rm -rf /etc/yum.repos.d/*
- COPY _build/configs/ubi9.repo /etc/yum.repos.d/ubi9.repo
- COPY _build/configs/pip.conf /etc/pip.conf
- COPY _build/configs/rootCA.crt /etc/pki/ca-trust/source/anchors/rootCA.crt
- RUN update-ca-trust
# enable SHA1 in rhel9 & gcc & python3-devel
- RUN $PKGMGR install -y crypto-policies-scripts gcc python3.11-devel
#- RUN update-crypto-policies --set DEFAULT:SHA1
prepend_galaxy:
- COPY _build/configs/ansible.cfg /etc/ansible/ansible.cfg
- ENV ANSIBLE_GALAXY_SERVER_AUTOMATION_HUB_URL=https://s01vl9989234.fr.net.intra
append_base:
- RUN alternatives --install /usr/bin/python3 python3 /usr/bin/python3.11 0
- RUN $PKGMGR install -y crypto-policies-scripts gcc python3.11-devel
- RUN update-crypto-policies --set DEFAULT:SHA1
prepend_final: |
RUN whoami
RUN cat /etc/os-release
append_final:
- RUN echo This is a post-install command!
We use this for different EE creations. the requirements.* files never have more than a couple of deps and collections. This is not what makes the difference.
To illustrate the change, which occurred 3 weeks ago, we were building in 4 to 13mns with ansible-builder v1 (but the Container file was pre-generated and a podman build cmd was used.)
Now with v3, using the same CI, but using a single cmd ansible-builder build -t name:tag -v3
13mns is the fastest build we have, and that's when we rebuild a successfully built image, i.e. only using cache.
A new or partly new image build takes from 30 to 50 mns.
BTW
update-crypto-policies --set DEFAULT:SHA1
to deactivate the ban on SHA1 on RHEL9I think both issues must stem from the fact that prepend_base
actions are run "in a builder image", I suppose, that is not the finally delivered image.
If I'm guessing alright, I should put those directives somewhere else, but I'm unsure where since the ContainerFile is hidden when using the ansible-builder build cmd. Maybe the doc could make clearer what is happening.
I can't explain why it now takes me 45~50 mns for a new build with juste 10 python deps 3 collections and python 3.11.
It use to take minutes only with ansible-builder 1.0
It's not that there are any step that takes very long but there seems to be so many steps and each seems to take at least 10s, even to process an ARG or run the simplest of RUN command.