RFE Remove explicit package installs

[trown]

In the current method of building up images, we pre-install all of the packages we might need in the base image, then feed this image to DIB to do the overcloud image builds, and finally modify the installed packages slightly to create the undercloud image.

This has the benefit of efficiency since we only download the packages once, but the drawback that we need to maintain a list of the packages to install. We recently updated our image building YAML to include the right upstream elements so that we will always get the right packages installed. However, as new packages are added to those elements, we will gradually lose our efficiency gain if we don't also update the package list.

Instead of maintaining a list of packages to install, we can instead pre-download every package from the delorean and delorean-deps repos without installing any of them. This will give us the same efficiency gain, without having to maintain any lists of packages.

There is another issue open #20, that would benefit from this approach. By using locally snapshot repos like this, we would be guaranteed to get the same content at run time as passed CI.

There is a WIP patch up to add the reposync to the build here. Beyond what is in that patch the other work items would be:

1. update tripleo-quickstart to explicitly install python-tripleoclient prior to running openstack undercloud install
2. remove all package lists from this project

[halcyondude]

I think this is the right direction and makes sense. It allows us to keep the time/bandwidth savings of having the packages pre-downloaded, while allowing upstream tooling to define the content and perform the (local rpm) install.

Regarding removing "all" package lists, I like the idea of retaining the capability to augment the package lists (defaulting lists to empty). While making the default / normative behavior to be a pass thru for upstream tooling, this will continue to allow developers and/or test to quickly add packages as part of the build process. #22 and #7 touch on this in different ways.

It would also enable scenarios/workflows around:

• iterative edit-build-debug cycles
• white-box testing
• prototyping / boot-strapping new features (birthing packages in the process).
• release/delivery/ci folk that might have reason to experiment. A recent example is sorting out RHEL 7.2 to 7.3 requirements, updates to centos/epel, broken delorean deps, etc...

[halcyondude]

---

@trown you had asked for file sizes, etc. I'm still making this work (WIP should turn to actual patch shortly) but here's the details: https://da.gd/skq67

114M delorean 562M delorean-mitaka-testing

---

A brief note on implementation, there are really 2 variants of specifying repositories being used. They are:

1. simple URI to .repo file (like delorean and delorean-deps)
2. an invocation of a tool installed by an rpm, that emits/places .repo files in /etc/yum.repos.d. Examples of this are the ceph hammer/jewel files, centos-release itself, downstream rhos-release tool (for OSP repos), etc.

The current patch addresses the first case, however we still have a jinja'd bash script that invokes ceph-hammer (in the default case). In addition the downstream / RDO on RHEL internal cases use an analogous tool. A complete implementation satisfies both cases, and removes the variability that these runtime tools introduce (vs. declarative+static inputs).

Supporting scenario 2 might come in a later patch (hoping to not need this). I have a few alternate implementations I'm kicking around now.

stage on buildhost

• reposync to build host
• modify .repo files to use file:// links
• inject rpm's and .repos to the base image
• is simple, makes debugging easy, and I like this approach. It's very clear and when we have the predictable dependency issues that regularly happen during development, the working directory is already set up to quickly iterate/fix and move on.
• a chroot jail is needed (or some other mechanism) such that we can handle the 2nd case above. This also get's a little messy when wanting a generic mechanism to specify what repo files are being used (in yaml) vs. burying this in a jinja'd bash file

reposync directly to the qcow

• reposync happens inside virt-customize
• don't need to worry about chroot jails, or modifying .repo files.
• just install repo files directly
• reposync isn't clean, as it get's a little more tricky to simply/cleanly reposync just what we want (and not all of everything in /etc/yum.repos.d

Neither is a slam dunk. Experimenting with both. Updates to come.