abstract: images built (and metadata needed to do so)

[halcyondude]

The current implementation has 2 disk-image-create passes, both hard coded (ironic python agent and overcloud-full). Older (and future) versions will require differences (e.g. discovery ramdisk for Kilo).

This issue tracks migrating these 2 hard coded passes (IPA, overcloud) to a declarative, data-driven idiom.

We will specify via YML a list of images to build. For each image the following shall be declarable:

1. Name
2. DIB elements
3. DIB packages
4. Custom env vars

Generating disk images will merely be a traversal of this list, and will maintain ability to rundisk-image-create tasks async, to leverage multi-core HW and nice disk subsystems/arrays.

[halcyondude]

WIP: https://review.gerrithub.io/#/c/273340/ feedback heartily welcomed

[halcyondude]

Now that we have tripleo-common support merged (vs. thin DIB wrapper), work here is proceeding.

[halcyondude]

After having fingers in this a bit, I'm headed in the following direction:

Current implementation The yaml format describing the images that tripleo-common expects is presently something we generate from jinja templates (hard coded) that ref a gaggle of env variables (artibdib*). We hard code the templates (one for IPA, one for overcloud). This allows us to run the image builds (using tripleo-common mechanism via python library) in parallel as ansible async tasks. This...

• reduces wall clock time, and is neat
• adds a layer of indirection that is not needed.
• It also increases the number of raw env variables that are exposed (and makes it more fragile)
• (perhaps most importantly), buries the capability to run async builds (dib passes in parallel) in this ansible role, when it should really be done in tripleo-common, so anyone using that library (including this role) benefits.
• Makes building kilo based images (needed for upgrade testing in CI) impossible, as the image loadout is different (discovery ramdisk vs. IPA)

Working on this now

• Remove a layer of this onion. This role will use the same YAML format that is expected by tripelo-common.
• We'll still template it (to leave an artifact and be consistent), but that template will effectively be a file copy. However if some other custom images are needed, it can still be overridden by a different template (vs. the default one)
• a bunch of role variables --> yaml. Follow ethos of "keep removing things until you can't" (less > more)

[halcyondude]

After discussion with @bcrochet going to look at adding an env dict to the yaml we pass to tripleo-common. There are a set of ENV variables that DIB uses to override (vs. command line args). In practice we need these. The corresponding change to upstream tripleo-common will allow for specifying env variables in the yaml, but will still allow for overriding via env (to remain consistent...ENV block wins all fights). Specifically we need the following:

• DIB_YUM_REPO_CONF
• DIB_DEFAULT_INSTALL_TYPE
• ELEMENTS PATH
• DIB_LOCAL_IMAGE
• DIB_CLOUD_INIT_ETC_HOSTS

FYI: @trown @weshayutin @bcrochet

[halcyondude]

WIP: https://review.gerrithub.io/#/c/276629/

[halcyondude]

tripleo-common change to have env variables as part of yaml input just merged.

https://github.com/openstack/tripleo-common/commit/b98590791c060928cbe43ee59082ad3cb684531e

proceeding...

[halcyondude]

https://review.gerrithub.io/#/c/276629/ is no longer WIP, waiting on gates to merge.