chef-cookbooks / community_cookbook_documentation

Pertinent documentation for all Chef owned community cookbooks
Apache License 2.0
39 stars 48 forks source link

Make Private Images Reproducable #15

Closed karmix closed 6 years ago

karmix commented 7 years ago

Jennifer and Tim did a great job explaining how Chef employees can configure their environment to use private images. The instruction for those that are not Chef employees, however simply says that the reader must obtain appropriate licenses and image, and leaves the rest for the reader to figure out. It can be frustrating and difficult for a non-Chef employee to collaborate if they are trying to contribute code that works in their environment to a Chef project, only to have CI/CD fail in a private image into which they have no insight.

While licensing restrictions may prevent Chef from distributing private images used in CI/CD, those restrictions do not apply to the code Chef uses to create the images. Please consider publishing (and referencing from these docs) the code Chef uses to generate the private images used by CI/CD. This would allow collaborators to use their own licenses to generate roughly identical private images that they could use for testing and debugging purposes. In the absence of licensing, detailed configuration information still help a collaborator deduce how their contribution might be behave in the private image's environment. If for some reason, Chef cannot publish the code, please describe the configuration of each of these images in enough detail that a collaborator would have a good chance of manually reproducing the image, themselves.

cheeseplus commented 7 years ago

The timing here is uncanny as this is something I've long been pushing that should become a reality in the coming month(s).

This has been historically complicated for various reasons over time, lots of being around making Windows images even with the assistance of Packer is painful at best as generally there still needs to be manual intervention at some juncture. Related, the process of building images takes a long time, is error prone, and requires Real Hardware which is surprisingly in short supply these days. This means ad-hoc and one-off builds from the last poor soul who ventured down this precarious path. Also, at least for a bit there were other folks/orgs making images that filled this gap better than we could have at the time but many of those have since faded off.

The good news is that things have improved dramatically on a few critical fronts. Packer is substantially better, we're more experienced with it, and most importantly we'll have hardware in a datacenter for the sole purpose of building images via packer for our beloved bento boxes. As this will make our (very specifically my) lives easier we're simultaneously looking to expand the matrix of images we regularly build both externally via bento as well as internally. That is to say we have a hard enough time ourselves building/finding quality images for "private" operating systems.

The short term plan is to bring Windows templates into the bento project. @tas50 has already been active in updating these in Boxcutter (a bento alternative) so we've got a decent handle on the work that needs to be done here even if the build process still might require manual steps - at least we can document this along with the templates. This will achieve the goal but we also intend to take it further if possible as we're investigating precisely what the terms of licensing for hosting evaluation Windows images mean for Chef as an organization.

Note: though primarily concerned with Windows images, this goes for the other private platforms as well with regard to having reproducible images from templates. Less so for hosting as most of private non-Windows platforms require keys or specific hardware.

tas50 commented 6 years ago

I'm pretty pumped that I can finally close this issue out. We have Windows images in the Bento project now. There's some rough edges and I'm slowly working to fix that all up, but we're no longer testing with private chef images and you can build your own images using packer + chef + some powershell.