come up with a design for config containers

[bcressey]

What I'd like: Quite a few issues - #2218, #1702, and many others - would ideally have a way to be quickly unblocked by targeted config file edits until the requested functionality could be integrated in the settings API. Currently this is strongly discouraged, albeit possible in limited ways through containers with the super_t label.

There are a few major concerns that motivate this limitation:

• settings can be changed via API at any time, clobbering any unexpected edits to the config file
• config files might move around across an update, breaking expectations on where to make the edit
• modifying certain config files could impact the security of the system

Any solution will need to address these concerns in some way.

Any alternatives you've considered: Keep the status quo.

[bcressey]

I sketched out a possible approach in https://github.com/bottlerocket-os/bottlerocket/issues/2218#issuecomment-1186045512 and https://github.com/bottlerocket-os/bottlerocket/issues/2218#issuecomment-1186084117.

The proposal is to add a new kind of host container - config containers - that will be run whenever config files are generated, and given a chance to modify the contents before they're copied into place.

I suggested this:

An alternative that would make "config containers" much easier to use would be to just mount all of /etc into the container and give them the super_t label to let them modify anything there.

However, that would also allow config containers to edit anything else in /etc. For example, they could create new systemd services and modify /etc/passwd to add new users to the host. I'm not prepared to go quite that far yet.

Instead I'd picture this working the way that the API settings datastore does during migrations, where a copy of the tree with only the generated config files would be mounted into each config container in succession.