Support multiple postinstall, build, finalize scripts

[nopeslide]

Mkosi supports multiple extra/skeleton trees, which make it easy to seperate "roles" while provisioning files, but these usually need to be reflected inside the hooks that are executed. While finalize scripts may simply source/include those, postinstall/build scripts do not have this capability. I would love to see support for multiple scripts or on a higher level something like mkosi.files but only for a single container.

[nopeslide]

I would also fancy something like the drop-in folders systemd provides. having mkosi.postinst.d, mkosi.extra.d, ... directories in which symlinks or files are located that are executed/copied in lexical order

[nopeslide]

I would open a PR for this but would like to have some feedback which variant is preferred:

• mkosi.roles, Roles=, --roles=
  • name is debatable
  • new dir, setting and parameter that specify a list of directories aka "roles"
  • a "role" is a dir containing mkosi files/dirs like mkosi.postinst, mkosi.extra, ... basically the same as an mkosi structure
  • mkosi applies each "role" in lexical order
  • any mkosi.default inside a "role" dir is ignored
  • the global/non-"role" scripts/trees are applied after each role
  • i.e. for mkosi.finalize the order would be mkosi.roles/*/mkosi.finalize and then mkosi.finalize itself
• drop-in folders
  • extend parameters/settings to accept multiple paths
  • provide a drop-in directory for each setting
  • i.e. mkosi.extra.d for multiple extra trees
  • i.e. mkosi.finalize.d for multiple finalize scripts
  • contents of an drop-in folder are applied in lexical order
  • after all drop-in files/dirs are applied, the non-drop-in are applied
  • i.e. for mkosi.finalize the order would be mkosi.finalize.d/* and then mkosi.finalize itself

the drop-in folders also fix the ambiguity that multiple trees can be specified inside mkosi.default but not via the file/dir-structure

[poettering]

Not sure i grok what the "roles" thing is supposed to be, but adding mkosi.finalize.d/ and mkosi.build.d/ definitely makes sense to me.

mkosi.extra.d/ i don#t understand? it's already a drop-in dir, and can receive files from many owners?

[nopeslide]

Not sure i grok what the "roles" thing is supposed to be

It is just another file system structure achieving the same thing. Instead of drop ins like

mkosi.finalize.d/a
mkosi.finalize.d/b

You would have

mkosi.roles/A/mkosi.finalize
mkosi.roles/B/mkosi.finalize

This would keep related trees and scripts in the same subdir, ie.

mkosi.roles/serviceXY/mkosi.extra/... #service config
mkosi.roles/serviceXY/mkosi.postinst #enable service

I currently build a lot of containers with mkosi and have multiple subsets of these that require the same files and calls. So by having multiple serviceXY dirs I could simply "link" them into the mkosi flow and have no need to touch each and every container.

but adding mkosi.finalize.d/ and mkosi.build.d/ definitely makes sense to me.

This would also compliment the undocumented (?) mkosi.default.d/

mkosi.extra.d/ i don#t understand? it's already a drop-in dir, and can receive files from many owners?

Inside mkosi.default you may specify multiple trees, but there is no mechanic to specify multiple trees via the file system (or at least none I know). An example: The proposed drop-ins

mkosi.extra.d/A/
mkosi.extra.d/B/

Would make the setting obsolete currently used by me

ExtraTrees=
 mkosi.extra.d/A/
 mkosi.extra.d/B/

[gdiscry]

I was looking into opening a new RFE about supporting multiple scripts, but found that this one already exists. It's a bit out of date but the idea is there. In my opinion, the mkosi.roles presented above can already be achieved with folders in mkosi.conf.d.

With mkosi.conf.d, it's possible to split the configuration in multiple logical subsets. There are probably multiple use cases for this, and the ones I can think of are readability (which is as always subjective) and sharing (by using symbolic links to common folders). This is even more useful when using [Match] and/or presets.

However, it's only possible to specify one prepare, build, postinst and finalize script. If multiple configuration files specify a Script, or if multiple folder contain a mkosi.{prepare,build,postinst,finalize} script, only one can be executed.

For example, I have a script that fixes /etc/nsswitch.conf in Debian because libnss-systemd does not add the systemd source to the shadow database. I would like to put it in a common mkosi.conf.d/systemd folder that I share between my images to install systemd and various adjacent packages (such as libnss-systemd on Debian).

For now, my hack is to put that script and other postinst scripts in mkosi.conf.d/<role>/mkosi.extra/mkosi.postinst.d/ and use a single mkosi.postinst script that uses run-parts and then remove those files. It's far from ideal and the setup could be greatly simplified if mkosi would directly handle multiple scripts.

The first step I would like to propose is to convert the …Script configurations to …Scripts taking a list of executable paths.

If there is still an interest for the mkosi.{prepare,build,postinst,finalize}.d folders, they could then be supported by listing and adding their executable files when loading the configuration.

[DaanDeMeyer]

@gdiscry Sounds good to me to turn the scripts into list settings

[DaanDeMeyer]

Let's close this one as completed since we now support specifying multiple scripts. If anyone needs mkosi.build.d or similar, feel free to open a new RFE for that