search

systemd / systemd

The systemd System and Service Manager
https://systemd.io
GNU General Public License v2.0
13.32k stars 3.81k forks source link

Parallelize systemd-modules-load #33069

Open ericcurtin opened 5 months ago

ericcurtin commented 5 months ago

Component

systemd-modules-load

Is your feature request related to a problem? Please describe

In our setup, we do not use udev in the initramfs because the hardware configuration is known at the time of the OS build. Consequently, we have a predefined list of kernel modules that need to be loaded during boot which is faster than udev. The current implementation of systemd-modules-load loads kernel modules sequentially, which can lead to slower boot times.

Describe the solution you'd like

We would like systemd-modules-load to be enhanced to load multiple kernel modules in parallel. By doing so, we can significantly reduce the overall time required to load all the necessary kernel modules during boot, improving the boot performance.

Describe alternatives you've considered

Sequential loading out of the box via systemd-modules-load, but we've seen faster boot speeds from similar tools that do this in parallel.

The main reason I open this is to start a design discussion on how it would be implemented in systemd-modules-load.

The systemd version you checked that didn't have the feature you are asking for

Latest version of systemd

poettering commented 5 months ago

systemd device units and stuff require udev to be up really, to find the devices and being able to mount that.

ericcurtin commented 5 months ago

@poettering so for this case, we will still bring up udev to avoid headaches like above, we still want to use udev eventually in the boot sequence.

This is for early performance-critical modules that we want to load as quickly as possible, we don't want to do it for every module anyway, if we can let udev control as much as possible we will, because udev makes our lives easier.

ericcurtin commented 5 months ago

Examples might be storage devices, network devices, etc. Depends on the bottleneck.

ericcurtin commented 5 months ago

systemd-modules-load even without parallelisation is more well suited than udev for the performance critical modules.

ericcurtin commented 5 months ago

We were thinking of no systemd-udevd in initramfs, just a minimal systemd service manager with systemd-modules-load but with systemd-udevd in the rootfs.

poettering commented 5 months ago

the usual way to prioritize certain device classes over others is via --prioritized-subsystem= on the "udevadm trigger" command line.

poettering commented 5 months ago

Note that iirc traditionally kernel synchronized module loading anyway, so what userspace parallelized there wasn't really improving anything.

ericcurtin commented 5 months ago

the usual way to priorize certain device classes over others is via --prioritized-subsystem= on the "udevadm trigger" command line.

Thanks for the pointer, we looked into this, but when trying to achieve super fast boots for things like <1 seconds, it's hard to beat, just load this list of pre-defined files, not great for generics, but when you have the ability to compose a specific rootfs and/or initramfs during an image build, it's worth populating the list of files for a given device.

ericcurtin commented 5 months ago

It's possible that I am being given inaccurate information from the implementers of an alternate kernel module loading tool that their multi-threaded solution is faster when indeed they haven't done precise measurements.

ericcurtin commented 5 months ago

Maybe they are seeing benefits in having everything queued up in kernelspace rather than waiting in userspace, I should quiz more.

poettering commented 5 months ago

why bother with kmod loading at all if you know your system? just link this into your kernel then?

ericcurtin commented 5 months ago

Because there's only one kernel rpm/build for many pieces of hardware (and we can't have more than one because reasons...)... But we can customise the initramfs/rootfs per device...

poettering commented 5 months ago

I mean, I am not against this if you want to teach systemd-modules-load this kind of parallelization. i.e. just iterate though the listed modules and fork off a new child for each of them to to the actual module loading in, and do the last one in the parent.

ericcurtin commented 5 months ago

The other implementation that is apparently faster is multi-threaded via pthreads, but if we got around to doing this would there be a preference on using multiple-processes vs multiple-threads? Just because it sounds like a preference towards multiple-processes from the last comment...

poettering commented 5 months ago

Usually we don't like threads very much in the systemd codebase, except where unavoidable.