Open mabod opened 4 years ago
"And this is true since July, not just recently."
July is where my log starts. I do not have anything earlier than that. The message may have started to occur earlier than that.
@mabod Thanks, thats quite relevant to note... It now sounds like it only appears since July... ;)
I checked my backup. I see the messages in the oldest backup I have from 27. April:
Apr 27 09:00:25 rakete udevadm[448]: systemd-udev-settle.service is deprecated.
By the way, the more detailed message which explicitly names zfs-import-cache.service
started with 7. August.
cc: @aerusso @InsanePrawn @rlaager
This is likely going to be non-trivial to fix without creating race condition boot failures. We really need the full proper solution of incremental assembly. That is, as ZFS devices appear, build up pool/vdev information in memory. Once there are sufficient disks to import the pool, (if not all disks are present) start a timer. At the earlier of all the disks appearing or the timer elapsing, import the pool. This would probably be in zed but could be in the kernel.
The mount generator should make filesystem mount units depend on some unit for the pool, which in some way only gets started once the pool has imported.
Edit: A way to make this work without all that effort might be to have the generator build a unit to import a specific pool and have that unit depend on the underlying devices. The concern I have with that is that it would presumably fail if a device is missing, defeating the pool redundancy.
This is what I'm imagining:
zfs-pool-member@.service
files that wait for /dev/%i
to be ready, with some timeout.zfs-pool@.service
that Requires=
and Before=
those member service files, and performs a pool importzfs-mount-generator
to make the mount units depend on the zfs-pool@.service
history_event-zfs-list-cacher.sh
to keep zfs-pool-member@
symlinks appropriately updated.That should work. It's slightly different than what I proposed, but likely far less work since it leverages systemd. The main difference is that I was proposing a relative timeout (once enough devices are present, start the timeout) and this would use an absolute timeout (start the timer for all disks initially). That's not likely a big difference in practice.
"And this is true since July, not just recently."
Thats pretty recent in regards to the OpenZFS release timetable, last release was from May. It does still needs to be fixed however!
The messages started to name "offending" units this summer, but the use of systemd-udev-settle.service
has been discouraged since 2011 and deprecated since ~2018.
Future versions of systemd will print this message on boot:
Usage of the systemd service unit systemd-udev-settle.service is deprecated. It
inserts artificial delays into the boot process without providing the
guarantees other subsystems traditionally assumed it provides. Relying on this
service is racy, and it is generally a bug to make use of it and depend on it.
Traditionally, this service's job was to wait until all devices a system
possesses have been fully probed and initialized, delaying boot until this
phase is completed. However, today's systems and hardware generally don't work
this way anymore, hardware today may show up any time and take any time to be
probed and initialized. Thus, in the general case, it's no longer possible to
correctly delay boot until "all devices" have been processed, as it is not
clear what "all devices" means and when they have been found. This is in
particular the case if USB hardware or network-attached hardware is used.
Modern software that requires some specific hardware (such as a network device
or block device) to operate should only wait for the specific devices it needs
to show up, and otherwise operate asynchronously initializing devices as they
appear during boot and during runtime without delaying the boot process.
It is a defect of the software in question if it doesn't work this way, and
still pulls systemd-udev-settle.service into the boot process.
Please file a bug report against the following units, with a request for it to
be updated to operate in a hotplug fashion without depending on
systemd-udev-settle.service:
If you have a fairly static set of pools then would simply maintaining a cachefile using zpool set cachefile=/etc/zfs/zpool.cache <pool>
for every pool you have (or maybe just your boot and root pools) not be enough?
I went ahead and did systemctl mask systemd-udev-settle
and zfs still works after reboot.
but that's a simple 1 disk laptop pool, should it work as well for more complex setups?
I tried this as well, but without systemd-udev-settle non of my pools gets imported during boot. I have three pools: a single disc pool, a RAID0 pool and a RAID10 pool. Non of them shows up.
ah, makes sense for me what I did: the pool is on a partition residing on the same physical device where the rootfs is, so it's obviously already online at the time when zfs loads...
I'm root on ZFS as well, so this dependency is useless (from what I'm reading) since I wouldn't be booting unless the relevant pools were already online. It also adds 7 seconds to my boot that I'd rather not have, if possible.
Additionally, I can't even manually mask it since I'm running on NixOS and it won't allow it due to this dependency (arguably, properly so, buuuuuut...)
❯ systemctl mask systemd-udev-settle
Failed to mask unit: File /etc/systemd/system/systemd-udev-settle.service already exists and is a symlink to /nix/store/sj364k7lsr47i87f7iv835lvvn7g4fqm-systemd-250.4/example/systemd/system/systemd-udev-settle.service.
I originally commented about this over on the nixos discussion about this https://github.com/NixOS/nixpkgs/issues/73095#issuecomment-1226217455 They managed to remove this dependency from a whole bunch of stuff EXCEPT for ZFS.
I'm root on ZFS as well, so this dependency is useless (from what I'm reading) since I wouldn't be booting unless the relevant pools were already online. It also adds 7 seconds to my boot that I'd rather not have, if possible.
The point is, that a single disc with root on zfs might work but not a pool with multiple discs.
Form one of my earlier posts:
I tried this as well, but without systemd-udev-settle non of my pools gets imported during boot. I have three pools: a single disc pool, a RAID0 pool and a RAID10 pool. Non of them shows up.
Is this so hard to fix? All you have to do this is listening for the udev events of the disks (I presume) you need to mount the pool before actually trying to. You don't actually have to do this using the library: a shell script parsing the output of udevadm monitor -s subsystem/devicetype
should work as well.
Also, if you know which devices you need ahead of time, you can just generate a systemd unit which specifies the dependency on the devices using the special .device
units.
Just want to add that you get the same warning with the zfs-import-scan.service
Apr 05 16:31:57 server-pi-14 udevadm[625]: systemd-udev-settle.service is deprecated. Please fix zfs-import-scan.service not to pull it in.
Is this issue being worked on?
The exposed zfs_vdev_open_timeout_ms
variable is not a solution. Not even a workaround. When I mask systemd-udev-settle.service
and set zfs_vdev_open_timeout_ms=30000
(30 s) none of my pools is coming up during boot.
Not to necrobump but I wanted to ask what was the status on this? systemd-udev-settle.service has been deperecated for quite a few years now.
FIX:
Edit your zfs-import-cache.service
, and add this line
After=<your service that maps the disks to the machine>
in my case, i am using Ceph RBD, so i added rbdmap.service
you can also add a Before=zfs-import-cache.service
in your disk mapping service.
@rev1akarevv1e : This is not a solution but a workaround. zfs-import-cache.service should work without the need to edit it.
By the way, your are referring to "disk mapping services". What is this for internal SATA devices?
@rev1akarevv1e : This is not a solution but a workaround. zfs-import-cache.service should work without the need to edit it.
By the way, your are referring to "disk mapping services". What is this for internal SATA devices?
It may be seen as a workaround, but in reality ZFS would need to know how the OS maps the devices, and since there are many ways to map a device imho it would make no sense for ZFS to have this feature when you can simply add this directive in systemd.
Use systemctl list-units --type=mount
to determine after which service zfs-import-cache.service
needs to start.
System information
Describe the problem you're observing
During every boot I get the following message:
udevadm[398]: systemd-udev-settle.service is deprecated. Please fix zfs-import-cache.service not to pull it in.
And this is true since July, not just recently. Nevertheless, the pools are imported just fine. But from my point of view that needs to be fixed.