search

openzfs / zfs

OpenZFS on Linux and FreeBSD
https://openzfs.github.io/openzfs-docs
Other
10.58k stars 1.75k forks source link

systemd-udev-settle.service is deprecated. Please fix zfs-import-cache.service not to pull it in. #10891

Open mabod opened 4 years ago

mabod commented 4 years ago

System information

Type Version/Name
Distribution Name Manjaro
Distribution Version Testing
Linux Kernel 5.4, 5.8
Architecture amd64
ZFS Version 0.8.4
SPL Version 0.8.4

Describe the problem you're observing

During every boot I get the following message:

udevadm[398]: systemd-udev-settle.service is deprecated. Please fix zfs-import-cache.service not to pull it in.

And this is true since July, not just recently. Nevertheless, the pools are imported just fine. But from my point of view that needs to be fixed.

PrivatePuffin commented 4 years ago

"And this is true since July, not just recently."

mabod commented 4 years ago

July is where my log starts. I do not have anything earlier than that. The message may have started to occur earlier than that.

PrivatePuffin commented 4 years ago

@mabod Thanks, thats quite relevant to note... It now sounds like it only appears since July... ;)

mabod commented 4 years ago

I checked my backup. I see the messages in the oldest backup I have from 27. April:

Apr 27 09:00:25 rakete udevadm[448]: systemd-udev-settle.service is deprecated.

By the way, the more detailed message which explicitly names zfs-import-cache.service started with 7. August.

behlendorf commented 4 years ago

cc: @aerusso @InsanePrawn @rlaager

rlaager commented 4 years ago

This is likely going to be non-trivial to fix without creating race condition boot failures. We really need the full proper solution of incremental assembly. That is, as ZFS devices appear, build up pool/vdev information in memory. Once there are sufficient disks to import the pool, (if not all disks are present) start a timer. At the earlier of all the disks appearing or the timer elapsing, import the pool. This would probably be in zed but could be in the kernel.

The mount generator should make filesystem mount units depend on some unit for the pool, which in some way only gets started once the pool has imported.

Edit: A way to make this work without all that effort might be to have the generator build a unit to import a specific pool and have that unit depend on the underlying devices. The concern I have with that is that it would presumably fail if a device is missing, defeating the pool redundancy.

aerusso commented 4 years ago

4943 I think would have addressed this, and it touched on many of your point, @rlaager.

This is what I'm imagining:

  1. zfs-pool-member@.service files that wait for /dev/%i to be ready, with some timeout.
  2. zfs-pool@.service that Requires= and Before= those member service files, and performs a pool import
  3. Code that lets us tell zpool import what specific devices to use, and not scan everything---not strictly necessary, but would be really great for a bunch of reasons.
  4. zfs-mount-generator to make the mount units depend on the zfs-pool@.service
  5. history_event-zfs-list-cacher.sh to keep zfs-pool-member@ symlinks appropriately updated.
rlaager commented 4 years ago

That should work. It's slightly different than what I proposed, but likely far less work since it leverages systemd. The main difference is that I was proposing a relative timeout (once enough devices are present, start the timeout) and this would use an absolute timeout (start the timer for all disks initially). That's not likely a big difference in practice.

rnhmjoj commented 3 years ago

"And this is true since July, not just recently."

Thats pretty recent in regards to the OpenZFS release timetable, last release was from May. It does still needs to be fixed however!

The messages started to name "offending" units this summer, but the use of systemd-udev-settle.service has been discouraged since 2011 and deprecated since ~2018.

Future versions of systemd will print this message on boot:

Usage of the systemd service unit systemd-udev-settle.service is deprecated. It
inserts artificial delays into the boot process without providing the
guarantees other subsystems traditionally assumed it provides. Relying on this
service is racy, and it is generally a bug to make use of it and depend on it.

Traditionally, this service's job was to wait until all devices a system
possesses have been fully probed and initialized, delaying boot until this
phase is completed. However, today's systems and hardware generally don't work
this way anymore, hardware today may show up any time and take any time to be
probed and initialized. Thus, in the general case, it's no longer possible to
correctly delay boot until "all devices" have been processed, as it is not
clear what "all devices" means and when they have been found. This is in
particular the case if USB hardware or network-attached hardware is used.

Modern software that requires some specific hardware (such as a network device
or block device) to operate should only wait for the specific devices it needs
to show up, and otherwise operate asynchronously initializing devices as they
appear during boot and during runtime without delaying the boot process.

It is a defect of the software in question if it doesn't work this way, and
still pulls systemd-udev-settle.service into the boot process.

Please file a bug report against the following units, with a request for it to
be updated to operate in a hotplug fashion without depending on
systemd-udev-settle.service:
savyajha commented 3 years ago

If you have a fairly static set of pools then would simply maintaining a cachefile using zpool set cachefile=/etc/zfs/zpool.cache <pool> for every pool you have (or maybe just your boot and root pools) not be enough?

mailinglists35 commented 2 years ago

I went ahead and did systemctl mask systemd-udev-settle and zfs still works after reboot. but that's a simple 1 disk laptop pool, should it work as well for more complex setups?

mabod commented 2 years ago

I tried this as well, but without systemd-udev-settle non of my pools gets imported during boot. I have three pools: a single disc pool, a RAID0 pool and a RAID10 pool. Non of them shows up.

mailinglists35 commented 2 years ago

ah, makes sense for me what I did: the pool is on a partition residing on the same physical device where the rootfs is, so it's obviously already online at the time when zfs loads...

pmarreck commented 2 years ago

I'm root on ZFS as well, so this dependency is useless (from what I'm reading) since I wouldn't be booting unless the relevant pools were already online. It also adds 7 seconds to my boot that I'd rather not have, if possible.

Additionally, I can't even manually mask it since I'm running on NixOS and it won't allow it due to this dependency (arguably, properly so, buuuuuut...)

❯ systemctl mask systemd-udev-settle
Failed to mask unit: File /etc/systemd/system/systemd-udev-settle.service already exists and is a symlink to /nix/store/sj364k7lsr47i87f7iv835lvvn7g4fqm-systemd-250.4/example/systemd/system/systemd-udev-settle.service.

I originally commented about this over on the nixos discussion about this https://github.com/NixOS/nixpkgs/issues/73095#issuecomment-1226217455 They managed to remove this dependency from a whole bunch of stuff EXCEPT for ZFS.

mabod commented 2 years ago

I'm root on ZFS as well, so this dependency is useless (from what I'm reading) since I wouldn't be booting unless the relevant pools were already online. It also adds 7 seconds to my boot that I'd rather not have, if possible.

The point is, that a single disc with root on zfs might work but not a pool with multiple discs.

Form one of my earlier posts:

I tried this as well, but without systemd-udev-settle non of my pools gets imported during boot. I have three pools: a single disc pool, a RAID0 pool and a RAID10 pool. Non of them shows up.

rnhmjoj commented 2 years ago

Is this so hard to fix? All you have to do this is listening for the udev events of the disks (I presume) you need to mount the pool before actually trying to. You don't actually have to do this using the library: a shell script parsing the output of udevadm monitor -s subsystem/devicetype should work as well. Also, if you know which devices you need ahead of time, you can just generate a systemd unit which specifies the dependency on the devices using the special .device units.

dvaerum commented 1 year ago

Just want to add that you get the same warning with the zfs-import-scan.service

Apr 05 16:31:57 server-pi-14 udevadm[625]: systemd-udev-settle.service is deprecated. Please fix zfs-import-scan.service not to pull it in.
mabod commented 1 year ago

Is this issue being worked on?

The exposed zfs_vdev_open_timeout_ms variable is not a solution. Not even a workaround. When I mask systemd-udev-settle.service and set zfs_vdev_open_timeout_ms=30000 (30 s) none of my pools is coming up during boot.

HikariNeee commented 7 months ago

Not to necrobump but I wanted to ask what was the status on this? systemd-udev-settle.service has been deperecated for quite a few years now.

rev1akarevv1e commented 5 months ago

FIX:

Edit your zfs-import-cache.service, and add this line

After=<your service that maps the disks to the machine>

in my case, i am using Ceph RBD, so i added rbdmap.service

you can also add a Before=zfs-import-cache.service in your disk mapping service.

mabod commented 5 months ago

@rev1akarevv1e : This is not a solution but a workaround. zfs-import-cache.service should work without the need to edit it.

By the way, your are referring to "disk mapping services". What is this for internal SATA devices?

rev1akarevv1e commented 5 months ago

@rev1akarevv1e : This is not a solution but a workaround. zfs-import-cache.service should work without the need to edit it.

By the way, your are referring to "disk mapping services". What is this for internal SATA devices?

It may be seen as a workaround, but in reality ZFS would need to know how the OS maps the devices, and since there are many ways to map a device imho it would make no sense for ZFS to have this feature when you can simply add this directive in systemd.

Use systemctl list-units --type=mount to determine after which service zfs-import-cache.service needs to start.