Checkservices. Unable to find/parse PID for systemd-udevd and shadowsocks service

[webcaptcha]

I put a hook to full system upgrade to launch checkservices command from https://archlinux.org/packages/?name=archlinux-contrib. Here is an output

:: Synchronizing package databases...
 core is up to date
 extra is up to date
 community is up to date
:: Starting full system upgrade...
 there is nothing to do
:: Run pacdiff
:: Reload systemd
:: Services with broken maps files
Error:: Unable to find pid file for shadowsocks-rust@config_rust.service.
Error:: Unable to parse pid file for systemd-udevd.service.
Found: 0
:: Services missing on the system bus
:: List failed units

Actually the question is how to fix that Errors. Either udevd service or shadowsocks service working without errors.

If it matters I'm using linux-hardened kernel.

> sudo systemctl show systemd-udevd.service | grep PID
[sudo] password for lex:
GuessMainPID=yes
MainPID=278
ControlPID=0
ExecMainPID=278

[lahwaacz]

See https://github.com/archlinux/contrib/issues/58#issuecomment-1168325981

[webcaptcha]

So I see the correct file should be /sys/fs/cgroup/system.slice/systemd-udevd.service/udev/cgroup.procs Is it what I can change manually?

[FranklinYu]

I think we should keep this issue open, since #58 is incorrectly deduplicated. This issue can cover all services with CGroup-delegation. checkservices should check the Delegate=pids to decide the correct Pid file.

[GnomeBeans]

get_broken_maps function experimental with Delegate pid option. Let me know if this might work.

get_broken_maps():
    local service path pidfile unit_path maps_path pids deleted
    local -a pids=()
    local -i pid=0
    for service in $(get_services()); do
        unit_path="$(systemctl --property ControlGroup --value show "$service")"
        # hack to fix to systemd internal cgroup escaping on slice
        unit_path="$(printf "$unit_path"|sed 's,\\x5c,\\,')"
        # get the right pidfile name
        pidfile=''

        if [[ -d "$SYSTEMD_CGROUP_BASE_PATH$unit_path/cgroup.procs" ]]; then
            pidfile="$SYSTEMD_CGROUP_BASE_PATH$unit_path/cgroup.procs"
        elif [[ -d "$SYSTEMD_CGROUP_BASE_PATH$unit_path/tasks" ]]; then
            pidfile="$SYSTEMD_CGROUP_BASE_PATH$unit_path/tasks"
        fi

        if [[ -z "$pidfile" ]]; then
            error "Unable to find pid file for $service."
            continue
        fi

        # skip non system units
        (( $USER_SLICE == 0 )) && [[ "$unit_path" =~ /user\.slice/ ]] && continue

        # parse pidfile
        pids=( $(< "$pidfile") )
        if (( "${#pids[*]}" == 0 )); then
            error "Unable to parse pid file for $service."
            continue
        fi

        # check for Delegate=pids option
        delegate_pids="$(systemctl --property Delegate --value show "$service")"
        if [[ $delegate_pids == yes ]]; then
            # get maps path for each pid
            for pid in "${pids[@]}"; do
                maps_path="/proc/$pid/maps"
                [[ -r "$maps_path" ]] || {
                    error "Unable to read maps file of $service for pid $pid."
                    continue
                }

                # only file mapped as executable
                deleted="$(grep -F '(deleted)' "$maps_path"|sed -nr 's|^\S+ ..x. \S+ \S+ \S+ \s+||p')"
                if [[ $deleted ]]; then
                    printf "%s\n" $service
                    break
                fi
            done
        else
            # handle services without Delegate=pids option
            maps_path="$SYSTEMD_CGROUP_BASE_PATH$unit_path/maps"
            [[ -r "$maps_path" ]] || {
                error "Unable to read maps file of $service."
                continue
            }

            # only file mapped as executable
            deleted="$(grep -F '(deleted)' "$maps_path"|sed -nr 's|^\S+ ..x. \S+ \S+ \S+ \s+||p')"
            if [[ $deleted ]]; then
                printf "%s\n" $service
                break
            fi
        fi
    done

[injiniero]

Suggested solution:

get_broken_maps() {
(...)
for service in $(get_services); do
        unit_path="$(systemctl --property ControlGroup --value show "$service")"
        # hack to fix to systemd internal cgroup escaping on slice
        unit_path="$(printf "$unit_path"|sed 's,\\x5c,\\,')"
# ---- PATCH START
        # has Delegate Subgroup?
        delegate_path="$(systemctl --property DelegateSubgroup --value show "$service")"
        [[ -n ${delegate_path} ]] && unit_path="${unit_path}/${delegate_path}"
# ---- PATCH END
        # get the right pidfile name
        pidfile=''
        for path in "$SYSTEMD_CGROUP_BASE_PATH$unit_path/cgroup.procs" \
            "$SYSTEMD_CGROUP_BASE_PATH$unit_path/tasks"; do
            [[ -r "$path" ]] && pidfile="$path" && continue
        done
(...)
}

[webcaptcha]

https://github.com/archlinux/contrib/issues/61#issuecomment-1841860760 @GnomeBeans Sorry for delay. Maybe there is a type with a script of maybe it has been written for not bash shell? If I just copy paste it gives me an error syntax error near unexpected token)'`

@injiniero Looks like it works for systemd-udevd service. I'm still get an Error for shadowsocks service.

[injiniero]

@injiniero Looks like it works for systemd-udevd service. I'm still get an Error for shadowsocks service.

Can you check this?

Get the path returned by: systemctl --property ControlGroup --value show shadowsocks-rust@config_rust.service

Add the path at the end of: /sys/fs/cgroup/<---- add here --->

And check the contents of the file: /sys/fs/cgroup/<---- add here --->/cgroup.procs

is empty that file?

Can you paste here the service file? You can get its contents with: sudo systemctl edit --full shadowsocks-rust@config_rust.service

[webcaptcha]

@injiniero since that I've changed the name of config file a little bit.

systemctl --property ControlGroup --value show shadowsocks-rust@config.service

/sys/fs/cgroup/system.slice/system-shadowsocks\x2drust.slice/shadowsocks-rust@config.service/cgroup.procs

And check the contents of the file: /sys/fs/cgroup/<---- add here --->/cgroup.procs

❯ cat /sys/fs/cgroup/system.slice/system-shadowsocks\\x2drust.slice/shadowsocks-rust@config.service/cgroup.procs
914

Can you paste here the service file?

[Unit]
Description=Shadowsocks-Rust Client Service
After=network.target
Wants=network-online.target

[Service]
Type=simple
DynamicUser=yes
NoNewPrivileges=yes
AmbientCapabilities=CAP_NET_BIND_SERVICE CAP_NET_ADMIN
ExecStart=/usr/bin/ssservice local --log-without-time -c /etc/shadowsocks-rust/%i.json

[Install]
WantedBy=multi-user.target

[injiniero]

/sys/fs/cgroup/system.slice/system-shadowsocks\x2drust.slice/shadowsocks-rust@config.service/cgroup.procs

I assume that the path returned was /sys/fs/cgroup/system.slice/system-shadowsocks\x2drust.slice/shadowsocks-rust@config.service/

When you cat the cgroup.procs in that path, you get a pid (914). So the path is valid and this is the service pid.

Maybe the problem is that the escaped character \x2d is not handled well in the script. This character is just a hyphen and I don't know why it's used escaped in path names.

Can you test this (just add the error line at the end of my patch):

# ---- PATCH START
        # has Delegate Subgroup?
        delegate_path="$(systemctl --property DelegateSubgroup --value show "$service")"
        [[ -n ${delegate_path} ]] && unit_path="${unit_path}/${delegate_path}"
        error "Unit-path: ${unit_path}"
# ---- PATCH END

You'll see an output like this:

Error:: Unit-path: /system.slice/accounts-daemon.service
Error:: Unit-path: /system.slice/bluetooth.service
Error:: Unit-path: /system.slice/colord.service
Error:: Unit-path: /system.slice/dbus-broker.service
Error:: Unit-path: /system.slice/gdm.service
(...)

Paste here the line with the shadowsocks service.

Remove the error line after the test.

[webcaptcha]

@injiniero

I assume that the path returned was

/system.slice/system-shadowsocks\x2drust.slice/shadowsocks-rust@config.service

Paste here the line with the shadowsocks service.

Error:: Unit-path: /system.slice/system-shadowsocks-rust.slice/shadowsocks-rust@config.service
Error:: Unable to find pid file for shadowsocks-rust@config.service.

[injiniero]

Your shell changes the escaped char "\x2d" for "-" as in: /system.slice/system-shadowsocks\x2drust.slice/shadowsocks-rust@config.service /system.slice/system-shadowsocks-rust.slice/shadowsocks-rust@config.service because of this the script can't find the right service path

At the beginning of the checkservices script, the line:

# bash options
shopt -s xpg_echo

forces bash to expand the escaped characters by default but only for the echo command and it's not used in the script when handling the unit paths. The error function that you have tested above uses printf function with "%s" as string format and this doesn't expand the escaped chars, but your script does. So I can't guess where's your particular problem. Just check that you're using the last version of checkservices and your bash settings.

[rkorn86]

It should be

unit_path="$(printf '%s' "$unit_path"|sed 's,\\x5c,\\,')"

instead of

unit_path="$(printf "$unit_path"|sed 's,\\x5c,\\,')"

on line 116, to keep special characters like \x2d in /system.slice/system-serial\x2dgetty.slice/serial-getty@ttyS0.service