clan / runit

3 stars 2 forks source link

Proposed ebuild changes #6

Open powerman opened 2 months ago

powerman commented 2 months ago

(As you didn't enabled github discussions for the repo yet let's discuss some proposed changes in an issue.)

  1. I think it makes sense to drop support for migration from old versions: migrate_from_211() and $pre_212. Ebuild with this version wasn't in portage for probably some years, unlikely anyone might need this today.
  2. Make dependency on openrc optional (with USE-flag, enabled by default for compatibility with current ebuild) and avoid installing anything from files/ if it's disabled. I'm not sure is there any Gentoo users who actually use runit for boot using these openrc-dependent scripts. I'm using custom /etc/runit/{1,3} which won't depend on openrc. The guy who reported Gentoo bug about broken shutdown also shown his boot script and they also wasn't openrc-dependent AFAIR. I've own ebuild in my overlay which installs /etc/runit/{1,3} and it'll be nice to have no conflict on these files with sys-process/runit (with openrc USE-flag disabled).
  3. Any symlinks created by ebuild should be fixed to relative ones. This is useful in chroot or bind to docker container cases. E.g. I'm using netdata for monitoring, and I run netdata in a docker, so to monitor runit services I need to bind-mount runit directories inside netdata container, and using exactly same paths as used in host may be inconvenient inside a container (e.g. netdata usually expects all mounts from host to be inside /host directory).
  4. Anyone who uses runit to boot will also use runit's runlevels (at least one extra runlevel with single-user mode is very useful for fixing boot issues). In this use case the /etc/service is not a directory but a symlink to /etc/runit/runsvdir/current which is in turn a symlink to directory like /etc/runit/runsvdir/default or /etc/runit/runsvdir/single (except all these symlinks should be relative). Current ebuild is designed in a way which kind of prevents this or at least makes it feels like something very unusual and not welcomed. I propose to somehow fix this. How exactly this should be done is to be discussed. Maybe we should install /etc/runit/runsvdir/{default,current,previous} when openrc USE-flag is disabled. Or maybe we should just remove some warnings from pkg_postinst(). Or maybe we shouldn't do anything but keeping in mind this is legal use case which must be supported without extra hassle because of current ebuild design. Not sure.
clan commented 2 months ago
  1. for ebuild related modification, maybe we should open a gentoo bug for discussion. I don't know Gentoo Dev's opinion. I agree with 1 & 2: remove the migration script; add a USE flag (if enabled, don't install any custom files, but can put in doc to provide a reference for new comers)
  2. i see there is only one symlink (/sbin/runsvdir-start -> ../etc/runit/2), are there any changes need to made on current ebuild?
  3. my understanding is that runit just have three stages (bootstrap, running, reboot or shutdown), are you propose to change this to add an extra stage? For the single-user mode, maybe we should do it in the bootstrap stage (check command line options)? can you give some sample code to demonstrate?

for my own case, in current, I only use runit in docker, and just use the binary (don't use any script provided by gentoo), with customed files. I haven't use runit in a physical system yet.

So my understanding is there are two kinds of job:

  1. for runit itself, focus on the C code only
  2. for runit's use case, focus on howto run services by using runit. For Gentoo, ebuild can handle some common case. The rest probably can be handled in runit's documentation?
fungilife commented 2 months ago

This is a bit unnerving to say, you are the single entity in charge of fixing runit against gcc 14 but you have never used it in a physical system. Some of us have been using it for many years, and with the exception of s6/66 we wouldn't replace it even if it meant keeping the last compiled version unchanged.

Since the original author is not up to the task of fixing the code so it compiles with gcc14 why not rename it runit2 (as in consolekit2) and start from 0 before you patch. So runit2 2.1.3 will be the first release

In either case many of us are grateful for keeping runit alive, in any way you choose.

powerman commented 2 months ago

for ebuild related modification, maybe we should open a gentoo bug for discussion

We can discuss it there, if this is more convenient for you, but I don't expect any input from Gentoo devs. No one except us bothers about runit support in Gentoo - it should be very clear from their attempt to drop this package at all. If you ask me - for me it's much more convenient to use Github.

I don't know Gentoo Dev's opinion.

I think only real way to get their opinion is to open next PR. I suppose if we won't do anything really weird in ebuild and/or patches then they won't bother, because they expect us to know how to support runit much better than they do (and this is probably true, because we actually use it).

i see there is only one symlink (/sbin/runsvdir-start -> ../etc/runit/2)

No, I was talking mostly about /etc/service and /etc/runit/runsvdir/{current,previous}. How these works is explained at https://smarden.org/runit/runsvchdir.8. Here are main points:

Here is example how it looks like on my system (I've 3 "runlevels": default, single and console):

# l -d /etc/sv /etc/service /etc/runit/runsvdir/*
drwxr-xr-x  2 portage portage 4096 Feb 27  2017 /etc/runit/runsvdir/console
lrwxrwxrwx  1 root    root       7 Aug 22 16:58 /etc/runit/runsvdir/current -> default
drwxr-xr-x  2 portage portage 4096 Jun 19 20:45 /etc/runit/runsvdir/default
lrwxrwxrwx  1 root    root       6 Aug 22 16:53 /etc/runit/runsvdir/previous -> single
drwxr-xr-x  2 portage portage 4096 Mar 27  2017 /etc/runit/runsvdir/single
lrwxrwxrwx  1 root    root      27 Jun 12 17:55 /etc/service -> /etc/runit/runsvdir/current
drwxr-xr-x 97 root    root    4096 Jun 13 21:18 /etc/sv

And here is my "single" runlevel's services:

# l /etc/runit/runsvdir/single
total 0
lrwxrwxrwx 1 root root 13 Feb 27  2017 acpid -> /etc/sv/acpid
lrwxrwxrwx 1 root root 19 Feb 27  2017 agetty-tty1 -> /etc/sv/agetty-tty1
lrwxrwxrwx 1 root root 19 Feb 27  2017 agetty-tty2 -> /etc/sv/agetty-tty2
lrwxrwxrwx 1 root root 19 Feb 27  2017 agetty-tty3 -> /etc/sv/agetty-tty3
lrwxrwxrwx 1 root root 19 Feb 27  2017 agetty-tty4 -> /etc/sv/agetty-tty4
lrwxrwxrwx 1 root root 19 Feb 27  2017 agetty-tty5 -> /etc/sv/agetty-tty5
lrwxrwxrwx 1 root root 19 Feb 27  2017 agetty-tty6 -> /etc/sv/agetty-tty6
lrwxrwxrwx 1 root root 16 Feb 27  2017 dnscache -> /etc/sv/dnscache
lrwxrwxrwx 1 root root 11 Feb 27  2017 gpm -> /etc/sv/gpm
lrwxrwxrwx 1 root root 20 Feb 27  2017 socklog-klog -> /etc/sv/socklog-klog
lrwxrwxrwx 1 root root 20 Feb 27  2017 socklog-unix -> /etc/sv/socklog-unix
lrwxrwxrwx 1 root root 15 Feb 27  2017 tinydns -> /etc/sv/tinydns

are there any changes need to made on current ebuild?

At minimum in non-openrc mode it shouldn't install /etc/service as a directory - because it'll break setup shown above. Maybe we should provide some dirs/symlinks in /etc/runit/runsvdir. Or none of these and let user configure these manually (to make it work this way I have to modify gentoo ebuild this way and keep modified ebuild in my own overlay for all these years). This is to be discussed.

my understanding is that runit just have three stages (bootstrap, running, reboot or shutdown), are you propose to change this to add an extra stage? For the single-user mode, maybe we should do it in the bootstrap stage (check command line options)? can you give some sample code to demonstrate?

No, in runit's way to do "runlevel" there is no differences in boot and shutdown stages, only stage which differs is "running" (/etc/runit/2), or, more precisely, the stage itself also won't differ - it still runs all services found in /etc/service, - but the /etc/service may points (thru /etc/runit/runsvdir/current) to different directories with services.

powerman commented 2 months ago

And to make the story about runsvchdir complete here is the snippet from my /etc/runit/1 which provides me a way to set "runlevel" in Grub, using kernel options, before booting the kernel:

    # Set runlevel to:
    # - single       if kernel has param: S or single
    # - RUNLEVELNAME if kernel has param: runlevel=RUNLEVELNAME
    # - default      if kernel has no params or unable to set requested runlevel
    grep -q '\(^\| \)\(S\|single\)\( \|$\)' /proc/cmdline && runlevel='single'
    runsvchdir ${runlevel:-default} || runsvchdir default

P.S. Looking at it now I doubt it actually supports "runlevel=something". Probably it was implemented but then was removed. Or maybe it works if kernel params became env vars somehow without extra code in /etc/runit/1. Anyway, it's not really important here and now. Single user mode and default works for sure, and I don't use anything else anyway.

powerman commented 2 months ago

This is a bit unnerving to say, you are the single entity in charge of fixing runit against gcc 14 but you have never used it in a physical system.

Well, I'm here too, and I'm using runit to boot some servers and workstations since 2004.

Some of us have been using it for many years, and with the exception of s6/66 we wouldn't replace it even if it meant keeping the last compiled version unchanged.

Well… I was arguing for this (keep runit unchanged) a lot, but @clan disagree and merged "nosync" patch. It won't change runit's default behaviour (it's a new feature) and it was made by runit's author, so I don't think we need to rename runit to runit2 just because of this small change. Other patches just fix gcc/clang issues, so they won't change runit itself and thus safe.

clan commented 2 months ago

This is a bit unnerving to say, you are the single entity in charge of fixing runit against gcc 14 but you have never used it in a physical system. Some of us have been using it for many years, and with the exception of s6/66 we wouldn't replace it even if it meant keeping the last compiled version unchanged.

debian & void-linux also have patches, I had prefer to using their patches (especially debian) and fix things on top of debian and contribute back. But it seems gentoo user don't agree all of the patches, so in this repo, I'll focus on runit itself, leave the distribution related things in distribution's place.

I'll try to setup a a physical system and using runit when have time. For my personal use, I use bash script, busybox init etc for some small system setup, it might be interesting to try runit too.

Since the original author is not up to the task of fixing the code so it compiles with gcc14 why not rename it runit2 (as in consolekit2) and start from 0 before you patch. So runit2 2.1.3 will be the first release

In either case many of us are grateful for keeping runit alive, in any way you choose.

clan commented 2 months ago

And to make the story about runsvchdir complete here is the snippet from my /etc/runit/1 which provides me a way to set "runlevel" in Grub, using kernel options, before booting the kernel:

  # Set runlevel to:
  # - single       if kernel has param: S or single
  # - RUNLEVELNAME if kernel has param: runlevel=RUNLEVELNAME
  # - default      if kernel has no params or unable to set requested runlevel
  grep -q '\(^\| \)\(S\|single\)\( \|$\)' /proc/cmdline && runlevel='single'
  runsvchdir ${runlevel:-default} || runsvchdir default

P.S. Looking at it now I doubt it actually supports "runlevel=something". Probably it was implemented but then was removed. Or maybe it works if kernel params became env vars somehow without extra code in /etc/runit/1. Anyway, it's not really important here and now. Single user mode and default works for sure, and I don't use anything else anyway.

Do you mean handle "runlevel=something" by runit itself? I doubt no, I prefer handle this in /etc/runit/1 for now, but I'll have a look at the commit history.

powerman commented 2 months ago

I'll try to setup a a physical system and using runit when have time.

You can use my power-misc/runit-scripts package from "powerman" overlay for ease setup. It installs these files:

# qlist runit-scripts
/etc/runit/1
/etc/runit/2
/etc/runit/3
/etc/runit/ctrlaltdel
/etc/runit/lib.sh
/etc/setfont

so there will be conflicts with openrc-specific runit (to avoid these you can also install sys-process/runit from my overlay too - it's same as yours except removed openrc).

Or you can just download these files from SRC_URI: https://powerman.name/download/Gentoo/runit-scripts-1.9.0.tgz.

Read/update /etc/runit/{1,3} before use, you'll need to setup few things first - hostname, networking, etc. Also you'll need to setup at least one agetty service in /etc/service to be able to login. A lot of service packages are also available in my overlay: https://github.com/powerman/powerman-overlay/tree/master/runit-service.

Try to boot by manually adding init=/sbin/runit-init to kernel params in grub (or whatever boot loader you use) to avoid breaking your default openrc/systemd/etc. boot.

Booting with runit is actually fun! :smile: Mentioned above /etc/runit/1 has just 250 lines and that's enough to boot most systems which use plain filesystem (no encryption/raid/LVM). My actual /etc/runit/1 differs mostly by extra 90 lines for complex networking setup.

powerman commented 2 months ago

Do you mean handle "runlevel=something" by runit itself? I doubt no, I prefer handle this in /etc/runit/1 for now, but I'll have a look at the commit history.

I mean this code snippet looks like it expects $runlevel to be already set from kernel option. As there is no code which does this in /etc/runit/1 I suppose $runlevel won't be set before that snippet at all. But there is a small chance it was set by either kernel or runit itself.

clan commented 2 months ago

Do you mean handle "runlevel=something" by runit itself? I doubt no, I prefer handle this in /etc/runit/1 for now, but I'll have a look at the commit history.

I mean this code snippet looks like it expects $runlevel to be already set from kernel option. As there is no code which does this in /etc/runit/1 I suppose $runlevel won't be set before that snippet at all. But there is a small chance it was set by either kernel or runit itself.

After check source code of runsvchdir, I think it is almost the same as 'ln -fs /etc/runit/runsvdir/${runlevel} /etc/runit/runsvdir/current" where runleve is determined from kernel cmdline or by /etc/runit/1 (such as if system can't boot then switch to emergency mode for recovery)?

I've a repo (https://github.com/clan/rescue) to build a initramfs img which find and mount the rootfs then switch to the found rootfs. Part of the code can be reused in /etc/runit/1

For my usage in docker, I did a bootstrap (similar to /etc/runit/1) before run runsvdir (/etc/runit/2):

#!/bin/bash

set -o errexit
set -o pipefail

type -t log > /dev/null 2>&1 || {
        log() { : ; }
        declare -f log
}

for f in /etc/runit/bootstrap.*; do
        [ ${f} == "/etc/runit/bootstrap.sh" ] && continue
        if [ ! -s "${f}" ] ; then
                log "${f} not exist or size is 0"
                continue
        fi
        source ${f}
done

In case of Gentoo, as William Hubbs said in https://bugs.gentoo.org/611846, I agree the idea "separate the boot scripts into their own package" which can solve your current problem?

At least, I think we can rewrite /etc/runit/1 ( 2, 3 too) by the style of shell above, i.e., don't do real things (unless it's essential), just provide a mechanism to execute user defined script (i.e, add some hook to do user's customization task). Some common usage (such as setup udev, find & mount filesystem) can be provided as a template, like what openrc did at sysinit & boot level. Your overlay of services can be merged into services templates running in stage 2. It's users' job to select which services to be enabled (create a symlink to template) or disabled (remove the symlink).

For a more ambitious vision, we can add a global USE flag (runit such as) if enabled, the service will be provided by each service's ebuild. Of course, I'm sure this won't happen soon. If it become true one day, that mean runit become another core init system.

powerman commented 2 months ago

I think it is almost the same as 'ln -fs

Yeah, more or less. It's also useful not just to set "a runlevel" on boot, but also to replace a whole set of running services at any moment of time actually.

I agree the idea "separate the boot scripts into their own package" which can solve your current problem?

Yes, I think it's the best way forward. Just make it possible to install no boot scripts too as one of options, so users can provide their own custom package.

I think we can rewrite /etc/runit/1 ( 2, 3 too) by the style of shell above, i.e., don't do real things (unless it's essential), just provide a mechanism to execute user defined script

To me it looks like over engineering without any real value. Runit is simple, and /etc/runit/1 also can be simple. If 250 lines can boot most systems then it's unlikely any needs in forcing some more complicated structure than just one shell script. One small script is much easier to review and update. People who dislike this idea usually prefer to boot their systems using openrc or systemd etc. just because they think "one small script in not maintainable, we must be able to modify it behaviour by installing some other packages". Other people who use runit to boot usually think other way: one small script is much easier for maintain, especially because no other packages may affect boot in some uncontrollable way. :smile: They're all correct, but as we're supporting runit then we shouldn't try to turn it into openrc-like thing. At least - unconditionally. If one of "optional boot scripts packages" will provide this style of managing boot scripts - it's okay to me, maybe it'll find it users too. But, honestly, before spending time on this I recommend you to at least make some survey in runit maillist, asking about how real runit users boot their systems - do they use just one shell script (maybe with a small helpers like mine lib.sh) or they split into multiple scripts executed in a way like yours bootstrap* files or they do something else…

Some common usage (such as setup udev, find & mount filesystem) can be provided as a template, like what openrc did at sysinit & boot level. Your overlay of services can be merged into services templates running in stage 2.

Let's go in small steps, one by one. Maybe we'll go into this, and maybe it'll be useful, but it's a lot of work. It's easy to make a 250 lines script which boot someone's system, but it's much more hard to make it boot anyone's system and chances are it won't be as small and easy to read and maintain this way. And if while trying to make it general we'll end up making another one openrc-like thing then we lose the whole point of runit and simplicity as result.

fungilife commented 2 months ago

I am trying to clear through some old patches used by void and artix and apply your set, I still get a couple of errors listed below.

https://github.com/void-linux/runit/pull/7/files https://gitea.artixlinux.org/packages/runit (5 patches)

I am unclear as to what 0001-0020 are and 0029, which fails to apply no matter what I do as it overlaps previous patches? I am using gcc 14.2.1

( ./compile tryuwtmp.c && cat uw_tmp.h1 >uw_tmp.h )
+ exec gcc -O2 -Wall -c tryuwtmpx.c
tryuwtmpx.c: In function ‘main’:
tryuwtmpx.c:8:14: error: invalid use of undefined type ‘struct futmpx’
    8 |   char *s =ut.ut_name;
      |              ^
tryuwtmpx.c: At top level:
tryuwtmpx.c:5:15: error: storage size of ‘ut’ isn’t known
    5 | struct futmpx ut;
      |               ^~
+ exec gcc -O2 -Wall -c tryuwtmp.c
rm -f tryuwtmp.o tryuwtmpx.o

I have managed to succesfully built and boot with 0001-0020 alone not 0029 but 29 either alone or along any others it fails.

clan commented 2 months ago

please see https://github.com/clan/runit/issues/1

this one is not a build error, it's part of a feature test to determine the contents of uw_tmp.h

BTW, what is 0029?

fungilife commented 2 months ago

0029-fix-ftbfs-with-gcc14.patch https://salsa.debian.org/debian/runit/-/raw/next/debian/patches/00

Is this useless now?

By the way, from my old patches the only one that seems to be valid and can comply with new patches is the following, can it be useful? :


Initialize all fields of sockaddr_in.
http://skarnet.org/cgi-bin/archive.cgi?2:mss:1163:201602:gpiglpbjdemlioaeabbn

--- runit-2.1.2/src/svlogd.c 2014-08-10 15:22:34.000000000 -0300
+++ runit-2.1.2/src/svlogd.c 2016-02-04 22:53:52.910965361 -0300
@@ -430,6 +430,7 @@
   ld->name =(char*)fn;
   ld->ppid =0;
   ld->match ='+';
+  ld->udpaddr.sin_family =AF_INET;
   ld->udpaddr.sin_port =0;
   ld->udponly =0;
   while (! stralloc_copys(&ld->prefix, "")) pause_nomem();
clan commented 2 months ago

0029-fix-ftbfs-with-gcc14.patch https://salsa.debian.org/debian/runit/-/raw/next/debian/patches/00

Is this useless now?

I think I've merged all the modification in.

By the way, from my old patches the only one that seems to be valid and can comply with new patches is the following, can it be useful? :

I think it's useful, but it seems current implementation only have problem on *BSD? I'll have a look at it when have time, thanks. In the future, maybe we need to support ipv6 too?

powerman commented 2 months ago

In the future, maybe we need to support ipv6 too?

This sounds reasonable, but it might not be easy to implement correctly. TBH I'd prefer to wait until either @g-pape implements this feature himself or we get real feature request from a couple of users who actually needs it and can't use IPv4 for some valid reason.

fungilife commented 1 month ago

https://git.disroot.org/joborun-pkg/jobcore/src/branch/main/runit

joborun on the other hand turns ipv6 off anywhere found "not our problem to solve". Why should an init and service supervision system need to get involved in networking? It can supervise software that provide networking as services/daemons .. why this push to do what systemd does? A motorcycle shouldn't need to make coffee.

I also think that Gerrit Pape is not really interested in doing anything more with runit, he has said for years "goals met, no bugs". s6-networking and dns tools have been kept separate from s6 s6-rc and independent. I think this concept is more appropriate than a swiss army butcher knife for cleaning nails and splitting wires.

From all the daemontool children runit has had the widest recognition, and for a good reason. My idea of a runit improvement would have been a rewrite using something like skalibs/execline instead of bash.

powerman commented 1 month ago

I'll try to setup a a physical system and using runit when have time.

Actually there is one more change needed to work in GUI with runit - replace sys-auth/elogind with version from my overlay "powerman" and add USE="cgroup-hybrid runit" for this package.

The ebuild diff is:

--- elogind-252.9-r2.ebuild 2024-09-21 17:15:02.000000000 +0300
+++ elogind-252.9-r2.ebuild 2024-09-22 17:23:45.818176967 +0300
@@ -21,7 +21,7 @@

 LICENSE="CC0-1.0 LGPL-2.1+ public-domain"
 SLOT="0"
-IUSE="+acl audit cgroup-hybrid debug doc +pam +policykit selinux test"
+IUSE="+acl audit cgroup-hybrid debug doc +pam +policykit runit selinux test"
 RESTRICT="!test? ( test )"

 BDEPEND="
@@ -114,7 +114,9 @@
        -Dman=auto
        -Dpolkit=$(usex policykit true false)
        -Dsmack=true
-       -Dcgroup-controller=openrc
+       $(use runit && echo -Dpoweroff-path=/sbin/elogind-poweroff)
+       $(use runit && echo -Dreboot-path=/sbin/elogind-reboot)
+       -Dcgroup-controller=$(usex runit none openrc)
        -Ddefault-hierarchy=${cgroupmode}
        -Ddefault-kill-user-processes=false
        -Dacl=$(usex acl true false)
@@ -137,6 +139,12 @@

    sed -e "s|@libdir@|$(get_libdir)|" "${FILESDIR}"/${PN}.conf.in > ${PN}.conf || die
    newconfd ${PN}.conf ${PN}
+
+   if use runit; then
+       exeinto /sbin
+       printf '#!/bin/sh\nexec /sbin/runit-init 0\n' | newexe - elogind-poweroff
+       printf '#!/bin/sh\nexec /sbin/runit-init 6\n' | newexe - elogind-reboot
+   fi
 }

 pkg_postinst() {

Not sure is we've a chance to add it to gentoo, see https://bugs.gentoo.org/723522

fungilife commented 1 month ago

elogind is gentoo specific and irrelevant to runit ... not a runit related issue but a gentoo choice.

fungilife commented 1 month ago

https://git.disroot.org/joborun-pkg/jobcore/src/branch/main/runit

powerman commented 1 month ago

It may make sense to release proposed ebuild changes together with update to runit-2.2.0.

clan commented 1 month ago

It may make sense to release proposed ebuild changes together with update to runit-2.2.0.

cool, I just know 2.2.0 is out, I'll adapt existed waiting (had been in waiting state for over 2 weeks) PR to 2.2.0.