Could we also have Heirloom NG as an alternative to GNU Coreutils in the future?

[takusuman]

I was taking a look at https://github.com/dslm4515/CMLFS/tree/bsd-userland and thought that it would be interesting to also have Heirloom NG as an alternative to GNU Coreutils. These tools were derived from UNIX v7, 4BSD and OpenSolaris, while the majority were rewritten from scratch based on the payloads defined for original UNIX tools. I've been maintaining them in the last two years, and also added a new chroot implementation based on the Solaris 2.5 one. It's the main toolset at Copacabana Linux, so it was also patched to build on musl without any problems. I know we have many problems to solve before, but I just wanted to show it as an alternative.

[dslm4515]

Perhaps change line 56 of nawk/main.c from:

int main(int argc, unsigned char *argv[], unsigned char *envp[])

to :

int main(int argc, char *argv[], char *envp[])

[dslm4515]

@takusuman

Were you able to build it in the end?

Yes, But when installing there is an error:

make[1]: Leaving directory '/mnt/cmlfs/sources/heirloom-ng-230120/setpgrp'
make[1]: Entering directory '/mnt/cmlfs/sources/heirloom-ng-230120/shl'
u=`uname`; \
if test "$u" != FreeBSD && test "$u" != HP-UX && \
    test "$u" != AIX && test "$u" != NetBSD && \
    test "$u" != OpenBSD && test "$u" != DragonFly ; \
then \
    /tmp/BUILD/llvmtools/ucbbin/install -c -g utmp -m 2755 shl /tmp/BUILD/llvmtools/bin/shl &&\
    llvm-strip -s -R .comment -R .note /tmp/BUILD/llvmtools/bin/shl &&\
    /llvmtools/bin/sh ../build/maninst -c -m 644 shl.1 /tmp/BUILD/llvmtools/share/man/5man/man1/shl.1; \
else \
    exit 0; \
fi
install: chown: /tmp/BUILD/llvmtools/bin/shl: Operation not permitted
make[1]: *** [Makefile:93: install] Error 1
make[1]: Leaving directory '/mnt/cmlfs/sources/heirloom-ng-230120/shl'
make: *** [makefile:37: install] Error 2

If yes, which are the versions of the musl libc, Linux kernel and LLVM?

musl libc 1.2.3
Linux kernel 6.1.8
LLVM 15.0.6

[dslm4515]

using heirloom-ng as my core utils, i cannot configure M4:

checking whether make supports nested variables... yes
usage: rm [-fir] file ...
Oops!

Your 'rm' program seems unable to run without file operands specified
on the command line, even when the '-f' option is present.  This is contrary
to the behaviour of most rm programs out there, and not conforming with
the upcoming POSIX standard: <http://austingroupbugs.net/view.php?id=542>

Please tell bug-automake@gnu.org about your system, including the value
of your $PATH and any error possibly output before this message.  This
can help us improve future automake versions.

Aborting the configuration process, to ensure you take notice of the issue.

You can download and install GNU coreutils to get an 'rm' implementation
that behaves properly: <http://www.gnu.org/software/coreutils/>.

If you want to complete the configuration process using your problematic
'rm' anyway, export the environment variable ACCEPT_INFERIOR_RM_PROGRAM
to "yes", and re-run configure.

configure: error: Your 'rm' program is bad, sorry.

[takusuman]

Rebuilt cgnutools and llvmtools far enough to reach heirloom-ng. I made sure to copy the build/mk.config and just removed the comments for easy editing. Build still fails at nawk:

make[1]: Entering directory '/mnt/cmlfs/sources/heirloom-ng-230120/nawk'
byacc -d awk.g.y
byacc: 46 shift/reduce conflicts, 178 reduce/reduce conflicts.
mv -f y.tab.c awk.g.c
(echo '1i'; echo '#include <inttypes.h>'; echo '.'; echo 'w';) | \
  ed -s y.tab.h
cc -Os -fomit-frame-pointer  -D_GNU_SOURCE -DUSE_TERMCAP  -D_FILE_OFFSET_BITS=64L  -I../libcommon -I../libuxre -DUXRE -c awk.g.c
flex  -t awk.lx.l > awk.lx.c
cc -Os -fomit-frame-pointer  -D_GNU_SOURCE -DUSE_TERMCAP  -D_FILE_OFFSET_BITS=64L  -I../libcommon -I../libuxre -DUXRE -c awk.lx.c
awk.lx.l:277:9: warning: using the result of an assignment as a condition without parentheses [-Wparentheses]
                if (c = *lexprog & 0377)
                    ~~^~~~~~~~~~~~~~~~~
awk.lx.l:277:9: note: place parentheses around the assignment to silence this warning
                if (c = *lexprog & 0377)
                      ^
                    (                  )
awk.lx.l:277:9: note: use '==' to turn this assignment into an equality comparison
                if (c = *lexprog & 0377)
                      ^
                      ==
1 warning generated.
cc -Os -fomit-frame-pointer  -D_GNU_SOURCE -DUSE_TERMCAP  -D_FILE_OFFSET_BITS=64L  -I../libcommon -I../libuxre -DUXRE -c b.c -std=c99
cc -Os -fomit-frame-pointer  -D_GNU_SOURCE -DUSE_TERMCAP  -D_FILE_OFFSET_BITS=64L  -I../libcommon -I../libuxre -DUXRE -c lib.c
cc -Os -fomit-frame-pointer  -D_GNU_SOURCE -DUSE_TERMCAP  -D_FILE_OFFSET_BITS=64L  -I../libcommon -I../libuxre -DUXRE -c main.c
main.c:56:5: error: second parameter of 'main' (argument array) must be of type 'char **'
int main(int argc, unsigned char *argv[], unsigned char *envp[])
    ^
main.c:56:5: error: third parameter of 'main' (environment) must be of type 'char **'
2 errors generated.
make[1]: *** [Makefile:136: main.o] Error 1
make[1]: Leaving directory '/mnt/cmlfs/sources/heirloom-ng-230120/nawk'
make: *** [makefile:23: all] Error 2

I'm not worried about the warning, but there is a problem with nawk/main.c ?

That's weird, it didn't happened at GCC + musl. Maybe it's a standards thing? Try using -std=c99 or something in nawk's CFLAGS at build/mk.config.

[takusuman]

@takusuman

Were you able to build it in the end?

Yes, But when installing there is an error:

make[1]: Leaving directory '/mnt/cmlfs/sources/heirloom-ng-230120/setpgrp'
make[1]: Entering directory '/mnt/cmlfs/sources/heirloom-ng-230120/shl'
u=`uname`; \
if test "$u" != FreeBSD && test "$u" != HP-UX && \
  test "$u" != AIX && test "$u" != NetBSD && \
  test "$u" != OpenBSD && test "$u" != DragonFly ; \
then \
  /tmp/BUILD/llvmtools/ucbbin/install -c -g utmp -m 2755 shl /tmp/BUILD/llvmtools/bin/shl &&\
  llvm-strip -s -R .comment -R .note /tmp/BUILD/llvmtools/bin/shl &&\
  /llvmtools/bin/sh ../build/maninst -c -m 644 shl.1 /tmp/BUILD/llvmtools/share/man/5man/man1/shl.1; \
else \
  exit 0; \
fi
install: chown: /tmp/BUILD/llvmtools/bin/shl: Operation not permitted
make[1]: *** [Makefile:93: install] Error 1
make[1]: Leaving directory '/mnt/cmlfs/sources/heirloom-ng-230120/shl'
make: *** [makefile:37: install] Error 2

Well, this isn't a build issue, thankfully. It's related to permissions. For some reason, for chown'ng these files, you need to be root. It's not anything really worrying to be honest, but I need to fix this later.

[takusuman]

using heirloom-ng as my core utils, i cannot configure M4:

checking whether make supports nested variables... yes
usage: rm [-fir] file ...
Oops!

Your 'rm' program seems unable to run without file operands specified
on the command line, even when the '-f' option is present.  This is contrary
to the behaviour of most rm programs out there, and not conforming with
the upcoming POSIX standard: <http://austingroupbugs.net/view.php?id=542>

Please tell bug-automake@gnu.org about your system, including the value
of your $PATH and any error possibly output before this message.  This
can help us improve future automake versions.

Aborting the configuration process, to ensure you take notice of the issue.

You can download and install GNU coreutils to get an 'rm' implementation
that behaves properly: <http://www.gnu.org/software/coreutils/>.

If you want to complete the configuration process using your problematic
'rm' anyway, export the environment variable ACCEPT_INFERIOR_RM_PROGRAM
to "yes", and re-run configure.

configure: error: Your 'rm' program is bad, sorry.

This was already patched in the code itself (see https://github.com/Projeto-Pindorama/heirloom-ng/commit/4d934377fc030c04a2ecf513a2fa14966fcc6982 and https://github.com/Projeto-Pindorama/heirloom-ng/commit/ef33ea2caa6cae165ece1f4ac785767a074a6ed3). Ironically, the cause for we having this error in the toolchain is Heirloom NG's "correctness" about UNIX standards. Please, let me explain: the default rm(1) command is updated to match the new POSIX standard (at least, partially), but the SUS variant (defined via -DSUS) isn't. To fix it, just disable the SUS variant install, by setting SUSBIN and SU3BIN as "dummy" directories.

[dslm4515]

Well, this isn't a build issue, thankfully. It's related to permissions.

Yeah, just like elftoolchainwith it's funky bmake build system. It won't let me install unless I am root... so it had to be built under chroot

[takusuman]

Well, this isn't a build issue, thankfully. It's related to permissions.

Yeah, just like elftoolchainwith it's funky bmake build system. It won't let me install unless I am root... so it had to be built under chroot

In Heirloom's case, it will install just fine, but it won't change permissions for the root user. So I think it will install just fine on CMLFS' toolchain.

[takusuman]

Just a small update: now Heirloom NG also has readlink, from BSD: https://github.com/Projeto-Pindorama/heirloom-ng/pull/19

[dslm4515]

I got around building CMLFS on i686 on an old laptop. Once again, GNU coreuntils fails to configure. Its getting annoying how unpredictable coreutils cab be. Then I read about why Chimera Linux does not use coreutils:

While coreutils may seem lightweight enough to not cause any issues already, there are some specific reasons the system uses a BSD-derived userland. The primary one is probably that the code of the BSD versions is overall much cleaner and easier to read. There are no cursed components such as gnulib, the codebase is leaner, and more aligned with the project’s goals.

So as long has heirloom-ng has utilities that accept the same arguments as the GNU counterparts, I am happy to drop GNU's coreutils in favor of Heirloom-ng or Chimerautils.

Alternatively, I could just fork GNU coreutils and clean up the code... But that too time consuming...

I'm planning to finish llvm-15.0.6 branch with dropping GNU's coreutils for heirloom-ng & Chimerautils.

[takusuman]

So as long has heirloom-ng has utilities that accept the same arguments as the GNU counterparts, I am happy to drop GNU's coreutils in favor of Heirloom-ng or Chimerautils.

For sure! You can compare, for example, Heirloom NG's seq(1) code with GNU's one and see per yourself which is more readable/hackable. Of course these newly implemented utilities on Heirloom NG still lacking some flags/functions that original GNU ones have, but it works thoroughly well for an UNIX-compatible system. You can also read the website if it interests you, it has some further information on the project.

[takusuman]

Looks like Heirloom NG has less dependencies

I mean, it has a lot less. Basically you will just need a C library and (n)curses. For the sake of justice with Chimera's utilities, I have to mention the fact that Heirloom NG utilities are fairly smaller and some even came back from UNIX v7 (with patches, of course). It's really good if you doesn't have space for bigger utilities and/or you want a real UNIX environment, but you may miss some "sugar" that GNU gives (and Chimera re-implements).

[dslm4515]

I tried compiling coreutils on a different cpu arch (amd64 vs i686) with same package versions... it won't configure under i686. I'm not sure if the host has too old of GNU autotools... I do hate how unpredictable it is.