Undefined references to osv's glibc layer

[fish4terrisa-MSDSM]

Compiling osv with gcc 13.2.1, I got the following errors:

undefined reference to `__isoc23_strtoul'
undefined reference to `__openat_2'

These errors are outputed with object files in libstdc++.a, and they are defined in recent glibc. Will we update the osv goibc layer and add these functions?

[wkozaczuk]

Unfortunately, this is something we have to keep fixing every time a new version of gcc tool chain is released.

Which Linux distribution and version is it? I remember that @nyh added support of gcc 13 in Fedora 38.

__openat2 should be easy to implement based on what I see here. Not sure about __isoc23_strtoul.

[fish4terrisa-MSDSM]

It's archlinux with gcc 13.2.1, with dependences installed by my self.(archlinux's newest boost library is also not compatible with osv, so i have to install boost 1.74 manually)

[fish4terrisa-MSDSM]

And, it's glibc 2.38

[wkozaczuk]

Is there a specific reason you need to build OSv on archlinux? You can always build OSv in a docker container running Ubuntu Fedora or other distribution supported by setup.py - see https://github.com/cloudius-systems/osv/tree/master/docker#docker-osv-builder. And then run it on archlinux.

[fish4terrisa-MSDSM]

Docker don't work well with openvz, and the disk space and cpu of my building vps is limited. And, my vps is also used as archlinux package building machine, so building on archlinux is very important for me. And, the old glibc layer may cause problems on running programs built with newer glibc, so it may be wiser to move to newer glibc as soon as possible

[fish4terrisa-MSDSM]

I have installed gcc-9 in my vps, maybe it won't cause porblems. However, I don't know how to make the build script to use g++-9 and gcc-9 as the compilers. Do you have any instructions?

[fish4terrisa-MSDSM]

I have installed gcc-9 in my vps, maybe it won't cause porblems. However, I don't know how to make the build script to use g++-9 and gcc-9 as the compilers. Do you have any instructions?

I found the solution, and it's compiling now, I will tell you the output after a few hours. Maybe you can tell the users the newest version of gcc which is compatible with osv in the Building-related docs?

[fish4terrisa-MSDSM]

I have installed gcc-9 in my vps, maybe it won't cause porblems. However, I don't know how to make the build script to use g++-9 and gcc-9 as the compilers. Do you have any instructions?

This time, it succeeded.It seems that gcc-9 is compatible with osv.

[nyh]

Compiling osv with gcc 13.2.1, I got the following errors:

undefined reference to `__isoc23_strtoul'
undefined reference to `__openat_2'

These errors are outputed with object files in libstdc++.a, and they are defined in recent glibc. Will we update the osv goibc layer and add these functions?

As @wkozaczuk noted, every time a new version of the compiler or libraries come out, I find myself fixing things in OSv to fix the build :-( Specifically for gcc 13.2.1 I actually fixed the build, so probably the difference comes from some other difference in your system.

Looking on what __openat_2 is, it appears to be related to the "fortify source" option, which I guess your system libstdc++ builds with, and fedora doesn't.

The situation of __isoc23_strtoul is stranger. If you get it while building the OSv kernel, not your application, I think there's some problem in the header files - where the build used your system's header files instead of the the ones from OSv's include directory. But perhaps a more like possibility is that your libstdc++ was built using that isoc23 (i.e., ISO C 23) stuff. Unfortunately we may need to implement these functions, but since I never had this problem myself, I never did and I don't know yet what's missing and why.

I should upgrade my Fedora, maybe I'll start getting these problems myself and fix them...

[wkozaczuk]

One more alternative you may try is running your application with Linux glibc (taken from your archlinux host) instead of the one OSv is built with and exposes by default. This should give you the most compatibility as the app and glibc would interact with OSv through the syscall interface instead of the local libc function call.

To give you some context, traditionally OSv would dynamically link an app with its own libc it comes. The OSv libc is based on musl adjusted where necessary to be compatible with glibc but also in part comes with its own implementation like pthreads for example. Unfortunately, we do not really track in any way which glibc version we comply with and it would not make much sense given OSv libc is only a subset of glibc that is large enough to run most apps. And sometimes one can experience incompatibility issues described here - #821

Fairly recently, we have enhanced OSv to be able to run statically linked executables or dynamically linked ones with Linux dynamic linker. This feature is so new that we have not updated the docs yet and capstan does not support it yet either. For some details on how to run your app this way please look at this commit - f0414b64a1f760c094b0503d5b334de4dcad55bf and this one 58d0565a01276f702ea54852eb9c00f2f65948fc on how to build the image.

There is also a way to build OSv kernel with most symbols and stdc++ and libc hidden. I hope this wiki should give you an idea - https://github.com/cloudius-systems/osv/wiki/Modularization#app-specific-kernel.

My hope is, that when you build OSV kernel with libc and stdc++ hidden, those 2 symbols will not be needed.

[nyh]

I'm trying to build OSv on Fedora 39, and after fixing https://github.com/cloudius-systems/osv/issues/1301, indeed the next problem is in the linking, indeed __isoc23_strtoul is needed by the new libstdc++:

LINK loader.elf
ld.bfd: /usr/lib/gcc/x86_64-redhat-linux/13/libstdc++.a(eh_alloc.o): in function `_GLOBAL__sub_I_eh_alloc.cc':
(.text.startup._GLOBAL__sub_I_eh_alloc.cc+0x1a9): undefined reference to `__isoc23_strtoul'
ld.bfd: /usr/lib/gcc/x86_64-redhat-linux/13/libstdc++.a(debug.o): in function `__gnu_debug::_Error_formatter::_M_error() const':
(.text._ZNK11__gnu_debug16_Error_formatter8_M_errorEv+0x50): undefined reference to `__isoc23_strtoul'

So I want to add this function.

[nyh]

According to https://gustedt.gitlabpages.inria.fr/c23-library/#strtol - and see also https://en.wikipedia.org/wiki/C23_(C_standard_revision) - the need for __isoc23_strtoul came when C23 decided that 50 years of tradition CAN be messed with, and strtoul should start accepting binary constants (0b...) . So glibc decided to have two functions, the original strtoul and the new __iso23_strtoul, and choose the right one based on the compiler version. I guess Fedora 39's libstdc++.a was compiled with the new C23.

As a quick-and-dirty workaround, we can alias __isoc23_strtoul to the existing strtoul. This will be ok for everything except a very new application that really wants to use the new C23 feature of binary constants, so eventually we really need to add the 0b thing :-(

[nyh]

The following patch adds a good-enough-for-me __isoc23_strtoul, as an alias, and OSv now works (I checked, as usual, the "rogue" application) on Fedora 39:

diff --git a/libc/aliases.ld b/libc/aliases.ld
index 5f3fc744..1695fb1e 100644
--- a/libc/aliases.ld
+++ b/libc/aliases.ld
@@ -76,3 +76,11 @@ _Exit = exit;

 __dn_expand = dn_expand;
 __sysconf = sysconf;
+
+/* glibc uses these functions as C23 versions of the existing functions,
+   which are supposed to also allow binary constants (0b...). So they
+   should be slightly different than the classic functions, but until
+   we implement that, lets at least support them as an alias - see
+   issue #1299.
+*/
+__isoc23_strtoul = strtoul;

[nyh]

In commit e5003602e487802ee6c6867031d6e41e90b3dfdd I implemented __isoc23_strtoul, and __isoc23_scanf, as aliases as I suggested above.

The default build on Fedora 39 now fails when running on __openat64_2 (note the 64, slightly different from the problem you originally had):

$ scripts/build
$ scripts/run.py       
OSv v0.57.0-145-gb5a9e82c
eth0: 192.168.122.15
Booted up in 162.39 ms
Cmdline: /cli
/usr/lib/libboost_filesystem.so.1.81.0: failed looking up symbol __openat64_2

[nyh]

__openat_2() added in commit cf20f21b6c0c04104e433cdbae846caa7d87fd72. I forgot the "Fixes" line so closing the issue manually.

[m-mueller678]

__isoc23_strtoll seems to be also missing

[nyh]

Yes :-( I added only the functions that I saw were needed in my tests, the code I tried to run didn't need __isoc23_strtoll. You can see the fix in https://github.com/cloudius-systems/osv/commit/e5003602e487802ee6c6867031d6e41e90b3dfdd - it's a trivial line in libc/aliases.ld which add a line like

__isoc23_strtoul = strtoul;

Patches that fix the problems you see on your system (or if you can find a "complete" list of these new functions) will be very welcome.