bebbo / libnix

libnix (v4): a C link library for AmigaOS/m68k
14 stars 9 forks source link

Failing to link against __INIT_LIST__ __EXIT_LIST__ #58

Closed mheyer32 closed 1 year ago

mheyer32 commented 1 year ago

DoomAttack fails to link with a recent version of the toolchain with:

m68k-amigaos-gcc /home/matze/DoomAttack/gnudoom/build/plugins/Chunky2Planar/Graffiti/startup.o /home/matze/DoomAttack/gnudoom/build/plugins/Chunky2Planar/Graffiti/funcs.o /home/matze/DoomAttack/gnudoom/build/plugins/Chunky2Planar/Graffiti/asmstuff.o /home/matze/DoomAttack/gnudoom/build/plugin_runtime.o -noixemul -g -ggdb -msmall-code -m68020-60 -nostartfiles -lstubs -o /home/matze/DoomAttack/gnudoom/build/plugins/Chunky2Planar/Graffiti/c2p_Graffiti
/home/matze/amigatoolchain/amiga-gcc-out/lib/gcc/m68k-amigaos/6.5.0b/../../../../m68k-amigaos/bin/ld: /home/matze/DoomAttack/gnudoom/build/plugin_runtime.o: in function `InitRuntime':
/home/matze/DoomAttack/gnudoom/plugin_runtime.c:57: undefined reference to `__INIT_LIST__'
/home/matze/amigatoolchain/amiga-gcc-out/lib/gcc/m68k-amigaos/6.5.0b/../../../../m68k-amigaos/bin/ld: /home/matze/DoomAttack/gnudoom/build/plugin_runtime.o: in function `CleanupRuntime':
/home/matze/DoomAttack/gnudoom/plugin_runtime.c:67: undefined reference to `__EXIT_LIST__'
collect2: error: ld returned 1 exit status
make: *** [Makefile:237: c2p_Graffiti] Error 1

The plugins in DoomAttack are a curious thing... I had to remind myself how that works. They get loaded at runtime via LoadSeq and compiled via -nostartfiles -lstubs. The plugins may use libnix functionality like printf; thus need some form of libnix initialization.

Back then I worked around the issue by providing my own InitRuntime() and CleanupRuntime() functions that emulate what the toolchain's runtime initialization for libs etc.:

https://github.com/mheyer32/DoomAttack/blob/master/gnudoom/plugin_runtime.c

There I refer to

extern const long __INIT_LIST__[];
extern const long __EXIT_LIST__[];

That doesn't seem to work anymore. Do you have some advice on how to do this better and get it working again?

bebbo commented 1 year ago

now you have to provide these by yourself and put that into the correct sections.

.section    .list___INIT_LIST__,"aw"
___INIT_LIST__:
        .long 0

.section    .list___EXIT_LIST__,"aw"
___EXIT_LIST__:
        .long 0

which can also be done in C with

__attribute__((section(".list___INIT_LIST__")))
long __INIT_LIST__[] = {0};
__attribute__((section(".list___EXIT_LIST__")))
long __EXIT_LIST__[] = {0};

That object file must be the first one. The linker collects the other entries into the same section. The value here is just a dummy and replaces the previously existing count.

Plus you need an exit object to append the a final 0.

__attribute__((section(".list___INIT_LIST__")))
long __INIT_LIST_END[] = {0};
__attribute__((section(".list___EXIT_LIST__")))
long __EXIT_LIST_END[] = {0};

Since elf doesn't support constructors and constructors are causing pita using sections per function,.. I've chosen this way.

You could achieve the same with an appropriate linker script, but that again causes hickups, e.g. kills the feature to insert jump tables if -smallcode is used.

mheyer32 commented 1 year ago

Is library initialization part of these init lists? In one particular case I can see it calling into __initstdio but I don't see it opening dos.library?

mheyer32 commented 1 year ago

Is this even correct? (Taken from disassembly)

000015f4 000015f4 <__INIT_LIST__>:
    15f4:   0000 0000       ori.b #0,d0

000015f8 000015f8 <__INIT_LIST_END>:
    15f8:   0000 0000       ori.b #0,d0         <<<<<<<<< shouldn't this be at the end?
    15fc:   0000 119a       ori.b #-102,d0
    1600:   0000 0062       ori.b #98,d0
    1604:   0000 1574       ori.b #116,d0
    1608:   0000 004e       ori.b #78,d0

0000160c 0000160c <__EXIT_LIST__>:
    160c:   0000 0000       ori.b #0,d0

00001610 00001610 <__EXIT_LIST_END>:
    1610:   0000 0000       ori.b #0,d0
    1614:   0000 1392       ori.b #-110,d0
    1618:   0000 0062       ori.b #98,d0
    161c:   0000 15b2       ori.b #-78,d0
    1620:   0000 004e       ori.b #78,d0

It looks as if the __INIT_LIST_END marker is put in as first element of the list, thus making the whole list invalid.

The object list is compiled such that the init and exit symbols are first and last, respectively: ${1}_SRC = plugin_init.c $$(call collect_sources,$$(${1}_DIR)) plugin_runtime.c plugin_exit.c

(the files are introduced in https://github.com/mheyer32/DoomAttack/commit/97bc1ebcb32db73e936c3a179b4017f10fd7f0b2 )

bebbo commented 1 year ago

what link spec is used?

mheyer32 commented 1 year ago

No own spec file is used, just 'regular' linking. This is an instance of a plugin linking:

m68k-amigaos-gcc -v -noixemul -g -ggdb -msmall-code -m68030 -nostartfiles /home/matze/DoomAttack/gnudoom/build//plugin_init.o /home/matze/DoomAttack/gnudoom/build/plugins/Sound_and_Music/DoomSound_Library/DAMusicStartup.o /home/matze/DoomAttack/gnudoom/build/plugins/Sound_and_Music/DoomSound_Library/stubs.o /home/matze/DoomAttack/gnudoom/build/plugins/Sound_and_Music/DoomSound_Library/funcs.o /home/matze/DoomAttack/gnudoom/build/plugin_runtime.o /home/matze/DoomAttack/gnudoom/build//plugin_exit.o -o /home/matze/DoomAttack/gnudoom/build//music_doomsound

bebbo commented 1 year ago

the exit file must be ordered behind -lstubs otherwise it isn't the last one.

mheyer32 commented 1 year ago

No matter how hard I try to link against the exit file last, it places the end marker at the front of the init list:

Using built-in specs.
COLLECT_GCC=m68k-amigaos-gcc
COLLECT_LTO_WRAPPER=/home/matze/amigatoolchain/amiga-gcc-out/libexec/gcc/m68k-amigaos/6.5.0b/lto-wrapper
Target: m68k-amigaos
Configured with: /home/matze/amigatoolchain/amiga-gcc/projects/gcc/configure --prefix=/home/matze/amigatoolchain/amiga-gcc-out --target=m68k-amigaos --enable-languages=c,c++,objc --enable-version-specific-runtime-libs --disable-libssp --disable-nls --with-headers=/home/matze/amigatoolchain/amiga-gcc/projects/newlib-cygwin/newlib/libc/sys/amigaos/include/ --disable-shared --enable-threads=no --with-stage1-ldflags='-dynamic-libgcc -dynamic-libstdc++' --with-boot-ldflags='-dynamic-libgcc -dynamic-libstdc++'
Thread model: single
gcc version 6.5.0b 221001103737 (GCC) 
COMPILER_PATH=/home/matze/amigatoolchain/amiga-gcc-out/m68k-amigaos/libnix/lib/:/home/matze/amigatoolchain/amiga-gcc-out/libexec/gcc/m68k-amigaos/6.5.0b/:/home/matze/amigatoolchain/amiga-gcc-out/libexec/gcc/:/home/matze/amigatoolchain/amiga-gcc-out/lib/gcc/m68k-amigaos/6.5.0b/../../../../m68k-amigaos/bin/
LIBRARY_PATH=/home/matze/amigatoolchain/amiga-gcc-out/m68k-amigaos/libnix/lib/:/home/matze/amigatoolchain/amiga-gcc-out/lib/gcc/m68k-amigaos/6.5.0b/:/home/matze/amigatoolchain/amiga-gcc-out/lib/gcc/:/home/matze/amigatoolchain/amiga-gcc-out/lib/gcc/m68k-amigaos/6.5.0b/../../../../m68k-amigaos/lib/
COLLECT_GCC_OPTIONS='-v' '-noixemul' '-g' '-ggdb' '-msmall-code' '-m68020-60' '-mtune=68030' '-nostartfiles' '-o' '/home/matze/DoomAttack/gnudoom/build/plugins/Sound_and_Music/DoomSound_Library/music_doomsound' '-B' '/home/matze/amigatoolchain/amiga-gcc-out/m68k-amigaos/libnix/lib/'
 /home/matze/amigatoolchain/amiga-gcc-out/libexec/gcc/m68k-amigaos/6.5.0b/collect2 -L/home/matze/amigatoolchain/amiga-gcc-out/lib -L/home/matze/amigatoolchain/amiga-gcc-out/m68k-amigaos/libnix/lib -amiga-debug-hunk -fl libm020 -o /home/matze/DoomAttack/gnudoom/build/plugins/Sound_and_Music/DoomSound_Library/music_doomsound -L/home/matze/amigatoolchain/amiga-gcc-out/m68k-amigaos/libnix/lib -L/home/matze/amigatoolchain/amiga-gcc-out/lib/gcc/m68k-amigaos/6.5.0b -L/home/matze/amigatoolchain/amiga-gcc-out/lib/gcc -ldebug /home/matze/DoomAttack/gnudoom/build/plugins/Sound_and_Music/DoomSound_Library/DAMusicStartup.o /home/matze/DoomAttack/gnudoom/build/plugin_init.o /home/matze/DoomAttack/gnudoom/build/plugins/Sound_and_Music/DoomSound_Library/stubs.o /home/matze/DoomAttack/gnudoom/build/plugins/Sound_and_Music/DoomSound_Library/funcs.o /home/matze/DoomAttack/gnudoom/build/plugin_runtime.o -lstubs /home/matze/DoomAttack/gnudoom/build/xplugin_exit.o -( -lnix20 -lnixmain -lnix -lstubs -lamiga -lgcc -)

leads to:

00001668 00001668 <__INIT_LIST__>:
    1668:   0000 0000       ori.b #0,d0

0000166c 0000166c <_term0__INIT_LIST>:
    166c:   0000 0000       ori.b #0,d0
    1670:   0000 120e       ori.b #14,d0
    1674:   0000 0062       ori.b #98,d0
    1678:   0000 15e8       ori.b #-24,d0
    167c:   0000 004e       ori.b #78,d0

00001680 00001680 <__EXIT_LIST__>:
    1680:   0000 0000       ori.b #0,d0

00001684 00001684 <_term0__EXIT_LIST>:
    1684:   0000 0000       ori.b #0,d0
    1688:   0000 1406       ori.b #6,d0
    168c:   0000 0062       ori.b #98,d0
    1690:   0000 1626       ori.b #38,d0
    1694:   0000 004e       ori.b #78,d0

See branch "fix_plugin_init" for my attempts to move it last https://github.com/mheyer32/DoomAttack/tree/fix_plugin_init

Try building the DoomSound plugin and observe the resulting object file:

make -B -j8 music_doomsound
m68k-amigaos-objdump -d -C -S  build/plugins/Sound_and_Music/DoomSound_Library/music_doomsound > plugin.txt
bebbo commented 1 year ago

guess you have to invoke m68k-amiagos-ld directly. Use -v to see what is generated from m68k-amigaos-gcc, copy that and move the endfile to the end

bebbo commented 1 year ago
define make_plugin
$$(eval $$(call build_rules,$$(BUILDDIR)))
${1}_DIR = ${2}
${1}_SRC = plugin_runtime.c $$(call collect_sources,$$(${1}_DIR)) plugin_exit.c
${1}_OBJS = $$(call create_objlist,$$(${1}_SRC), $$(BUILDDIR))
${1}_OBJS_STRIPPED = $$(patsubst %,%.stripped,$$(${1}_OBJS))
${1}_TARGET = $$(BINDIR)/${3}

${1} : CFLAGS+=-D__ASM_ROUTINES__ -noixemul
${1} : $$(${1}_OBJS)
    $$(MKDIR) $$(dir $$(${1}_TARGET))
    $$(LD) $$(filter-out %plugin_exit.o,$$^) -amiga-debug-hunk -fl libm020 \
    -L$$(PREFIX)/lib -L$$(PREFIX)/m68k-amigaos/libnix/lib -L$$(PREFIX)/m68k-amigaos/libnix/lib -L/$$(PREFIX)/lib/gcc/m68k-amigaos/6.5.0b -L$$(PREFIX)/lib/gcc \
    -o $$(<D)/${1} \
    "-(" -lnix20 -lnixmain -lnix -lstubs -lamiga -lgcc "-)" $$(BUILDDIR)/plugin_exit.o
    $$(STRIP)  $$(<D)/${1} -o $$(${1}_TARGET)
endef

don't forget to add LD = m68k-amigaos-ld ==>

...
00002c80 00002c80 ___INIT_LIST__:
    2c80:       0000 0000       ori.b #0,d0
    2c84:       0000 0dc6       ori.b #-58,d0
                        2c84: RELOC32   .text
    2c88:       0000 0062       ori.b #98,d0
    2c8c:       0000 1554       ori.b #84,d0
                        2c8c: RELOC32   .text
    2c90:       0000 004e       ori.b #78,d0

00002c94 00002c94 ___INIT_LIST_END:
    2c94:       0000 0000       ori.b #0,d0

00002c98 00002c98 ___EXIT_LIST__:
    2c98:       0000 0000       ori.b #0,d0
    2c9c:       0000 0fbe       ori.b #-66,d0
                        2c9c: RELOC32   .text
    2ca0:       0000 0062       ori.b #98,d0
    2ca4:       0000 1592       ori.b #-110,d0
                        2ca4: RELOC32   .text
    2ca8:       0000 004e       ori.b #78,d0

00002cac 00002cac ___EXIT_LIST_END:
    2cac:       0000 0000       ori.b #0,d0
...
mheyer32 commented 1 year ago

Well, that worked! As always, thank you very much!

bebbo commented 1 year ago

Since there will be further changes, I'm reopening this ticket.

The good news: the endfile will be removed

bebbo commented 1 year ago

use this code in plugin_runtime.c:

__attribute__((section(".list___INIT_LIST__")))
long __INIT_LIST__[] = {0};
__attribute__((section(".list___EXIT_LIST__")))
long __EXIT_LIST__[] = {0};
__attribute__((section(".list___ZZZZ__")))
long __ZZZZ__[] = {0};

and the last list is also terminated with a zero. No plugin_exit.c needed. And you can go back to invoke gcc instead of ld

mheyer32 commented 1 year ago

Is this already assumed working? I updated the toolchain today. I then reverted my pluign-linking related changes and replaced the corresponding lines in plugin_runtime.c with the above. This is what objdump shows me now:

0000171c 0000171c <__INIT_LIST__>:
    171c:   0000 0000       ori.b #0,d0
    1720:   0000 12c2       ori.b #-62,d0
    1724:   0000 0062       ori.b #98,d0
    1728:   0000 169c       ori.b #-100,d0
    172c:   0000 004e       ori.b #78,d0

00001730 00001730 <__EXIT_LIST__>:
    1730:   0000 0000       ori.b #0,d0
    1734:   0000 14ba       ori.b #-70,d0
    1738:   0000 0062       ori.b #98,d0
    173c:   0000 16da       ori.b #-38,d0
    1740:   0000 004e       ori.b #78,d0
    0001744 00001744 <_EH_FRAME_BEGINS__>:
    1744:   0000 0004       ori.b #4,d0

I could be wrong, but it looks as if the endmarkers are missing.

bebbo commented 1 year ago

it's almost live^^ needed to work out some fixes. Plus I wrote some doc: https://github.com/bebbo/amiga-gcc/wiki/Constructors,-Destructors,-Managed-Lists

bebbo commented 1 year ago

it's live!

mheyer32 commented 1 year ago

Ok, I see what you've done here... the start-count of one list becomes the end marker of the previous one, with ZZZZ being the last list and thus the last endmarker.

DoomAttack compiles.

ADoom has issues, though with a debug build:

make -j8 DEBUG=1 -B doomsound_midi.library

GNU C11 (GCC) version 6.5.0b 221101094013 (m68k-amigaos)
    compiled by GNU C version 9.4.0, GMP version 6.2.0, MPFR version 4.0.2, MPC version 1.1.0, isl version none
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
ignoring nonexistent directory "/home/matze/amigatoolchain/amiga-gcc-out/libexec/gcc/m68k-amigaos/lib/gcc/m68k-amigaos/6.5.0b/include"
ignoring nonexistent directory "/home/matze/amigatoolchain/amiga-gcc-out/libexec/gcc/m68k-amigaos/m68k-amigaos/ndk-include"
ignoring nonexistent directory "/home/matze/amigatoolchain/amiga-gcc-out/libexec/gcc/m68k-amigaos/m68k-amigaos/sys-include"
ignoring nonexistent directory "/home/matze/amigatoolchain/amiga-gcc-out/libexec/gcc/m68k-amigaos/m68k-amigaos/include"
#include "..." search starts here:
#include <...> search starts here:
 ./
 /home/matze/amigatoolchain/amiga-gcc-out/include
 /home/matze/amigatoolchain/amiga-gcc-out/m68k-amigaos/libnix/include
 /home/matze/amigatoolchain/amiga-gcc-out/lib/gcc/m68k-amigaos/6.5.0b/include
 /home/matze/amigatoolchain/amiga-gcc-out/m68k-amigaos/ndk-include
 /home/matze/amigatoolchain/amiga-gcc-out/m68k-amigaos/sys-include
 /home/matze/amigatoolchain/amiga-gcc-out/m68k-amigaos/include
End of search list.
GNU C11 (GCC) version 6.5.0b 221101094013 (m68k-amigaos)
    compiled by GNU C version 9.4.0, GMP version 6.2.0, MPFR version 4.0.2, MPC version 1.1.0, isl version none
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: 8cf4b688a66e593cf766f96818a0fdb3
COLLECT_GCC_OPTIONS='-noixemul' '-D' 'USENOIXEMUL' '-m68020-60' '-mtune=68030' '-msmall-code' '-mregparm=4' '-Werror' '-Wimplicit' '-Wstrict-prototypes' '-Wdouble-promotion' '-fstrict-aliasing' '-g' '-ggdb' '-D' 'DEBUG' '-Og' '-ffast-math' '-fno-omit-frame-pointer' '-v' '-I' './' '-ramiga-lib' '-fbaserel' '-o' 'doomsound_midi.library' '-B' '/home/matze/amigatoolchain/amiga-gcc-out/m68k-amigaos/libnix/lib/'
 /home/matze/amigatoolchain/amiga-gcc-out/lib/gcc/m68k-amigaos/6.5.0b/../../../../m68k-amigaos/bin/as --traditional-format -m68020 -sc -o /tmp/ccrKkt9o.o /tmp/cc2bXavq.s
COMPILER_PATH=/home/matze/amigatoolchain/amiga-gcc-out/m68k-amigaos/libnix/lib/:/home/matze/amigatoolchain/amiga-gcc-out/libexec/gcc/m68k-amigaos/6.5.0b/:/home/matze/amigatoolchain/amiga-gcc-out/libexec/gcc/:/home/matze/amigatoolchain/amiga-gcc-out/lib/gcc/m68k-amigaos/6.5.0b/../../../../m68k-amigaos/bin/
LIBRARY_PATH=/home/matze/amigatoolchain/amiga-gcc-out/m68k-amigaos/libnix/lib/libb/:/home/matze/amigatoolchain/amiga-gcc-out/lib/gcc/m68k-amigaos/6.5.0b/libb/:/home/matze/amigatoolchain/amiga-gcc-out/lib/gcc/m68k-amigaos/6.5.0b/../../../../m68k-amigaos/lib/libb/:/home/matze/amigatoolchain/amiga-gcc-out/m68k-amigaos/libnix/lib/:/home/matze/amigatoolchain/amiga-gcc-out/lib/gcc/m68k-amigaos/6.5.0b/:/home/matze/amigatoolchain/amiga-gcc-out/lib/gcc/:/home/matze/amigatoolchain/amiga-gcc-out/lib/gcc/m68k-amigaos/6.5.0b/../../../../m68k-amigaos/lib/
COLLECT_GCC_OPTIONS='-noixemul' '-D' 'USENOIXEMUL' '-m68020-60' '-mtune=68030' '-msmall-code' '-mregparm=4' '-Werror' '-Wimplicit' '-Wstrict-prototypes' '-Wdouble-promotion' '-fstrict-aliasing' '-g' '-ggdb' '-D' 'DEBUG' '-Og' '-ffast-math' '-fno-omit-frame-pointer' '-v' '-I' './' '-ramiga-lib' '-fbaserel' '-o' 'doomsound_midi.library' '-B' '/home/matze/amigatoolchain/amiga-gcc-out/m68k-amigaos/libnix/lib/'
 /home/matze/amigatoolchain/amiga-gcc-out/libexec/gcc/m68k-amigaos/6.5.0b/collect2 -L/home/matze/amigatoolchain/amiga-gcc-out/lib -L/home/matze/amigatoolchain/amiga-gcc-out/m68k-amigaos/libnix/lib -m amiga_bss -fl libb -amiga-debug-hunk -fl libm020 -o doomsound_midi.library /home/matze/amigatoolchain/amiga-gcc-out/m68k-amigaos/libnix/lib/libinit.o -L/home/matze/amigatoolchain/amiga-gcc-out/m68k-amigaos/libnix/lib -L/home/matze/amigatoolchain/amiga-gcc-out/lib/gcc/m68k-amigaos/6.5.0b -L/home/matze/amigatoolchain/amiga-gcc-out/lib/gcc /tmp/ccb6Nf9n.o /tmp/ccrKkt9o.o -( -lnix20 -lnixmain -lnix -lstubs -lamiga -lgcc -)
/home/matze/amigatoolchain/amiga-gcc-out/lib/gcc/m68k-amigaos/6.5.0b/../../../../m68k-amigaos/bin/ld: /home/matze/amigatoolchain/amiga-gcc-out/m68k-amigaos/libnix/lib/libb/libm020/libnix.a(abort.o): in function `abort':
abort.o:(.text+0x34): undefined reference to `exit'
/home/matze/amigatoolchain/amiga-gcc-out/lib/gcc/m68k-amigaos/6.5.0b/../../../../m68k-amigaos/bin/ld: /home/matze/amigatoolchain/amiga-gcc-out/m68k-amigaos/libnix/lib/libb/libm020/libnix.a(raise.o): in function `raise':
raise.o:(.text+0x84): undefined reference to `exit'
/home/matze/amigatoolchain/amiga-gcc-out/lib/gcc/m68k-amigaos/6.5.0b/../../../../m68k-amigaos/bin/ld: /home/matze/amigatoolchain/amiga-gcc-out/m68k-amigaos/libnix/lib/libb/libm020/libnix20.a(__initstdio.o): in function `__initstdio':
__initstdio.o:(.text+0x290): undefined reference to `_WBenchMsg'
/home/matze/amigatoolchain/amiga-gcc-out/lib/gcc/m68k-amigaos/6.5.0b/../../../../m68k-amigaos/bin/ld: __initstdio.o:(.text+0x346): undefined reference to `exit'
collect2: error: ld returned 1 exit status
make: *** [Makefile:213: doomsound_midi.library] Error 1
bebbo commented 1 year ago

__assert_func is not a smart choice for a library/plugin, since it pulls in the whole libnix library...

mheyer32 commented 1 year ago

Agreed, but that only affects the debug build of the library. What am I missing to get it linked, though?

bebbo commented 1 year ago

Agreed, but that only affects the debug build of the library. What am I missing to get it linked, though?

that's not only linking, you'd also had to call the init stuff... ... better provide an own assert_func using RawDoFormat or so...

musplay.c wants it:

00000000         *UND*      0002 83 ___assert_func
bebbo commented 1 year ago

__asert_func not writes to Output() using Write... ... might help

mheyer32 commented 1 year ago

Bebbo, I appreciate how you always go above and beyond!