Method dispatch failures in gnustep-base with -fobjc-runtime=gnustep-2.0

[ngrewe]

I'm currently seeing problems with message dispatch when compiling gnustep-base using the 2.0 runtime ABI. This test program, compiled under Ubuntu 18.04 with clang-9, current libobjc master (8249036) and gnustep-base master (7d1a5cceb), fails with a stack overflow due to a loop created by failing to raise an NSInvalidArgumentException during message forwarding.

#import <Foundation/NSString.h>

int main() {
  @autoreleasepool {
        [NSString stringWithFormat: @"foo %d", 42]; 
  }
}

This is the start of the backtrace:

#41653 0x00007f9fe0a38787 in +[NSException raise:format:] (self=0x7f9fe0f7fa38 <._OBJC_CLASS_NSException>, _cmd=0x7f9fe0febb08 <.objc_selector_raise:format:_v320:81624>, name=0x7f9fe0fda438 <.objc_str_NSInvalidArgumentException>, format=0x7f9fe0fdea78)
    at NSException.m:1448
#41654 0x00007f9fe0a9c762 in -[NSObject doesNotRecognizeSelector:] (self=0x7f9fe0fa7170 <._OBJC_CLASS_NSString>, _cmd=0x7f9fe0ff5048 <.objc_selector_doesNotRecognizeSelector:_v240:8:16>, 
    aSelector=0x7f9fe0ff1b88 <.objc_selector_stringWithFormat:arguments:_320:816[1{__va_list_tag=II^v^v}]24>) at NSObject.m:1738
#41655 0x00007f9fe0a9c825 in -[NSObject forwardInvocation:] (self=0x7f9fe0fa7170 <._OBJC_CLASS_NSString>, _cmd=0x7f9fe0ff1768 <.objc_selector_forwardInvocation:_v240:816>, anInvocation=0x14ecda8) at NSObject.m:1759
#41656 0x00007f9fe0b973df in GSFFIInvocationCallback (cif=0x14ecc50, retp=0x7fff34dac1f0, args=0x7fff34dac050, user=0x14ecc08) at GSFFIInvocation.m:606
#41657 0x00007f9fde9cfb4f in ffi_closure_unix64_inner () from /usr/lib/x86_64-linux-gnu/libffi.so.6
#41658 0x00007f9fde9cff16 in ffi_closure_unix64 () from /usr/lib/x86_64-linux-gnu/libffi.so.6
#41659 0x00007f9fe0a387cd in +[NSException raise:format:arguments:] (self=0x7f9fe0f7fa38 <._OBJC_CLASS_NSException>, _cmd=0x7f9fe0fee478 <.objc_selector_raise:format:arguments:_v400:81624[1{__va_list_tag=II^v^v}]32>, 
    name=0x7f9fe0fda438 <.objc_str_NSInvalidArgumentException>, format=0x7f9fe0fdea78, argList=0x7fff34dac3d0) at NSException.m:1461
#41660 0x00007f9fe0a38787 in +[NSException raise:format:] (self=0x7f9fe0f7fa38 <._OBJC_CLASS_NSException>, _cmd=0x7f9fe0febb08 <.objc_selector_raise:format:_v320:81624>, name=0x7f9fe0fda438 <.objc_str_NSInvalidArgumentException>, format=0x7f9fe0fdea78)
    at NSException.m:1448
#41661 0x00007f9fe0a9c762 in -[NSObject doesNotRecognizeSelector:] (self=0x7f9fe0f93ff8 <._OBJC_CLASS_NSObject>, _cmd=0x7f9fe0ff5048 <.objc_selector_doesNotRecognizeSelector:_v240:8:16>, aSelector=0x7f9fe0fec098 <.objc_selector_leakAt:_240:8^16>) at NSObject.m:1738
#41662 0x00007f9fe0a9c825 in -[NSObject forwardInvocation:] (self=0x7f9fe0f93ff8 <._OBJC_CLASS_NSObject>, _cmd=0x7f9fe0ff1768 <.objc_selector_forwardInvocation:_v240:816>, anInvocation=0x14e8548) at NSObject.m:1759
#41663 0x00007f9fe0b973df in GSFFIInvocationCallback (cif=0x14da2c0, retp=0x7fff34dac770, args=0x7fff34dac5d0, user=0x14da278) at GSFFIInvocation.m:606
#41664 0x00007f9fde9cfb4f in ffi_closure_unix64_inner () from /usr/lib/x86_64-linux-gnu/libffi.so.6
#41665 0x00007f9fde9cff16 in ffi_closure_unix64 () from /usr/lib/x86_64-linux-gnu/libffi.so.6
#41666 0x00007f9fe09d7c69 in +[NSCharacterSet _staticSet:length:number:] (self=0x7f9fe0f748d8 <._OBJC_CLASS_NSCharacterSet>, _cmd=0x7f9fe0fef7c8 <.objc_selector__staticSet:length:number:_320:8r^v16I24i28>, bytes=0x7f9fe0c83110 <nonBaseCharSet>, length=122880, 
    number=7) at NSCharacterSet.m:681
#41667 0x00007f9fe09d7f38 in +[NSCharacterSet nonBaseCharacterSet] (self=0x7f9fe0f748d8 <._OBJC_CLASS_NSCharacterSet>, _cmd=0x7f9fe0fef878 <.objc_selector_nonBaseCharacterSet_160:8>) at NSCharacterSet.m:753
#41668 0x00007f9fe0b09479 in +[NSString initialize] (self=0x7f9fe0fa7170 <._OBJC_CLASS_NSString>, _cmd=0x141bb10) at NSString.m:827
#41669 0x00007f9fe0377286 in objc_send_initialize () from /usr/local/lib/libobjc.so.4.6
#41670 0x00007f9fe0381e96 in slowMsgLookup () from /usr/local/lib/libobjc.so.4.6
#41671 0x00007f9fe0387670 in objc_msgSend_fpret () from /usr/local/lib/libobjc.so.4.6
#41672 0x00007f9fe0a9bc46 in +[NSObject initialize] (self=0x7f9fe0f93ff8 <._OBJC_CLASS_NSObject>, _cmd=0x141bb10) at NSObject.m:1095
#41673 0x00007f9fe0377286 in objc_send_initialize () from /usr/local/lib/libobjc.so.4.6
#41674 0x00007f9fe0376efd in objc_send_initialize () from /usr/local/lib/libobjc.so.4.6
#41675 0x00007f9fe0381e96 in slowMsgLookup () from /usr/local/lib/libobjc.so.4.6
#41676 0x00007f9fe0387670 in objc_msgSend_fpret () from /usr/local/lib/libobjc.so.4.6
#41677 0x000000000040089b in main () at test.m:5

The first interesting stack frame is 41666:

#41666 0x00007f9fe09d7c69 in +[NSCharacterSet _staticSet:length:number:] (self=0x7f9fe0f748d8 <._OBJC_CLASS_NSCharacterSet>, _cmd=0x7f9fe0fef7c8 <.objc_selector__staticSet:length:number:_320:8r^v16I24i28>, bytes=0x7f9fe0c83110 <nonBaseCharSet>, length=122880, 
    number=7) at NSCharacterSet.m:681
681           [[NSObject leakAt: &cache_set[number]] release];

+leakAt: is defined in a category (cf. Additions/NSObject+GNUstepBase.m), so it shouldn't trigger second-chance dispatch.

Things get interesting again in frame 41659. At this point, base has decided that it cannot forward the message, and tries to raise the NSInvalidArgumentException, which then fails itself because [NSString stringWithFormat:arguments:] cannot be resolved:

#41659 0x00007f9fe0a387cd in +[NSException raise:format:arguments:] (self=0x7f9fe0f7fa38 <._OBJC_CLASS_NSException>, _cmd=0x7f9fe0fee478 <.objc_selector_raise:format:arguments:_v400:81624[1{__va_list_tag=II^v^v}]32>, 
    name=0x7f9fe0fda438 <.objc_str_NSInvalidArgumentException>, format=0x7f9fe0fdea78, argList=0x7fff34dac3d0) at NSException.m:1461
1461      reason = [NSString stringWithFormat: format arguments: argList];

Now +stringWithFormat:arguments: is also defined in a category (cf. Additions/NSString+GNUstepBase.m). So my working theory currently is that there is some problem either with class methods in categories, or with categories in different compilation units (we already have tests for instance methods in the same compilation unit).

I'll try and produce some reduced test cases for that soon.

[davidchisnall]

Is this being linked with BFD ld? It looks like it's a known bug where ld.bfd -r discards the sections too early and you end up with the categories not existing in the final binary. Linking with Gold or LLD should fix it.

[triplef]

Ah right, I think we ran into some similar issues with GNUstep on Android that were fixed by using the Gold linker.

Is there a way we can modify GNUstep make to always use the Gold or LLD linker with the 2.0 runtime so no one has to run into this again?

[plaurent]

I was able to compile and run this program with no issue under Ubuntu 19.04 using clang-8, and ~the default ld which happens to be symlinked to ld.bfd~ correction: using ld.gold. The build script I used was this one: https://github.com/plaurent/gnustep-build/blob/master/ubuntu-19.04-clang-8.0-runtime-2.0/GNUstep-buildon-ubuntu1904.sh

It would be interesting to try to replicate your setup.

Did you compile clang-9 from scratch, or did you get it from a repository? Did you have the following set: export RUNTIME_VERSION=gnustep-2.0 ?

[plaurent]

Scratch that comment about LD being ld.bfd -- was actually using ld.gold

[ngrewe]

Turns out to be the linker issue… sorry for the noise!

Is there a way we can modify GNUstep make to always use the Gold or LLD linker with the 2.0 runtime so no one has to run into this again?

I'll try to put together an autoconf check for this issue.

[davidchisnall]

@ngrewe, either gold or lld works, though there are a few things that trigger issues with LLD (SOGo, for example) and require GOLD. If you look in the FreeBSD ports tree for things that have gnustep on the USES line and are marked LLD_UNSAFE you can find some examples.

[ngrewe]

FYI: I've got a PR over at gnustep/tools-make#4 that partly addresses this. Unfortunately it turned out that you can't easily reproduce the ld.bfd problem in a conftest, but the least we can do is to provide the user with some indication that they might be using a linker that won't produce working binaries…