[rtsan] glibc 2.40 regressions in tests

[mgorny]

The following rtsan tests fail with glibc 2.40:

  RealtimeSanitizer-Unit :: ./Rtsan-x86_64-Test/RtsanFileTest/OpenDiesWhenRealtime
  RealtimeSanitizer-Unit :: ./Rtsan-x86_64-Test/RtsanFileTest/OpenatDiesWhenRealtime
  RealtimeSanitizer-Unit :: ./Rtsan-x86_64-Test/RtsanOpenedFileTest/PreadDiesWhenRealtime
  RealtimeSanitizer-Unit :: ./Rtsan-x86_64-Test/RtsanOpenedFileTest/ReadDiesWhenRealtime

If I apply #108057, the additional tests also fail:

  RealtimeSanitizer-Unit :: ./Rtsan-x86_64-FileOffset64-Test/RtsanFileTest/OpenDiesWhenRealtime                                
  RealtimeSanitizer-Unit :: ./Rtsan-x86_64-FileOffset64-Test/RtsanFileTest/OpenatDiesWhenRealtime
  RealtimeSanitizer-Unit :: ./Rtsan-x86_64-FileOffset64-Test/RtsanOpenedFileTest/PreadDiesWhenRealtime
  RealtimeSanitizer-Unit :: ./Rtsan-x86_64-FileOffset64-Test/RtsanOpenedFileTest/ReadDiesWhenRealtime

All failures follow the same pattern:

FAIL: RealtimeSanitizer-Unit :: ./Rtsan-x86_64-Test/12/19 (3380 of 7243)
******************** TEST 'RealtimeSanitizer-Unit :: ./Rtsan-x86_64-Test/12/19' FAILED ********************
Script(shard):
--
GTEST_OUTPUT=json:/var/tmp/portage/sys-libs/compiler-rt-sanitizers-20.0.0_pre20240910/work/compiler-rt_build/lib/rtsan/tests/./Rtsan-x86_64-Test-RealtimeSanitizer-Unit-877-12-19.json GTEST_SHUFFLE=0 GTEST_TOTAL_SHARDS=19 GTEST_SHARD_INDEX=12 /var/tmp/portage/sys-libs/compiler-rt-sanitizers-20.0.0_pre20240910/work/compiler-rt_build/lib/rtsan/tests/./Rtsan-x86_64-Test
--

Script:
--
/var/tmp/portage/sys-libs/compiler-rt-sanitizers-20.0.0_pre20240910/work/compiler-rt_build/lib/rtsan/tests/./Rtsan-x86_64-Test --gtest_filter=RtsanFileTest.OpenatDiesWhenRealtime
--
/var/tmp/portage/sys-libs/compiler-rt-sanitizers-20.0.0_pre20240910/work/compiler-rt/lib/rtsan/tests/rtsan_test_utilities.h:40: Failure
Death test: RealtimeInvoke(std::forward<Function>(Func))
    Result: failed to die.
 Error msg:
[  DEATH   ] 

/var/tmp/portage/sys-libs/compiler-rt-sanitizers-20.0.0_pre20240910/work/compiler-rt/lib/rtsan/tests/rtsan_test_utilities.h:40
Death test: RealtimeInvoke(std::forward<Function>(Func))
    Result: failed to die.
 Error msg:
[  DEATH   ] 

********************

[cjappl]

Thanks for filing - I've subscribed to this issue so I can follow it.

I will see if I can look into it, but I don't have a great dev environment to fix this. If anyone feels encouraged to put up a PR in the meantime I am happy to review it.

[mgorny]

CC @MaskRay by any chance?

[cjappl]

To add a little background, what is happening here is this means that we are not intercepting these functions properly.

Our unit tests do the following:

• Set up a lambda, which calls a call that should be in our interceptors list
• Invoke the lambda in a real-time context (where it should print a stack and die) ExpectRealtimeDeath
• Invoke the lambda in a normal context (where it should be fine) ExpectNonRealtimeSurvival

If you have a debugger, or time to look into it, you could step into the function we should be intercepting (open, openat, pread, read) and see what is really being called.

• Have the arguments changed in a subtle way?
• Has the name of the function been changed? (Similar to in _FILE_OFFSET_64, open64 is called instead of open

Any discrepancy between the interceptor we have defined in rtsan_interceptors and this real function you step into will result in the call not being intercepted, and our tests not dying.

If we can figure this out, I think it will be trivial to write the correct interceptors and tests and unblock you

[mgorny]

I'm guessing it's due to _FORTIFY_SOURCE:

4       open("foo", O_RDONLY);
(gdb) s
_Z4openPKcU17pass_object_size1i (__path=<optimized out>, __oflag=0) at /usr/include/bits/fcntl2.h:69
69    return __open_2 (__path, __oflag);

Perhaps 155b7a12820ec45095988b6aa6e057afaf2bc892 is relevant here?

[cjappl]

@nikic - any idea what could be going on here? I do remember some previous issues with _FORTIFY_SOURCE that were fixed by that patch.

[nikic]

Can't say offhand what goes wrong here. I see that the interceptors still incorrectly read the mode argument without checking oflag first. But that shouldn't manifest as a failure to intercept.

[cjappl]

Thanks for reminding me about that @nikic ! I put up a PR to fix that issue at #108291 . Still figuring out what to do about fcntl. I agree both of these are tangential to this issue though

[cjappl]

I was thinking about this today and trying to come to some solution.

I came across this article by @MaskRay https://maskray.me/blog/2022-11-06-fortify-source.

Specifically this quote stood out

When a sanitizer is used, generally _FORTIFY_SOURCE should be disabled since sanitizer runtime does not implement most *_chk functions. Using _FORTIFY_SOURCE will regress error checking (asan/hwasan/tsan) or cause false positives (msan).

Is it worthwhile to just skip these tests if _FORTIFY_SOURCE is enabled? XFail them? Ignore them? Can we change the code to ensure the offending checks are never run on fortified source?

[mgorny]

Hmm, I've just noticed that we're actually disabling _FORTIFY_SOURCE in Gentoo if ASAN or MSAN is enabled. I suppose we ought to extend the list to include RTSAN.