cormacrelf
commented
10 months ago This is because there are two versions of libcxx in use. This exact problem also occurs when there are multiple versions of libcxx in use.
A simple way to see the problem is making a shell with llvmPackages_16.libcxx in it. As it turns out, llvmPackages_16.libcxx is different from the one in stdenv, and you get two different libcxx-es in $NIX_LDFLAGS. We also include jemalloc. Running clang++ hello.cxx -ljemalloc; ./a.out then segfaults. In fact it segfaults on exactly the same line of code as @a-n-n-a-l-e-e: this one, where encoding is a null function pointer and the code jumps to it.
I don't know how stdenv could include a libcxx other than the one in llvmPackages_XX, but I'm sure it's going to break a bunch more things than just jemalloc.
Using the current head of the release-23.11 branch (please note I am using aarch64, not x86_64 as OP was):
# shell.nix
let
nixpkgs = builtins.fetchTarball {
url = "https://github.com/NixOS/nixpkgs/archive/e4e2a842524fae32efac9649aeb1edfed7e7ae72.tar.gz";
sha256 = "18aqndvq9x5vhyqzrs167s0jilii41fy5bdbyjx1ar2l3gy5p26x";
};
pkgs = import nixpkgs { config = {}; };
in
pkgs.mkShell {
packages = [
pkgs.jemalloc
pkgs.llvmPackages_16.libcxx
pkgs.llvmPackages_16.libcxxabi
];
}$ nix-shell
[nix-shell:~/.../jemalloc-libc++-test]$ echo $NIX_LDFLAGS
# newlines added
-L/nix/store/1r9jhkzfz6q55qciyla3mz7nmh2w6cmz-jemalloc-5.3.0/lib
-L/nix/store/zq45ffwwcy2mw8mnnj14532gma3dsl05-libcxx-16.0.6/lib
-L/nix/store/1hr6xc1r3y6i9n6bkvci6a7kscv6cx8h-libcxxabi-16.0.6/lib
-L/nix/store/vf7k7l7ny35g1nmninaqjbryl2xqkag5-libcxx-16.0.6/lib
-L/nix/store/x622mzi1g7v7nh7zq5r8d0nxbhrg9px3-libcxxabi-16.0.6/lib
-L/nix/store/l92ls1vba62zsywdj81hlgmgx5adn6bn-compiler-rt-libc-16.0.6/lib
-L/nix/store/bj25mj5bpvbx03gqzf7hbpk6n7w4khr6-libobjc-11.0.0/lib
-L/nix/store/1r9jhkzfz6q55qciyla3mz7nmh2w6cmz-jemalloc-5.3.0/lib
-L/nix/store/zq45ffwwcy2mw8mnnj14532gma3dsl05-libcxx-16.0.6/lib
-L/nix/store/1hr6xc1r3y6i9n6bkvci6a7kscv6cx8h-libcxxabi-16.0.6/lib
-L/nix/store/vf7k7l7ny35g1nmninaqjbryl2xqkag5-libcxx-16.0.6/lib
-L/nix/store/x622mzi1g7v7nh7zq5r8d0nxbhrg9px3-libcxxabi-16.0.6/lib
-L/nix/store/l92ls1vba62zsywdj81hlgmgx5adn6bn-compiler-rt-libc-16.0.6/lib
-L/nix/store/bj25mj5bpvbx03gqzf7hbpk6n7w4khr6-libobjc-11.0.0/libNote:
- Duplicate libcxx
-L/nix/store/zq45ffwwcy2mw8mnnj14532gma3dsl05-libcxx-16.0.6/liband-L/nix/store/vf7k7l7ny35g1nmninaqjbryl2xqkag5-libcxx-16.0.6/lib(otool -lon libjemalloc shows it is using the former) - Duplicate libcxxabi
- Only one compiler-rt, only one libobjc
- libobjc is stuck on LLVM 11
There are a few differences in nix derivation show for each of the libcxxs. Maybe this will give some kind of clue. For the one jemalloc links (/nix/store/zq45ff...):
# newlines added
"nativeBuildInputs": "/nix/store/4xivbfq2sf0mcgql4j6ly93jwxjb2n31-cmake-3.27.7
/nix/store/k3s4yv7swx52vhpd7z5jhymn0ji5g2ni-ninja-1.11.1
/nix/store/7b0rz3bnx7msw2wawkv1hhn5lqf1b0wi-python3-3.11.6
/nix/store/0m03fp4cj5vhr34a8rvwr1y1y74fjbd7-fix-darwin-dylib-names-hook",For llvmPackages_16.libcxx (/nix/store/vf7k7l7...):
"nativeBuildInputs": "/nix/store/1l2hahnabzw0rmrmsxryxchf6lazghz9-cmake-minimal-3.27.7
/nix/store/azfh76jsad19zlxd1mh0b79lhl75x9rp-ninja-1.11.1
/nix/store/sxhmccj49jphf2n1nhybc9a4l9s4qkf0-python3-minimal-scproxy-3.11.6
/nix/store/rdd6cn1r40m1zd704d7lrm2abkkl5flm-fix-darwin-dylib-names-hook",And finally, what happens when you try to link a hello world C program with jemalloc in that shell:
$ echo 'int main() {}' > hello.cxx
$ clang++ hello.cxx -ljemalloc
$ ./a.out
Segmentation fault: 11
$ lldb ./a.out
...
warning: (arm64) .../jemalloc-libc++-test/a.out(0x0000000100000000) address 0x0000000100000000 maps to more than one section: a.out.__TEXT and a.out.__TEXT
Process 85326 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
frame #0: 0x0000000000000000
error: memory read failed for 0x0
Target 0: (a.out) stopped.
(lldb) bt
* frame #0: 0x0000000000000000
frame #1: 0x000000010046dbb8 libc++.1.0.dylib`std::__1::__stdinbuf<char>::imbue(std::__1::locale const&) + 52
frame #2: 0x000000010046d41c libc++.1.0.dylib`std::__1::DoIOSInit::DoIOSInit() + 148
frame #3: 0x000000010046e9d4 libc++.1.0.dylib`_GLOBAL__I_000100 + 72
frame #4: 0x000000019e8581d8 dyld`invocation function for block in dyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const::$_0::operator()() const + 168
...
frame #10: 0x000000019e899904 dyld`dyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const + 432
...
frame #16: 0x000000019e854d90 dyld`dyld4::Loader::runInitializersBottomUpPlusUpwardLinks(dyld4::RuntimeState&) const + 304
frame #17: 0x000000019e878984 dyld`dyld4::APIs::runAllInitializersForMain() + 468
frame #18: 0x000000019e83d2d0 dyld`dyld4::prepare(dyld4::APIs&, dyld3::MachOAnalyzer const*) + 3480
frame #19: 0x000000019e83be18 dyld`start + 1964You can also see that a C program does not segfault, because (I believe) there is only one libc++.so linked at runtime to satisfy libjemalloc.dylib. And hence there is no funny business, dyld can run the initialisers without failing.
$ echo 'int main() {}' > hello.c
$ clang hello.c -ljemalloc
$ ./a.out
abathur
nixos-discourse
veprbl
ghost
reckenrode
Describe the bug
libcxx is a c++ standard library used on x64 darwin and when it is loaded by dlopen or as a result of a dlopen eg:
import requestson python the program segfaults in the initialization of libcxx.older versions of libcxx [6-14] dlopen fine. when libcxx is linked into main executable dlopen is also fine.
the macOS system was created by the steps outlined https://github.com/kholia/OSX-KVM selecting the
big surinstall media and run under qemu on linux.note that a vm setup using the
montereyinstall media does not have this issue.Steps To Reproduce
just need to dlopen the lib. see below for a nix-shell script which builds for libcxx[6, 16].
Expected behavior
libcxx loads into the address space of the program and the initialization routines succeed.
Additional context
dltest_15_2023-11-23-183122_odel.crash.txt dltest_16_2023-11-23-183122_odel.crash.txt python3.11_2023-11-22-002452_odel.crash.txt
Notify maintainers
@dtzWill @Ericson2314 @lovek323 @primeos @alyssais @RaitoBezarius @rrbutani @sternenseemann
Metadata
Priorities
Add a :+1: reaction to issues you find important.