search

facebookarchive / BOLT

Binary Optimization and Layout Tool - A linux command-line utility used for optimizing performance of binaries
2.51k stars 176 forks source link

Assertion `Data.isValidOffset(Offset) && "wrong LSDA address"' failed. #61

Open derek-zhang opened 5 years ago

derek-zhang commented 5 years ago

Hi, Anyone can give some help ? Why LSDA address is wrong? My binary was compiled by gcc8.2 on Linux, I built the binary with '-Wl,--emit-relocs' and -fno-reorder-blocks-and-partition

PERF2BOLT: Starting data aggregation job for perf.data.bak PERF2BOLT: spawning perf job to read branch events PERF2BOLT: spawning perf job to read mem events PERF2BOLT: spawning perf job to read process events PERF2BOLT: spawning perf job to read task events BOLT-INFO: Target architecture: x86_64 BOLT-INFO: first alloc address is 0x400000 BOLT-INFO: creating new program header table at address 0x3400000, offset 0x3000000 BOLT-INFO: enabling relocation mode BOLT-WARNING: build-id will not be checked because we could not read one from input binary BOLT-WARNING: split function detected on input : _ZN3rec5xxxxxx7XXXXXXX8instanceEv.cold.224/1. The support is limited in relocation mode. PERF2BOLT: waiting for perf mmap events collection to finish... PERF2BOLT: parsing perf-script mmap events output PERF2BOLT: waiting for perf task events collection to finish... PERF2BOLT: parsing perf-script task events output PERF2BOLT: input binary is associated with 1 PID(s) PERF2BOLT: waiting for perf events collection to finish... PERF2BOLT: parse branch events... PERF2BOLT: read 9910905 samples and 157521236 LBR entries PERF2BOLT: 0 samples (0.0%) were ignored PERF2BOLT: traces mismatching disassembled function contents: 7533595 (5.1%) PERF2BOLT: out of range traces involving unknown regions: 60876241 (41.2%) BOLT-INFO: forcing -jump-tables=move as PIC jump table was detected in function _ZN3xxx17xxxSSSS14calc_xxxxEmiRdS1_dRKSsiibii perf2bolt: /ssd2/xxxx/tool/llvm/llvm/tools/llvm-bolt/src/Exceptions.cpp:141: void llvm::bolt::BinaryFunction::parseLSDA(llvm::ArrayRef, uint64_t): Assertion `Data.isValidOffset(Offset) && "wrong LSDA address"' failed.

0 0x00000000014e1b18 llvm::sys::PrintStackTrace(llvm::raw_ostream&) /ssd2/xxxx/tool/llvm/llvm/lib/Support/Unix/Signals.inc:398:22

1 0x00000000014e1bab PrintStackTraceSignalHandler(void*) /ssd2/xxxx/tool/llvm/llvm/lib/Support/Unix/Signals.inc:462:1

2 0x00000000014e004a llvm::sys::RunSignalHandlers() /ssd2/xxxx/tool/llvm/llvm/lib/Support/Signals.cpp:49:19

3 0x00000000014e148f SignalHandler(int) /ssd2/xxxx/tool/llvm/llvm/lib/Support/Unix/Signals.inc:252:1

4 0x00007f89e6cb2fa0 __restore_rt (/opt/compiler/gcc-8.2/lib/libpthread.so.0+0x10fa0)

5 0x00007f89e67fba48 __GI_raise /home/liruihao/mygcc82/glibc-2.21/signal/../sysdeps/unix/sysv/linux/raise.c:55:10

6 0x00007f89e67e72d9 __GI_abort /home/liruihao/mygcc82/glibc-2.21/stdlib/abort.c:91:7

7 0x00007f89e67e71af get_sysdep_segment_value /home/liruihao/mygcc82/glibc-2.21/intl/loadmsgcat.c:541:8

8 0x00007f89e67e71af _nl_load_domain.cold.0 /home/liruihao/mygcc82/glibc-2.21/intl/loadmsgcat.c:1002:34

9 0x00007f89e67f4992 (/opt/compiler/gcc-8.2/lib/libc.so.6+0x2d992)

10 0x000000000057963f llvm::bolt::BinaryFunction::parseLSDA(llvm::ArrayRef, unsigned long) /ssd2/xxxxx/tool/llvm/llvm/tools/llvm-bolt/src/Exceptions.cpp:141:3

11 0x00000000005be797 llvm::bolt::RewriteInstance::disassembleFunctions() /ssd2/xxxx/tool/llvm/llvm/tools/llvm-bolt/src/RewriteInstance.cpp:2765:25

12 0x00000000005b2358 operator() /ssd2/xxxxx/tool/llvm/llvm/tools/llvm-bolt/src/RewriteInstance.cpp:1052:21

13 0x00000000005b2358 llvm::bolt::RewriteInstance::run()::'lambda'(std::set<unsigned long, std::less, std::allocator > const&, bool)::operator()(std::set<unsigned long, std::less, std::allocator > const&, bool) const (./perf2bolt+0x5b2358)

14 0x00000000005b25e3 llvm::bolt::RewriteInstance::run() /ssd2/xxxxx/tool/llvm/llvm/tools/llvm-bolt/src/RewriteInstance.cpp:1089:21

15 0x000000000040fc6e main /ssd2/xxxxx/tool/llvm/llvm/tools/llvm-bolt/src/llvm-bolt.cpp:312:61

16 0x00007f89e67e8b8e __libc_start_main /home/liruihao/mygcc82/glibc-2.21/csu/libc-start.c:289:16

17 0x000000000040e489 _start /home/liruihao/mygcc82/glibc-2.21/csu/../sysdeps/x86_64/start.S:121:0

============

I tried to comment out LSDA code in bolt RewriteInstance.cpp. then I can get a perf2bolt result But when I try to do llvm-bolt, an error occur as below.

===========

BOLT-WARNING: unable to disassemble instruction at offset 0x53 (address 0x1b459d3) in function K_XX_XX/sha1-x86_64.o/1(2) BOLT-WARNING: unable to disassemble instruction at offset 0x1 (address 0x1b46c81) in function K256/sha256-x86_64.o/1(2) BOLT-WARNING: unable to disassemble instruction at offset 0x15 (address 0x1b4b715) in function K512/sha512-x86_64.o/1(*2) BOLT-ERROR: cannot process binaries with unmarked object in code at address 0x1b5bd80 belonging to section .text in relocation mode.

Any help is much appreciated

maksfb commented 5 years ago

Even though you compile with -fno-reorder-blocks-and-partition flag that prevents code splitting, your input binary contains a function _ZN3rec5xxxxxx7XXXXXXX8instanceEv that is split. Could you verify that the module containing this function is compiled with -fno-reorder-blocks-and-partition?

derek-zhang commented 5 years ago

Sure, I recompiled my binary and the depended modules with -fno-reorder-blocks-and-partition But I still get an error: BOLT-WARNING: split function detected on input : ssl_get_prev_session.cold.5/1. The support is limited in relocation mode. I think ssl_get_prev_session is a system/public library openssl and I just link to it by using "-lssl". So, is that means I have to recompile all the system/public libraries I used ? That's a lot of work.....

very appreciate your help!

PERF2BOLT: Starting data aggregation job for perf.data PERF2BOLT: spawning perf job to read branch events PERF2BOLT: spawning perf job to read mem events PERF2BOLT: spawning perf job to read process events PERF2BOLT: spawning perf job to read task events BOLT-INFO: Target architecture: x86_64 BOLT-INFO: first alloc address is 0x400000 BOLT-INFO: creating new program header table at address 0x3400000, offset 0x3000000 BOLT-INFO: enabling relocation mode BOLT-WARNING: build-id will not be checked because we could not read one from input binary BOLT-WARNING: split function detected on input : ssl_get_prev_session.cold.5/1. The support is limited in relocation mode. PERF2BOLT: waiting for perf mmap events collection to finish... PERF2BOLT: parsing perf-script mmap events output PERF2BOLT: waiting for perf task events collection to finish... PERF2BOLT: parsing perf-script task events output PERF2BOLT: input binary is associated with 1 PID(s) PERF2BOLT: waiting for perf events collection to finish... PERF2BOLT: parse branch events... PERF2BOLT: read 539566 samples and 8301315 LBR entries PERF2BOLT: 0 samples (0.0%) were ignored PERF2BOLT: traces mismatching disassembled function contents: 393923 (5.1%) PERF2BOLT: out of range traces involving unknown regions: 3139369 (40.4%) BOLT-INFO: forcing -jump-tables=move as PIC jump table was detected in function _ZN3xxx17xxxxxxxx14xxxxxxxEmiRdS1_dRKSsiibii BOLT-WARNING: 1 collisions detected while hashing binary objects. Use -v=1 to see the list. PERF2BOLT: processing branch events... PERF2BOLT: wrote 77376 objects and 0 memory objects to perf.fdata

BTW,

  1. should I recompile my gcc8.2 with -fno-reorder-blocks-and-partition ?
  2. I compiled my binary using gcc4.8.2 and still get the same LSDA error:

perf2bolt: /ssd2/xxx/tool/llvm/llvm/tools/llvm-bolt/src/Exceptions.cpp:141: void llvm::bolt::BinaryFunction::parseLSDA(llvm::ArrayRef, uint64_t): Assertion `Data.isValidOffset(Offset) && "wrong LSDA address"' failed.

HShan886 commented 2 years ago

I get same an assert error, any help will be pleasure

aaupov commented 2 years ago

I get same an assert error, any help will be pleasure

We've recently added cold fragment support that would skip cases that BOLT can't process. So it's likely a different issue. Can you please provide the full log?

HShan886 commented 2 years ago

My full log: PERF2BOLT: Starting data aggregation job for perf.data PERF2BOLT: spawning perf job to read branch events PERF2BOLT: spawning perf job to read mem events PERF2BOLT: spawning perf job to read process events PERF2BOLT: spawning perf job to read task events BOLT-INFO: Target architecture: x86_64 BOLT-INFO: BOLT version: 571d6a7120c2b637db2bb46fe3029f9d8576ab86 BOLT-INFO: first alloc address is 0x400000 BOLT-INFO: creating new program header table at address 0x9200000, offset 0x8e00000 BOLT-INFO: disabling -align-macro-fusion in non-relocation mode BOLT-INFO: enabling lite mode BOLT-WARNING: __stop_google_malloc (0x4c68aaf) does not have any section BOLT-INFO: pre-processing profile using perf data aggregator BOLT-WARNING: build-id will not be checked because we could not read one from input binary PERF2BOLT: waiting for perf mmap events collection to finish... PERF2BOLT: parsing perf-script mmap events output PERF2BOLT: waiting for perf task events collection to finish... PERF2BOLT: parsing perf-script task events output PERF2BOLT: input binary is associated with 1 PID(s) PERF2BOLT: waiting for perf events collection to finish... PERF2BOLT: parse branch events... PERF2BOLT: read 720642 samples and 11514877 LBR entries PERF2BOLT: 0 samples (0.0%) were ignored PERF2BOLT: traces mismatching disassembled function contents: 0 (0.0%) PERF2BOLT: out of range traces involving unknown regions: 318112 (2.9%) BOLT-WARNING: Ignored 0 functions due to cold fragments. BOLT-INFO: forcing -jump-tables=move as PIC jump table was detected in function _ZN6localtime4yy11bs19ActiveSegment17CompleteEPNS2_15ContextE perf2bolt: /opt/llvm-project/bolt/lib/Core/Exceptions.cpp:114: void llvm::bolt::BinaryFunction::parseLSDA(llvm::ArrayRef, uint64_t): Assertion `Data.isValidOffset(Offset) && "wrong LSDA address"' failed.

0 0x00000000011a8845 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) /opt/llvm-project/llvm/lib/Support/Unix/Signals.inc:565:22

1 0x00000000011a88fc PrintStackTraceSignalHandler(void*) /opt/llvm-project/llvm/lib/Support/Unix/Signals.inc:632:1

2 0x00000000011a68d7 llvm::sys::RunSignalHandlers() /opt/llvm-project/llvm/lib/Support/Signals.cpp:97:20

3 0x00000000011a828c SignalHandler(int) /opt/llvm-project/llvm/lib/Support/Unix/Signals.inc:407:1

4 0x00007f48a5870620 __restore_rt sigaction.c:0:0

5 0x00007f48a498f277 raise (/lib64/libc.so.6+0x36277)

6 0x00007f48a4990968 abort (/lib64/libc.so.6+0x37968)

7 0x00007f48a4988096 __assert_fail_base (/lib64/libc.so.6+0x2f096)

8 0x00007f48a4988142 (/lib64/libc.so.6+0x2f142)

9 0x00000000028163ee llvm::bolt::BinaryFunction::parseLSDA(llvm::ArrayRef, unsigned long) /opt/llvm-project/bolt/lib/Core/Exceptions.cpp:114:3

10 0x0000000000d58448 llvm::bolt::RewriteInstance::disassembleFunctions() /opt/llvm-project/bolt/lib/Rewrite/RewriteInstance.cpp:2763:25

11 0x0000000000d4c19b llvm::bolt::RewriteInstance::run() /opt/llvm-project/bolt/lib/Rewrite/RewriteInstance.cpp:765:27

12 0x000000000040bcb9 main /opt/llvm-project/bolt/tools/driver/llvm-bolt.cpp:250:49

13 0x00007f48a497b445 __libc_start_main (/lib64/libc.so.6+0x22445)

14 0x000000000040a732 _start (/home/haishan.fhs/bin/llvm-bolt+0x40a732)

PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace. Stack dump:

  1. Program arguments: perf2bolt -p perf.data -o perf.fdata pangu_blockserver --update-debug-sections
HShan886 commented 2 years ago

@aaupov that's great, the code is ready for upstream?

aaupov commented 2 years ago

@aaupov

that's great, the code is ready for upstream?

Hi @Haishan312, sorry for taking so long on this issue. It looks like a separate issue from cold fragment support, so it has not been fixed yet.