dwarf_child: DW_DLE_ABBREV_MISSING

[yota9]

The update-debug-sections doesn't work after the "Rewrite of .debug_info section" patch. Reproduce steps:

1. Get the test binary https://github.com/yota9/BOLT/blob/gcgort/bolt/gcgort
2. Run llvm-bolt gcgort -o gcgort.bolt -update-debug-sections
3. Run dwarfdump -a gcgsort.bolt , skip the file to the end and check for the error

Error that was observed: dwarfdump ERROR: dwarf_child: DW_DLE_ABBREV_MISSING(394) Unable to find abbrev for DIE

[ayermolo]

@yota9 I tried it locally with TOT bolt, and I wasn't to repro the issue. dwarfdump finished without issues.

[yota9]

@ayermolo Very strange, I continue to get the error

[ayermolo]

@ayermolo Very strange, I continue to get the error

Is it with TOT dwarfdump and llvm-bolt?

[yota9]

The tot llvm-bolt and the dwardump --version tells 2020-01-14 10:13:32-08:00 Package Version "20200114". The llvm-dwarfdump shows these errors:

:warning: DWARF unit at offset 0x0000007c contains invalid abbreviation 595 at offset 0x0000026b, valid abbreviations are 1-38
warning: DWARF unit at offset 0x0000071b contains invalid abbreviation 181 at offset 0x000008e1, valid abbreviations are 1-38
warning: DWARF unit at offset 0x000010d2 contains invalid abbreviation 73 at offset 0x000012f1, valid abbreviations are 1-38
.......

I'm using aarch64 machine, but it should not be the case here

[ayermolo]

The tot llvm-bolt and the dwardump --version tells 2020-01-14 10:13:32-08:00 Package Version "20200114". The llvm-dwarfdump shows these errors:

:warning: DWARF unit at offset 0x0000007c contains invalid abbreviation 595 at offset 0x0000026b, valid abbreviations are 1-38
warning: DWARF unit at offset 0x0000071b contains invalid abbreviation 181 at offset 0x000008e1, valid abbreviations are 1-38
warning: DWARF unit at offset 0x000010d2 contains invalid abbreviation 73 at offset 0x000012f1, valid abbreviations are 1-38
.......

I'm using aarch64 machine, but it should not be the case here

ah cool I see the warning. Let me dig in today.

[yota9]

At some point everything seems to be messed (Name, Language, Producer), probably just become incoherent with other dwarf sections :

0x0000024b:       DW_TAG_compile_unit
                    DW_AT_name  ("u*\005")
                    DW_AT_language      (0xaf)
                    DW_AT_stmt_list     (0x00000004)
                    DW_AT_low_pc        (0x000220000000c407)
                    DW_AT_ranges        (0x00000200
                       [0x0000000000800624, 0x0000000000800798))
                    DW_AT_comp_dir      ("")
                    DW_AT_producer      ("\230\001\023\336")
                    DW_AT_unknown_2905  ("")

Let me dig in today.

Thank you! :)

[ayermolo]

OK, so here is what happened. All of CUs share the same abbrev table. Except first CU only has/uses in abbrev table DW_TAG_compile_unit and DW_TAG_subprogram. While second CU has all the other fun stuff. Including

0x0000023a:     DW_TAG_lexical_block [21] *
                  DW_AT_low_pc [DW_FORM_addr] (0x0000000000400d68)
                  DW_AT_high_pc [DW_FORM_addr]  (0x0000000000400edc)

Which gets converted in to

0x0000023a:     DW_TAG_lexical_block [21] *
                  DW_AT_low_pc [DW_FORM_addr] (0x0000000000000000)
                  DW_AT_ranges [DW_FORM_sec_offset] (0x00000200
                     [0x0000000000c00ea4, 0x0000000000c01018))

When we write out .debug_abbrev we rely on abbr_offset. So we write out the unmodified one, and then ignore all other CUs and their abbrev modifications.

[ayermolo]

@yota9 How did you create that test? I tried to re-create something locally using c++/thinlto, but not able to. :/

[yota9]

@ayermolo I'm sorry, to be honest I didn't understand exactly what the problem was, I hope I will understand a bit more in code :)

How did you create that test?

This is the golang binary because :) I have no idea how force to generate such thing using C/C++ compiler. The test I've finally made on my DWARF patch review was the C hello world + I've hexedited the dwarf by my hands :)

[ayermolo]

@ayermolo I'm sorry, to be honest I didn't understand exactly what the problem was, I hope I will understand a bit more in code :)

How did you create that test?

This is the golang binary because :) I have no idea how force to generate such thing using C/C++ compiler. The test I've finally made on my DWARF patch review was the C hello world + I've hexedited the dwarf by my hands :)

I am not familiar with golang. What are the steps to build binary that produces this debug info? Maybe I can start that as a baseline and reduce. To minimize by hand hacking.

[yota9]

Well, I'm absolutely not into the golang either :) But as for beginning you can get golang compiler and some of the test from test folder and make go build file.go. If you would like to use external linker you can do this go build -a -ldflags='-linkmode=external -extld=clang -extldflags "-no-pie -Wl,--compress-debug-sections=none -Wl,--emit-relocs"' file.go The golang produces dwarf by default. P.S. I've looked at the test I've used again (gcgort.go) and I was wrong, it is not the smallest one. But even with smaller test like struct0.go unstripped file is still 3.5 mb, so not really a big difference)

[ayermolo]

Well, I'm absolutely not into the golang either :) But as for beginning you can get golang compiler and some of the test from test folder and make go build file.go. If you would like to use external linker you can do this go build -a -ldflags='-linkmode=external -extld=clang -extldflags "-no-pie -Wl,--compress-debug-sections=none -Wl,--emit-relocs"' file.go The golang produces dwarf by default. P.S. I've looked at the test I've used again (gcgort.go) and I was wrong, it is not the smallest one. But even with smaller test like struct0.go unstripped file is still 3.5 mb, so not really a big difference)

Presumably not all tests exhibit this behavior. What were the flags to produce that particular binary? Was LTO used?

[yota9]

The command string above was used with for gcgort.go test with golang 1.16.5.

Was LTO used

I suppose golang does not support LTO at all.

[ayermolo]

https://reviews.llvm.org/D118517

[ayermolo]

committed to upstream

[yota9]

Thanks @ayermolo ! I will check it soon too :)