search

xoofx / LibObjectFile

LibObjectFile is a .NET library to read, manipulate and write linker and executable object files (e.g ELF, PE, DWARF, ar...)
BSD 2-Clause "Simplified" License
159 stars 11 forks source link

Multiple LineSequences not supported properly #12

Closed damageboy closed 3 years ago

damageboy commented 4 years ago

Hi, I'm trying to generate an object file with multiple source file worth of debug information as part of an effort to make JitDasm support dumping JIT results all the way down to ELF.

As such, I want to provide multiple line sequences in one object file. LibObjectFile's current API does allow for this, as can be seen from my code here: https://github.com/damageboy/JitDasm/blob/wip/dumptoelf/JitDasm/ElfDumper.cs#L155

The resulting object file seems to provide a partially working result, but the various elf dumping utilities seem to complain about the length field:

readelf -W --debug-dump=rawline /tmp/x.o:

Raw dump of debug contents of section .debug_line:

  Offset:                      0x0
  Length:                      120
  DWARF Version:               2
  Prologue Length:             74
  Minimum Instruction Length:  1
  Initial value of 'is_stmt':  1
  Line Base:                   -5
  Line Range:                  14
  Opcode Base:                 13

 Opcodes:
  Opcode 1 has 0 args
  Opcode 2 has 1 arg
  Opcode 3 has 1 arg
  Opcode 4 has 1 arg
  Opcode 5 has 1 arg
  Opcode 6 has 0 args
  Opcode 7 has 0 args
  Opcode 8 has 0 args
  Opcode 9 has 1 arg
  Opcode 10 has 0 args
  Opcode 11 has 0 args
  Opcode 12 has 1 arg

 The Directory Table (offset 0x1b):
  1     /home/dmg/projects/jitdasm/hw

 The File Name Table (offset 0x3a):
  Entry Dir     Time    Size    Name
  1     1       0       0       Wowza.cs
  2     1       0       0       Wowza2.cs

 Line Number Statements:
  [0x00000054]  Set column to 9
  [0x00000056]  Extended opcode 2: set Address to 0x5f2
  [0x00000061]  Special opcode 12: advance Address by 0 to 0x5f2 and Line by 7 to 8
  [0x00000062]  Special opcode 146: advance Address by 10 to 0x5fc and Line by 1 to 9
  [0x00000063]  Special opcode 132: advance Address by 9 to 0x605 and Line by 1 to 10
  [0x00000064]  Advance Line by 1 to 11
  [0x00000066]  Advance PC by 8 to 0x60d
  [0x00000068]  Extended opcode 1: End of Sequence

  [0x0000006b]  Set column to 9
  [0x0000006d]  Set File Name to entry 2 in the File Name Table
  [0x0000006f]  Extended opcode 2: set Address to 0x6b2
  [0x0000007a]  Special opcode 12: advance Address by 0 to 0x6b2 and Line by 7 to 8
  [0x0000007b]  Special opcode 146: advance Address by 10 to 0x6bc and Line by 1 to 9

readelf: Warning: The length field (0x2010391) in the debug_line header is wrong - the section is too small

As you can see the warning is:

Warning: The length field (0x2010391) in the debug_line header is wrong - the section is too small

Other tools complain about the same thing (objdump) or crash (llvm-dwarfdump).

I'm not familiar enough with DWARF to say with any certainty that this form of serializing is "legal" to begin with (e.g. one filename table with multiple entries and multiple sequences).

I will mention that while researching I found out that when compiling a shared object with multiple source files with gcc/clang, the resulting .debug_line section that is produced is entirely different when compared to LibObjectFile, where multiple tables appear in succession, in theory, one per file in succession:

Raw dump of debug contents of section .debug_line:

  Offset:                      0x0
  Length:                      261
  DWARF Version:               4
  Prologue Length:             33
  Minimum Instruction Length:  1
  Maximum Ops per Instruction: 1
  Initial value of 'is_stmt':  1
  Line Base:                   -5
  Line Range:                  14
  Opcode Base:                 13

 Opcodes:
  Opcode 1 has 0 args
  Opcode 2 has 1 arg
  Opcode 3 has 1 arg
  Opcode 4 has 1 arg
  Opcode 5 has 1 arg
  Opcode 6 has 0 args
  Opcode 7 has 0 args
  Opcode 8 has 0 args
  Opcode 9 has 1 arg
  Opcode 10 has 0 args
  Opcode 11 has 0 args
  Opcode 12 has 1 arg

 The Directory Table is empty.

 The File Name Table (offset 0x1d):
  Entry Dir Time    Size    Name
  1 0   0   0   small.cpp

 Line Number Statements:
  [0x0000002b]  Extended opcode 2: set Address to 0x1100
  [0x00000036]  Advance Line by 10 to 11
  [0x00000038]  Copy
  [0x00000039]  Set column to 9
  [0x0000003b]  Set prologue_end to true
  [0x0000003c]  Advance PC by constant 17 to 0x1111
  [0x0000003d]  Special opcode 48: advance Address by 3 to 0x1114 and Line by 1 to 12
  [0x0000003e]  Set column to 10
  [0x00000040]  Special opcode 104: advance Address by 7 to 0x111b and Line by 1 to 13
  [0x00000041]  Set column to 17
  [0x00000043]  Set is_stmt to 0
  [0x00000044]  Special opcode 103: advance Address by 7 to 0x1122 and Line by 0 to 13
  [0x00000045]  Set column to 21
  [0x00000047]  Special opcode 47: advance Address by 3 to 0x1125 and Line by 0 to 13
  [0x00000048]  Set column to 19
  [0x0000004a]  Special opcode 61: advance Address by 4 to 0x1129 and Line by 0 to 13
  [0x0000004b]  Set column to 2
  [0x0000004d]  Special opcode 47: advance Address by 3 to 0x112c and Line by 0 to 13
  [0x0000004e]  Set column to 23
  [0x00000050]  Set is_stmt to 1
  [0x00000051]  Special opcode 91: advance Address by 6 to 0x1132 and Line by 2 to 15
  [0x00000052]  Set column to 30
  [0x00000054]  Set is_stmt to 0
  [0x00000055]  Special opcode 61: advance Address by 4 to 0x1136 and Line by 0 to 15
  [0x00000056]  Set column to 23
  [0x00000058]  Special opcode 47: advance Address by 3 to 0x1139 and Line by 0 to 15
  [0x00000059]  Set column to 14
  [0x0000005b]  Special opcode 103: advance Address by 7 to 0x1140 and Line by 0 to 15
  [0x0000005c]  Set column to 23
  [0x0000005e]  Set is_stmt to 1
  [0x0000005f]  Special opcode 48: advance Address by 3 to 0x1143 and Line by 1 to 16
  [0x00000060]  Set column to 31
  [0x00000062]  Set is_stmt to 0
  [0x00000063]  Special opcode 61: advance Address by 4 to 0x1147 and Line by 0 to 16
  [0x00000064]  Set column to 23
  [0x00000066]  Special opcode 47: advance Address by 3 to 0x114a and Line by 0 to 16
  [0x00000067]  Set column to 14
  [0x00000069]  Special opcode 103: advance Address by 7 to 0x1151 and Line by 0 to 16
  [0x0000006a]  Set column to 23
  [0x0000006c]  Set is_stmt to 1
  [0x0000006d]  Special opcode 48: advance Address by 3 to 0x1154 and Line by 1 to 17
  [0x0000006e]  Set column to 43
  [0x00000070]  Set is_stmt to 0
  [0x00000071]  Special opcode 187: advance Address by 13 to 0x1161 and Line by 0 to 17
  [0x00000072]  Set column to 41
  [0x00000074]  Advance PC by constant 17 to 0x1172
  [0x00000075]  Special opcode 33: advance Address by 2 to 0x1174 and Line by 0 to 17
  [0x00000076]  Set column to 14
  [0x00000078]  Special opcode 33: advance Address by 2 to 0x1176 and Line by 0 to 17
  [0x00000079]  Set column to 16
  [0x0000007b]  Set is_stmt to 1
  [0x0000007c]  Special opcode 49: advance Address by 3 to 0x1179 and Line by 2 to 19
  [0x0000007d]  Set column to 25
  [0x0000007f]  Set is_stmt to 0
  [0x00000080]  Special opcode 61: advance Address by 4 to 0x117d and Line by 0 to 19
  [0x00000081]  Set column to 23
  [0x00000083]  Special opcode 61: advance Address by 4 to 0x1181 and Line by 0 to 19
  [0x00000084]  Set column to 32
  [0x00000086]  Special opcode 33: advance Address by 2 to 0x1183 and Line by 0 to 19
  [0x00000087]  Set column to 13
  [0x00000089]  Special opcode 47: advance Address by 3 to 0x1186 and Line by 0 to 19
  [0x0000008a]  Set column to 36
  [0x0000008c]  Set is_stmt to 1
  [0x0000008d]  Advance Line by -6 to 13
  [0x0000008f]  Special opcode 89: advance Address by 6 to 0x118c and Line by 0 to 13
  [0x00000090]  Set column to 2
  [0x00000092]  Set is_stmt to 0
  [0x00000093]  Special opcode 131: advance Address by 9 to 0x1195 and Line by 0 to 13
  [0x00000094]  Set column to 12
  [0x00000096]  Set is_stmt to 1
  [0x00000097]  Special opcode 83: advance Address by 5 to 0x119a and Line by 8 to 21
  [0x00000098]  Set column to 5
  [0x0000009a]  Set is_stmt to 0
  [0x0000009b]  Special opcode 47: advance Address by 3 to 0x119d and Line by 0 to 21
  [0x0000009c]  Set column to 0
  [0x0000009e]  Set is_stmt to 1
  [0x0000009f]  Advance PC by constant 17 to 0x11ae
  [0x000000a0]  Special opcode 37: advance Address by 2 to 0x11b0 and Line by 4 to 25
  [0x000000a1]  Set column to 10
  [0x000000a3]  Set prologue_end to true
  [0x000000a4]  Advance PC by constant 17 to 0x11c1
  [0x000000a5]  Special opcode 48: advance Address by 3 to 0x11c4 and Line by 1 to 26
  [0x000000a6]  Set column to 22
  [0x000000a8]  Special opcode 104: advance Address by 7 to 0x11cb and Line by 1 to 27
  [0x000000a9]  Set column to 29
  [0x000000ab]  Set is_stmt to 0
  [0x000000ac]  Special opcode 103: advance Address by 7 to 0x11d2 and Line by 0 to 27
  [0x000000ad]  Set column to 33
  [0x000000af]  Special opcode 47: advance Address by 3 to 0x11d5 and Line by 0 to 27
  [0x000000b0]  Set column to 31
  [0x000000b2]  Special opcode 61: advance Address by 4 to 0x11d9 and Line by 0 to 27
  [0x000000b3]  Set column to 14
  [0x000000b5]  Special opcode 47: advance Address by 3 to 0x11dc and Line by 0 to 27
  [0x000000b6]  Set column to 32
  [0x000000b8]  Set is_stmt to 1
  [0x000000b9]  Special opcode 91: advance Address by 6 to 0x11e2 and Line by 2 to 29
  [0x000000ba]  Set column to 39
  [0x000000bc]  Set is_stmt to 0
  [0x000000bd]  Special opcode 61: advance Address by 4 to 0x11e6 and Line by 0 to 29
  [0x000000be]  Set column to 32
  [0x000000c0]  Special opcode 47: advance Address by 3 to 0x11e9 and Line by 0 to 29
  [0x000000c1]  Set column to 23
  [0x000000c3]  Special opcode 103: advance Address by 7 to 0x11f0 and Line by 0 to 29
  [0x000000c4]  Set column to 33
  [0x000000c6]  Set is_stmt to 1
  [0x000000c7]  Special opcode 48: advance Address by 3 to 0x11f3 and Line by 1 to 30
  [0x000000c8]  Set column to 41
  [0x000000ca]  Set is_stmt to 0
  [0x000000cb]  Special opcode 61: advance Address by 4 to 0x11f7 and Line by 0 to 30
  [0x000000cc]  Set column to 33
  [0x000000ce]  Special opcode 47: advance Address by 3 to 0x11fa and Line by 0 to 30
  [0x000000cf]  Set column to 24
  [0x000000d1]  Special opcode 103: advance Address by 7 to 0x1201 and Line by 0 to 30
  [0x000000d2]  Set column to 34
  [0x000000d4]  Set is_stmt to 1
  [0x000000d5]  Special opcode 48: advance Address by 3 to 0x1204 and Line by 1 to 31
  [0x000000d6]  Set column to 54
  [0x000000d8]  Set is_stmt to 0
  [0x000000d9]  Special opcode 187: advance Address by 13 to 0x1211 and Line by 0 to 31
  [0x000000da]  Set column to 52
  [0x000000dc]  Advance PC by constant 17 to 0x1222
  [0x000000dd]  Special opcode 33: advance Address by 2 to 0x1224 and Line by 0 to 31
  [0x000000de]  Set column to 25
  [0x000000e0]  Special opcode 33: advance Address by 2 to 0x1226 and Line by 0 to 31
  [0x000000e1]  Set column to 28
  [0x000000e3]  Set is_stmt to 1
  [0x000000e4]  Special opcode 49: advance Address by 3 to 0x1229 and Line by 2 to 33
  [0x000000e5]  Set column to 37
  [0x000000e7]  Set is_stmt to 0
  [0x000000e8]  Special opcode 61: advance Address by 4 to 0x122d and Line by 0 to 33
  [0x000000e9]  Set column to 35
  [0x000000eb]  Special opcode 61: advance Address by 4 to 0x1231 and Line by 0 to 33
  [0x000000ec]  Set column to 44
  [0x000000ee]  Special opcode 33: advance Address by 2 to 0x1233 and Line by 0 to 33
  [0x000000ef]  Set column to 25
  [0x000000f1]  Special opcode 47: advance Address by 3 to 0x1236 and Line by 0 to 33
  [0x000000f2]  Set column to 48
  [0x000000f4]  Set is_stmt to 1
  [0x000000f5]  Advance Line by -6 to 27
  [0x000000f7]  Special opcode 89: advance Address by 6 to 0x123c and Line by 0 to 27
  [0x000000f8]  Set column to 14
  [0x000000fa]  Set is_stmt to 0
  [0x000000fb]  Special opcode 131: advance Address by 9 to 0x1245 and Line by 0 to 27
  [0x000000fc]  Set column to 18
  [0x000000fe]  Set is_stmt to 1
  [0x000000ff]  Special opcode 83: advance Address by 5 to 0x124a and Line by 8 to 35
  [0x00000100]  Set column to 11
  [0x00000102]  Set is_stmt to 0
  [0x00000103]  Special opcode 47: advance Address by 3 to 0x124d and Line by 0 to 35
  [0x00000104]  Advance PC by 6 to 0x1253
  [0x00000106]  Extended opcode 1: End of Sequence

  Offset:                      0x109
  Length:                      101
  DWARF Version:               4
  Prologue Length:             34
  Minimum Instruction Length:  1
  Maximum Ops per Instruction: 1
  Initial value of 'is_stmt':  1
  Line Base:                   -5
  Line Range:                  14
  Opcode Base:                 13

 Opcodes:
  Opcode 1 has 0 args
  Opcode 2 has 1 arg
  Opcode 3 has 1 arg
  Opcode 4 has 1 arg
  Opcode 5 has 1 arg
  Opcode 6 has 0 args
  Opcode 7 has 0 args
  Opcode 8 has 0 args
  Opcode 9 has 1 arg
  Opcode 10 has 0 args
  Opcode 11 has 0 args
  Opcode 12 has 1 arg

 The Directory Table is empty.

 The File Name Table (offset 0x126):
  Entry Dir Time    Size    Name
  1 0   0   0   small2.cpp

 Line Number Statements:
  [0x00000135]  Extended opcode 2: set Address to 0x1260
  [0x00000140]  Special opcode 6: advance Address by 0 to 0x1260 and Line by 1 to 2
  [0x00000141]  Set column to 9
  [0x00000143]  Set prologue_end to true
  [0x00000144]  Special opcode 104: advance Address by 7 to 0x1267 and Line by 1 to 3
  [0x00000145]  Set column to 10
  [0x00000147]  Special opcode 104: advance Address by 7 to 0x126e and Line by 1 to 4
  [0x00000148]  Set column to 17
  [0x0000014a]  Set is_stmt to 0
  [0x0000014b]  Special opcode 103: advance Address by 7 to 0x1275 and Line by 0 to 4
  [0x0000014c]  Set column to 19
  [0x0000014e]  Special opcode 47: advance Address by 3 to 0x1278 and Line by 0 to 4
  [0x0000014f]  Set column to 2
  [0x00000151]  Special opcode 47: advance Address by 3 to 0x127b and Line by 0 to 4
  [0x00000152]  Set column to 16
  [0x00000154]  Set is_stmt to 1
  [0x00000155]  Special opcode 91: advance Address by 6 to 0x1281 and Line by 2 to 6
  [0x00000156]  Set column to 17
  [0x00000158]  Set is_stmt to 0
  [0x00000159]  Special opcode 47: advance Address by 3 to 0x1284 and Line by 0 to 6
  [0x0000015a]  Set column to 13
  [0x0000015c]  Special opcode 61: advance Address by 4 to 0x1288 and Line by 0 to 6
  [0x0000015d]  Set column to 25
  [0x0000015f]  Set is_stmt to 1
  [0x00000160]  Special opcode 87: advance Address by 6 to 0x128e and Line by -2 to 4
  [0x00000161]  Set column to 2
  [0x00000163]  Set is_stmt to 0
  [0x00000164]  Special opcode 131: advance Address by 9 to 0x1297 and Line by 0 to 4
  [0x00000165]  Set column to 12
  [0x00000167]  Set is_stmt to 1
  [0x00000168]  Special opcode 79: advance Address by 5 to 0x129c and Line by 4 to 8
  [0x00000169]  Set column to 5
  [0x0000016b]  Set is_stmt to 0
  [0x0000016c]  Special opcode 47: advance Address by 3 to 0x129f and Line by 0 to 8
  [0x0000016d]  Advance PC by 2 to 0x12a1
  [0x0000016f]  Extended opcode 1: End of Sequence

I don't know if this last bit is even important, as I don't happen to think that the current layout used by LibObjectFile is wrong in any way, I'm just mentioning this for completeness sake.

xoofx commented 4 years ago

Thanks! It seems that there are 2 issues:

  1. A debug_line section can have multiple line program tables. Something that I completely missed in the specs.
  2. Output multiple line sequences is not generating a valid debug_line section.

For 1, that should be an easy fix. Need to investigate 2.

xoofx commented 4 years ago

I have tried to fix both issues by commit c678e7c8a59623c2b250be28c19b91b22023cfb6 but I will have to push more tests later.

In the meantime, can you try with latest master to check if it is fixing your issue?

xoofx commented 4 years ago

So with the new system, you can output multiple line program table and you should be able to create 2 compilation units associated with 2 different line program table.

xoofx commented 4 years ago

It seems that multiple line sequences within a line program table is actually not generated by gcc. Otherwise, one compilation unit is associated with one line program table. Note also that you might need to generate also a .symtab for the functions in your ELF.

damageboy commented 4 years ago

Thanks @xoofx, I'll give it a go in a couple of hours and report right back!

damageboy commented 4 years ago

Looking good!

I've tested the multiple line-sequences in on table so far, and the previous warning no longer appears.

This is good enough for objdump for now.

I will also re-write the code to output one linetable per file, which is what gcc/clang -shared produce for me, and will report back.

If all is good I'll close the issue. Thanks for everything!