My clang / gcc binaries fail disassembly

[ceesb]

I'm experimenting with C code that has a large data segment and relatively little code. I'm happy to share the binaries with you, but I prefer not to post them publicly. If you want them, send me a ping, and I can email them. Hope this helps!

Issue 1, for gcc binaries (-O2 or not) ddisasm hangs

If I compile with gcc (Ubuntu 7.5.0-3ubuntu1~18.04), ddisasm will hang in the disassembling phase (with or without "-O2"). When I interrupt I see this:

$ gcc -O2 -Wall -o wbc_gcc wbc.c && ./wbc_gcc 00112233445566778899aabbccddeeff
255fb7f6c6456e0420de3aff5d1247b9
$ ddisasm  --ir retargeted_wbc_gcc.gtirb --asm retargeted_wbc_gcc.s wbc_gcc
Building the initial gtirb representation  (19ms)
Decoding the binary  (1s)
Disassembling ^CInterrupt signal in rule:
propagated_data_access((EA+Mult),Mult,EA_ref) :- 
   propagated_data_access(EA,Mult,EA_ref),
   data_byte( _tmp_0,_),
   possible_data_limit( _tmp_0),
   data_access_pattern( _tmp_0,S,0,From),
   data_access_pattern(EA_ref,S,Mult,From),
   last_data_access( _tmp_0,Last),
   data_access_pattern(Last,Size,Mult,_),
   Last > EA,
   (Last+Size) <= (EA+Mult),
    _tmp_0 = (EA+Mult).
in file data_access_analysis.dl [254:1-274:7]

Issue 2, for clang -O2 ddisasm produces a broken asm

If I compile with clang (6.0.0-1ubuntu2) and "-O2", ddisasm finishes but the assembly will not compile.

$ clang -O2 -Wall -o wbc_clang wbc.c && ./wbc_clang 00112233445566778899aabbccddeeff
255fb7f6c6456e0420de3aff5d1247b9
$ ddisasm  --ir retargeted_wbc_clang.gtirb --asm retargeted_wbc_clang.s wbc_clang
Building the initial gtirb representation  (27ms)
Decoding the binary  (1s)
Disassembling  (11s)
Populating gtirb representation  (68s)
Computing intra-procedural SCCs  (0ms)
Computing no return analysis  (186ms)
Detecting additional functions  (793ms)
Printing assembler  (24s)
$ gcc -o retargeted_wbc_clang retargeted_wbc_clang.s
/usr/bin/ld: /tmp/ccQujJfU.o: relocation R_X86_64_32S against `.rodata' can not be used when making a PIE object; recompile with -fPIC
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status

Issue 3, for clang -O2 with relocations ddisasm finds no errors

On the clang with "-O2" binary for which ddisasm produces broken asm, I tried to run ddisasm with --self-diagnose, but it reports everything is fine. Attempting to compile the resulting assembly still leads to an error.

$ clang -O2 -Wall -Xlinker "--emit-relocs" -o wbc_clang_relocs wbc.c && ./wbc_clang_relocs 00112233445566778899aabbccddeeff
255fb7f6c6456e0420de3aff5d1247b9
$ ddisasm  --self-diagnose --ir retargeted_wbc_clang_relocs.gtirb --asm retargeted_wbc_clang_relocs.s wbc_clang_relocs
Building the initial gtirb representation  (45ms)
Decoding the binary  (1s)
Disassembling  (11s)
Populating gtirb representation  (72s)
Computing intra-procedural SCCs  (1ms)
Computing no return analysis  (185ms)
Detecting additional functions  (803ms)
Printing assembler  (26s)
Perfoming self diagnose (this will only give the right results if the target program contains all the relocation information)
Self diagnose completed: No errors found
$ gcc -o retargeted_wbc_clang_relocs retargeted_wbc_clang_relocs.s
/usr/bin/ld: /tmp/cc86oslH.o: relocation R_X86_64_32S against undefined symbol `_disambig_94898743884080' can not be used when making a PIE object; recompile with -fPIC
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status

Clang without explicit optimization works!

If I compile with clang (6.0.0-1ubuntu2) and omit "-O2, all is well!

$ clang -Wall -o wbc_clang_noopt wbc.c && ./wbc_clang_noopt 00112233445566778899aabbccddeeff
255fb7f6c6456e0420de3aff5d1247b9
$ ddisasm  --ir retargeted_wbc_clang_noopt.gtirb --asm retargeted_wbc_clang_noopt.s wbc_clang_noopt
Building the initial gtirb representation  (23ms)
Decoding the binary  (1s)
Disassembling  (11s)
Populating gtirb representation  (69s)
Computing intra-procedural SCCs  (3ms)
Computing no return analysis  (175ms)
Detecting additional functions  (763ms)
Printing assembler  (25s)
$ gcc -o retargeted_wbc_clang_noopt retargeted_wbc_clang_noopt.s
$ ./retargeted_wbc_clang_noopt 00112233445566778899aabbccddeeff
255fb7f6c6456e0420de3aff5d1247b9

Ddisasm version is git commit 15f97c87a423b5e7fe69f863f6804ce3dc5724bb.

$ ddisasm --version
1.0.0 (15f97c8 2020-05-27)

[aeflores]

Hi @ceesb, thanks for trying ddisasm! It will be much easier for me to diagnose the problems if I have access to the binaries. Email will work.

• Issue 1 might be a performance issue, but it is hard to know without trying myself.
• Issue 2: One thing to consider is whether the disassembled binary is PIE or not. Have you tried reassembling with the flag -no-pie? I think in Ubuntu18 gcc generates PIE by default but clang does not.
• Issue 3: The error here is different, there seems to be an undefined symbol and the name of the symbol is just the suffix that we add whenever we have multiple symbols with the same name. This is probably a bug.

[ceesb]

Thanks for the reply!

Indeed issue 2 disappears with -no-pie. When I add -no-pie for issue 3 I'm left with undefined refs. I'll mail you the bins for issue 1 and 3.

[aeflores]

Hi @ceesb, issue 1 is a performance issue. It is a combination of a relatively large data section with the fact that there are few data accesses detected. That Datalog rule that you pointed out is behaving very badly. Fortunately, I have a fix almost ready! I will let you know once it makes its way to master.

I can't reproduce issue 3 with the last ddisasm version (commit 39c696e), is it possible that it got fixed? Let me know if that is not the case.

[aeflores]

Alright, https://github.com/GrammaTech/ddisasm/commit/07793370907bbc41df2d3c223683ad3b22e9ec81 should fix issue 1!

[ceesb]

Nice! Indeed I see issue 1 is solved. Can you tell me what was the key difference between the gcc binary that caused the hang and the clang binary that didn't?

Issue 3 is still open for 07793370907bbc41df2d3c223683ad3b22e9ec81 as pasted below; maybe I'm linking to an older dependency when building ddisasm?

$ ddisasm --version
1.0.0 (0779337 2020-06-01)
$ ddisasm  --self-diagnose --ir retargeted_wbc_clang_relocs.gtirb --asm retargeted_wbc_clang_relocs.s wbc_clang_relocs
Building the initial gtirb representation  (25ms)
Decoding the binary  (1s)
Disassembling (21s)
Populating gtirb representation  (86s)
Computing intra-procedural SCCs  (1ms)
Computing no return analysis  (195ms)
Detecting additional functions  (808ms)
Printing assembler  (39s)
Perfoming self diagnose (this will only give the right results if the target program contains all the relocation information)
Self diagnose completed: No errors found
$ gcc -o retargeted_wbc_clang_relocs retargeted_wbc_clang_relocs.s -no-pie
/tmp/cc2zMVWX.o: In function `AES_encrypt':
(.text+0x147): undefined reference to `_disambig_94828702669104'
(.text+0x158): undefined reference to `_disambig_94828702669104'
(.text+0x191): undefined reference to `_disambig_94828702669104'
(.text+0x1a2): undefined reference to `_disambig_94828702669104'
(.text+0x1d2): undefined reference to `_disambig_94828702669104'
/tmp/cc2zMVWX.o:(.text+0x1e3): more undefined references to `_disambig_94828702669104' follow
collect2: error: ld returned 1 exit status

[aeflores]

Regarding the difference between gcc and clang. It has to do with the specific patterns in which data is accessed. You can get an idea, if you generate assembly files with the --debug option. That will print assembly with extra annotations (this assembly won't be reassembleable but it is useful for understanding). In the gcc binary, ddisasm detects a few access patterns (e.g. at address L_3600: or at .L_1df40). These patterns are propagated through the data section (see annotations preferred_data_access). Because the data section is big and the accesses are sparse, the propagation was taking a long time (the way this is propagated now changed to do it more efficiently). In the clang binary, none of these accesses are detected, so no propagation takes place.

As for why no accesses are detected, it probably has to do with how addresses are computed in the clang binary. For what I can tell, the address computation that is going on to access these tables in the data section is quite intricate and even the data accesses for the gcc binary are inaccurate (though that does not necessarily break the assembly).

[aeflores]

Regarding issue 3, I finally managed to reproduce it using the master version of LIEF. We tend to stick to stable releases, that is why I couldn't reproduce it. Nonetheless, I believe this is still a ddisasm bug. I will look into it further.