Inaccurate debug information in binaries compiled with "-O2" and "-O3"

[edumoot]

The debug information for variables within the for loop, particularly at line 25 and line 26, is misleading, consistently showing global_shift = 0 and global_var = 470777959 during step-in or step-over operations (Godbolt). However, the final results are accurate. When a breakpoint is set at line 34, the debug information correctly displays global_var = 9 and global_shift = 16777200.

This issue can be reproduced in LLVM version of 18.1.8, 17.0.6, and 16.0.3 :

clang -g -O3 -o 420_O3.out 420.c
clang -g -O2 -o 420_O2.out 420.c

(lldb) file 420_O3.out
(lldb) b 25
(lldb) r
[...]
* thread #1, name = '420_O3.out', stop reason = breakpoint 1.1
    frame #0: 0x0000555555555141 420_O3.out`main [inlined] func_1 at 420.c:25:20
   22       for (global_var = 4; global_var != 9; ++global_var)
   23       {
   24           unsigned char shift_val = 254;
-> 25           global_flag++;
   26           global_shift |= (0xF1BB << (*local_var_ptr)) | ((shift_val << 15) * ((*local_var_ptr << global_flag) >= (*global_ptr)));
   27       }
   28       return global_union;
(lldb) p global_var
(int) 470777959
(lldb) p global_shift
(unsigned int) 0

...
(lldb) s
(lldb) s
(lldb) s
...
(lldb) p global_shift
(unsigned int) 0
(lldb) p global_var
(int) 470777959

It also can be repoduced in GDB context:

$ gdb 420_O3.out
GNU gdb (Ubuntu 15.0.50.20240403-0ubuntu1) 15.0.50.20240403-git
....

(gdb) b 25
(gdb) r
[...]

(gdb) p global_var
$1 = 470777959
(gdb) p global_shift
$2 = 0
(gdb) s
(gdb) s
(gdb) s
(gdb) p global_var
$3 = 470777959
(gdb) p global_shift
$4 = 0

[edumoot]

The test case cat 420.c

union U1 {
    short short_val;
    volatile int int_val;
    unsigned int uint_val;
    short another_short;
    volatile int another_int;
};

static int global_var = 0x1C0F8067;
static int *global_ptr = &global_var;
static int **volatile global_ptr_ref = &global_ptr;
static int global_counter = 0x95751CF;
static volatile unsigned char global_flag = 9;
static unsigned int global_shift = 0;
static union U1 global_union = {-4};

static union U1 func_1(void)
{
    int *local_var_ptr = &global_var;
    (*global_ptr_ref) = local_var_ptr;

    for (global_var = 4; global_var != 9; ++global_var)
    {
        unsigned char shift_val = 254;
        global_flag++;
        global_shift |= (0xF1BB << (*local_var_ptr)) | ((shift_val << 15) * ((*local_var_ptr << global_flag) >= (*global_ptr)));
    }
    return global_union;
}

int main (void)
{
    func_1();
    return 0;
}

[llvmbot]

@llvm/issue-subscribers-debuginfo

Author: Yachao Zhu (edumoot)

The debug information for variables within the for loop, particularly at line 25 and line 26, is misleading, consistently showing `global_shift = 0` and `global_var = 470777959` during step-in or step-over operations [(Godbolt)](https://clang.godbolt.org/z/T1aadhhsf). However, the final results are accurate. When a breakpoint is set at line 34, the debug information correctly displays `global_var = 9` and `global_shift = 16777200`. This issue can be reproduced in LLVM version of 18.1.8, 17.0.6, and 16.0.3 : ``` clang -g -O3 -o 420_O3.out 420.c clang -g -O2 -o 420_O2.out 420.c (lldb) file 420_O3.out (lldb) b 25 (lldb) r [...] * thread #1, name = '420_O3.out', stop reason = breakpoint 1.1 frame #0: 0x0000555555555141 420_O3.out`main [inlined] func_1 at 420.c:25:20 22 for (global_var = 4; global_var != 9; ++global_var) 23 { 24 unsigned char shift_val = 254; -> 25 global_flag++; 26 global_shift |= (0xF1BB << (*local_var_ptr)) | ((shift_val << 15) * ((*local_var_ptr << global_flag) >= (*global_ptr))); 27 } 28 return global_union; (lldb) p global_var (int) 470777959 (lldb) p global_shift (unsigned int) 0 ... (lldb) s (lldb) s (lldb) s ... (lldb) p global_shift (unsigned int) 0 (lldb) p global_var (int) 470777959 ``` It also can be repoduced in GDB context: ``` $ gdb 420_O3.out GNU gdb (Ubuntu 15.0.50.20240403-0ubuntu1) 15.0.50.20240403-git .... (gdb) b 25 (gdb) r [...] (gdb) p global_var $1 = 470777959 (gdb) p global_shift $2 = 0 (gdb) s (gdb) s (gdb) s (gdb) p global_var $3 = 470777959 (gdb) p global_shift $4 = 0 ```

[edumoot]

The issue continues to occur in LLVM 20.

clang version 20.0.0git (https://github.com/llvm/llvm-project.git e004566547bb13386ee30c78176dd7988c42860a)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/local/bin
Build config: +unoptimized, +assertions

(lldb) version
lldb version 20.0.0git (https://github.com/llvm/llvm-project.git revision e004566547bb13386ee30c78176dd7988c42860a)
  clang revision e004566547bb13386ee30c78176dd7988c42860a
  llvm revision e004566547bb13386ee30c78176dd7988c42860a

[jmorse]

Looking at the assembly, at -O2 the whole of the function gets unrolled, the value calculated for global_shift compressed to a single operation, and all the loads/stores for global_shift get elided into a single instruction. However because one of the globals is marked volatile, a collection of loads and stores can't be deleted (as that's not permitted for volatile accesses) even though their results are simply discarded, and so you get the spurious stepping behaviour.

This is an artefact of the optimisation and code that's being compiled: using globals and volatile qualifiers causes certain operations to be "frozen" and untransformed, which necessarily means you observe partial + incomplete states of the transformed program. There's nothing we could do differently in debug-info to represent this.

[dwblaikie]

If I understand correctly - the non-volatile static writes are being optimized away, right? (because they don't escape, are only used in one function, so the operations are localized - but the final side effect is externalized/written to the global)

In /theory/ we could make a location description for the global that includes the hoisting into local (or even constant over certain instructions) storage.

I think we're a /long/ way from ever doing that - and doing that across translation units would be even harder (like we probably do escape analysis and localize operations even when the global isn't TU-local, so long as no external functions are called that could observe the global state), as I don't think DWARF offers any real way to handle that (different locations for the same variable in different compile units)