LLDB lags while stopping at a breakpoint for binaries linked with SOLD

[tapthaker]

I can reproduce this internally but can't share a reproducer publicly. Any tips for me to debug this further will be helpful.

[rui314]

My very wild guess is that the executable created by sold lacks some quick-lookup table or something and lldb falls back to the linear search. But it's very hard to find the exact reason without the executable.

Can you identify what function takes such a long time in lldb? If we can find that function, then the thing that the function handles is the data that causes the slowdown.

[rui314]

One thing I can think of is the lack of source filenames in debug symbols (https://github.com/bluewhalesystems/sold/blob/1136fda7277fb0d199ebf41e09fbd46fb7bfe725/macho/input-files.cc#L1267-L1275). I'm not sure if it really affects the lldb's behavior, but that's obviously different from lld/ld64's output, so it's worth to make a change to sold to fill proper values. Let me implement it. I believe I can finish it this week.

[tapthaker]

I think this might be it. If I compare the two binaries with Bloaty, I do see that the StringTable w/ LLD is way bigger in size:

Binary linked with SOLD:

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  37.5%   413Mi  36.6%   413Mi    String Table
  26.8%   295Mi  26.1%   295Mi    __TEXT,__text
  14.4%   158Mi  14.0%   158Mi    Symbol Table
   7.6%  83.2Mi   7.4%  83.2Mi    __TEXT,__cstring
   0.0%       0   2.4%  27.0Mi    __DATA,__bss
   1.9%  21.2Mi   1.9%  21.2Mi    __DATA,__llvm_prf_data
   1.7%  18.3Mi   1.6%  18.3Mi    __TEXT,__const
   1.5%  16.2Mi   1.5%  16.7Mi    [64 Others]
   1.4%  15.4Mi   1.4%  15.4Mi    __LLVM_COV,__llvm_covfun
   1.1%  12.4Mi   1.1%  12.4Mi    __DATA,__llvm_prf_names
   1.0%  11.2Mi   1.0%  11.2Mi    __DATA,__objc_const
   0.9%  9.40Mi   0.8%  9.40Mi    __DATA_CONST,__const
   0.8%  9.26Mi   0.8%  9.26Mi    __DATA,__data
   0.8%  8.36Mi   0.7%  8.36Mi    __TEXT,__eh_frame
   0.5%  5.37Mi   0.5%  5.37Mi    Code Signature
   0.5%  5.18Mi   0.5%  5.18Mi    __DATA,__llvm_prf_cnts
   0.4%  4.46Mi   0.4%  4.46Mi    __TEXT,__unwind_info
   0.4%  4.40Mi   0.4%  4.40Mi    __DATA,__objc_data
   0.3%  3.64Mi   0.3%  3.64Mi    __TEXT,__swift5_reflstr
   0.3%  3.60Mi   0.3%  3.60Mi    __LLVM_COV,__llvm_covmap
   0.3%  3.00Mi   0.3%  3.00Mi    __TEXT,__swift5_fieldmd
 100.0%  1.08Gi 100.0%  1.10Gi    TOTAL

Binary linked with LLD:

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  54.2%   811Mi  53.2%   811Mi    String Table
  19.7%   294Mi  19.3%   294Mi    __TEXT,__text
  10.6%   158Mi  10.4%   158Mi    Symbol Table
   5.5%  83.1Mi   5.4%  83.1Mi    __TEXT,__cstring
   0.0%       0   1.8%  26.9Mi    __DATA,__bss
   1.4%  21.2Mi   1.4%  21.2Mi    __DATA,__llvm_prf_data
   1.1%  16.3Mi   1.1%  16.3Mi    __TEXT,__const
   1.0%  15.4Mi   1.0%  15.4Mi    __LLVM_COV,__llvm_covfun
   0.9%  13.6Mi   0.9%  14.0Mi    [59 Others]
   0.8%  12.4Mi   0.8%  12.4Mi    __DATA,__llvm_prf_names
   0.7%  11.2Mi   0.7%  11.2Mi    __DATA,__objc_const
   0.6%  9.38Mi   0.6%  9.38Mi    __DATA_CONST,__const
   0.6%  9.25Mi   0.6%  9.25Mi    __DATA,__data
   0.6%  8.40Mi   0.6%  8.40Mi    __TEXT,__eh_frame
   0.5%  7.29Mi   0.5%  7.29Mi    Code Signature
   0.4%  5.77Mi   0.4%  5.77Mi    __TEXT,__unwind_info
   0.3%  5.18Mi   0.3%  5.18Mi    __DATA,__llvm_prf_cnts
   0.3%  4.40Mi   0.3%  4.40Mi    __DATA,__objc_data
   0.2%  3.60Mi   0.2%  3.60Mi    __LLVM_COV,__llvm_covmap
   0.2%  3.14Mi   0.2%  3.14Mi    __TEXT,__swift5_reflstr
   0.2%  3.00Mi   0.2%  3.00Mi    __TEXT,__swift5_fieldmd
 100.0%  1.46Gi 100.0%  1.49Gi    TOTAL

If I compare the StringTable contents b/w the two; I see that SOLD is missing paths to the sourcefiles.

[rui314]

@tapthaker Does the above change make any difference?

[tapthaker]

@rui314 : I think we should re-open this.

The issue seems to be related to the symbols of code coverage. Debugging LLDB we figured out that Reading memory for GetNonCallSiteUnwindPlanFromAssembly seems to slow down while reading around ___llvm_profile_runtime_user

[tapthaker]

Fixed by https://github.com/bluewhalesystems/sold/commit/8fc7f88cf1c8a0b46268aa4152fbf1eea7b0ce96

[tapthaker]

Closing this !