[SR-2136] LLDB asserting on missing glibc module: Ubuntu, master-next, build layout only

[a0332500-857b-4d6a-874e-50b63b66b387]

Previous ID	SR-2136
Radar	rdar://problem/27459201
Original Reporter	@trfiala
Type	Bug
Status	Closed
Resolution	Done

Environment

swift-lldb/master-next branch, Ubuntu 16.04 x86_64, Ubuntu 14.04 x86_64. This happens only with the build layout of lldb/swift. It does not manifest when using the install layout of lldb and swift.

Additional Detail from JIRA

| | | |------------------|-----------------| |Votes | 0 | |Component/s | LLDB for Swift | |Labels | Bug, Linux | |Assignee | @trfiala | |Priority | Medium | md5: 6e445793b80bac593e7606ca7900ace4

relates to:

• SR-1300 LLDB REPL tests fail on Ubuntu when not run from install layout

Issue Description:

After merging LLVM.org LLDB trunk @r276079 into swift-lldb/master-next, we're getting an assert in many of the LLDB Swift tests here:

swift/lib/ClangImporter/ClangImporter.cpp:402: Assertion !GlibcModuleMapPath.empty() failed

It looks like we're not finding a resource dir to hand over to Swift when we create a clang importer as part of setting up a SwiftASTContext for each module. This might be a temporal issue where we would have found it later.

[a0332500-857b-4d6a-874e-50b63b66b387]

I'm going to grab this since the most likely candidate for addressing is out this week.

[a0332500-857b-4d6a-874e-50b63b66b387]

The backtrace for the assert looks like this when running one of the Python-driven LLDB tests against a Swift test subject:

(lldb) thread list
Process 18364 stopped

• thread #1: tid = 18364, 0x00007f29bbb06267 libc.so.6`__GI_raise(sig=6) + 55 at raise.c:55, name = 'python', stop reason = signal SIGABRT
  (lldb) bt
• thread #1: tid = 18364, 0x00007f29bbb06267 libc.so.6`__GI_raise(sig=6) + 55 at raise.c:55, name = 'python', stop reason = signal SIGABRT
• frame #0: 0x00007f29bbb06267 libc.so.6`__GIraise(sig=6) + 55 at raise.c:55
  frame #1: 0x00007f29bbb07eca libc.so.6`\_GIabort + 362 at abort.c:89
  frame #2: 0x00007f29bbaff03d libc.so.6`\_assert_fail_base(fmt=\<unavailable>, assertion=\<unavailable>, file=\<unavailable>, line=\<unavailable>, function=\<unavailable>) + 285 at assert.c:92
  frame #3: 0x00007f29bbaff0f2 libc.so.6`GI_assert_fail(assertion=\<unavailable>, file=\<unavailable>, line=\<unavailable>, function=\<unavailable>) + 66 at assert.c:101
  frame #4: 0x00007f29b39a234c _lldb.so`getNormalInvocationArguments(invocationArgStrs=size=512, ctx=0x000000000122b270, importerOpts=0x000000000122ac68) + 3292 at ClangImporter.cpp:402
  frame #5: 0x00007f29b39a0352 _lldb.so`swift::ClangImporter::create(ctx=0x000000000122b270, importerOpts=0x000000000122ac68, tracker=0x0000000000000000) + 290 at ClangImporter.cpp:570
  frame #6: 0x00007f29b300c10e _lldb.so`lldb_private::SwiftASTContext::GetClangImporter(this=0x000000000122a5a0) + 222 at SwiftASTContext.cpp:3299
  frame #7: 0x00007f29b300d9ec _lldb.so`lldb_private::SwiftASTContext::CreateInstance(language=eLanguageTypeSwift, module=0x00000000011dd170, target=0x0000000000000000, extra_options=0x0000000000000000) + 5404 at SwiftASTContext.cpp:1730
  frame #8: 0x00007f29b309a3de _lldb.so`lldb_private::TypeSystem::CreateInstance(language=eLanguageTypeSwift, module=0x00000000011dd170) + 126 at TypeSystem.cpp:36
  frame #9: 0x00007f29b309af36 _lldb.so`lldb_private::TypeSystemMap::GetTypeSystemForLanguage(this=0x00000000011dd2c8, language=eLanguageTypeSwift, module=0x00000000011dd170, can_create=true) + 470 at TypeSystem.cpp:278
  frame #10: 0x00007f29b2e48fb4 _lldb.so`lldb_private::Module::GetTypeSystemForLanguage(this=0x00000000011dd170, language=eLanguageTypeSwift) + 52 at Module.cpp:430
  frame #11: 0x00007f29b3237512 _lldb.so`SymbolFileDWARF::GetTypeSystemForLanguage(this=0x00000000012281d0, language=eLanguageTypeSwift) + 130 at SymbolFileDWARF.cpp:519
  frame #12: 0x00007f29b3239101 _lldb.so`SymbolFileDWARF::ParseCompileUnitFunction(this=0x00000000012281d0, sc=0x00007ffe6e2559b0, die=0x00007ffe6e2554f8) + 97 at SymbolFileDWARF.cpp:1008
  frame #13: 0x00007f29b323da14 _lldb.so`SymbolFileDWARF::ResolveSymbolContext(this=0x00000000012281d0, so_addr=0x00007ffe6e255938, resolve_scope=127, sc=0x00007ffe6e2559b0) + 804 at SymbolFileDWARF.cpp:2096
  frame #14: 0x00007f29b3074950 _lldb.so`lldb_private::SymbolVendor::ResolveSymbolContext(this=0x00000000012255a0, so_addr=0x00007ffe6e255938, resolve_scope=127, sc=0x00007ffe6e2559b0) + 192 at SymbolVendor.cpp:302
  frame #15: 0x00007f29b2e49878 _lldb.so`lldb_private::Module::ResolveSymbolContextForAddress(this=0x00000000011dd170, so_addr=0x00007ffe6e255938, resolve_scope=127, sc=0x00007ffe6e2559b0, resolve_tail_call_address=false) + 568 at Module.cpp:567
  frame #16: 0x00007f29b2dca6f7 _lldb.so`lldb_private::Address::CalculateSymbolContext(this=0x00007ffe6e255938, sc=0x00007ffe6e2559b0, resolve_scope=127) const + 231 at Address.cpp:869
  frame #17: 0x00007f29b2fe48e1 _lldb.so`lldb_private::CompileUnit::ResolveSymbolContext(this=0x0000000001228e70, file_spec=0x0000000001225630, line=16, check_inlines=true, exact=false, resolve_scope=127, sc_list=0x00007ffe6e255c60) + 769 at CompileUnit.cpp:425
  frame #18: 0x00007f29b35e336a _lldb.so`lldb_private::BreakpointResolverFileLine::SearchCallback(this=0x0000000001225610, filter=0x0000000001225280, context=0x00007ffe6e255d28, addr=0x0000000000000000, containing=false) + 506 at BreakpointResolverFileLine.cpp:87
  frame #19: 0x00007f29b2e91442 _lldb.so`lldb_private::SearchFilter::DoModuleIteration(this=0x0000000001225280, context=0x00007ffe6e255f00, searcher=0x0000000001225610) + 626 at SearchFilter.cpp:191
  frame #20: 0x00007f29b2e911a5 _lldb.so`lldb_private::SearchFilter::Search(this=0x0000000001225280, searcher=0x0000000001225610) + 245 at SearchFilter.cpp:118
  frame #21: 0x00007f29b35e09d1 _lldb.so`lldb_private::BreakpointResolver::ResolveBreakpoint(this=0x0000000001225610, filter=0x0000000001225280) + 33 at BreakpointResolver.cpp:62
  frame #22: 0x00007f29b2d431d3 _lldb.so`lldb_private::Breakpoint::ResolveBreakpoint(this=0x00000000012252a0) + 115 at Breakpoint.cpp:355
  frame #23: 0x00007f29b315eac5 _lldb.so`lldb_private::Target::AddBreakpoint(this=0x0000000001211630, bp_sp=lldb_private::Breakpoint @ 0x00000000012252a0, internal=false) + 341 at Target.cpp:714
  frame #24: 0x00007f29b315fe03 _lldb.so`lldb_private::Target::CreateBreakpoint(this=0x0000000001211630, filter_sp=lldb_private::SearchFilter @ 0x0000000001225280, resolver_sp=lldb_private::BreakpointResolver @ 0x0000000001225610, internal=false, request_hardware=false, resolve_indirect_symbols=true) + 307 at Target.cpp:691
  frame #25: 0x00007f29b31602b8 _lldb.so`lldb_private::Target::CreateBreakpoint(this=0x0000000001211630, containingModules=0x0000000000000000, file=0x0000000001222480, line_no=16, offset=0, check_inlines=eLazyBoolYes, skip_prologue=eLazyBoolYes, internal=false, hardware=false, move_to_nearest_code=eLazyBoolYes) + 1144 at Target.cpp:432
  frame #26: 0x00007f29b19116c9 _lldb.so`lldb::SBTarget::BreakpointCreateByLocation(this=0x0000000001222460, sb_file_spec=0x00007ffe6e257470, line=16, offset=0) + 361 at SBTarget.cpp:826
  frame #27: 0x00007f29b191154d _lldb.so`lldb::SBTarget::BreakpointCreateByLocation(this=0x0000000001222460, sb_file_spec=0x00007ffe6e257470, line=16) + 45 at SBTarget.cpp:805
  frame #28: 0x00007f29b1911503 _lldb.so`lldb::SBTarget::BreakpointCreateByLocation(this=0x0000000001222460, file="main.swift", line=16) + 83 at SBTarget.cpp:798
  frame #29: 0x00007f29b1a2b8be lldb.so`wrap_SBTarget_BreakpointCreateByLocation_SWIG_0((null)=0x0000000000000000, args=0x00007f29abf42500) + 574 at LLDBWrapPython.cpp:47643
  frame #30: 0x00007f29b19e112b _lldb.so`_wrap_SBTarget_BreakpointCreateByLocation(self=0x0000000000000000, args=0x00007f29abf42500) + 683 at LLDBWrapPython.cpp:47801
  frame #31: 0x00000000004c0b69 python`PyEval_EvalFrameEx + 28521
  frame #32: 0x00000000004b7986 python`PyEval_EvalCodeEx + 1046
  frame #33: 0x00000000004bfc61 python`PyEval_EvalFrameEx + 24673
  frame #34: 0x00000000004b7986 python`PyEval_EvalCodeEx + 1046
  frame #35: 0x00000000004bfc61 python`PyEval_EvalFrameEx + 24673
  frame #36: 0x00000000004b7986 python`PyEval_EvalCodeEx + 1046
  frame #37: 0x00000000004bfc61 python`PyEval_EvalFrameEx + 24673
  frame #38: 0x00000000004b7986 python`PyEval_EvalCodeEx + 1046
  frame #39: 0x00000000004bfc61 python`PyEval_EvalFrameEx + 24673
  frame #40: 0x00000000004bf10f python`PyEval_EvalFrameEx + 21775
  frame #41: 0x00000000004b7986 python`PyEval_EvalCodeEx + 1046
  frame #42: 0x00000000004d3cf3 python`__lldb_unnamed_symbol3329$$python + 387
  frame #43: 0x00000000004a45de python`PyObject_Call + 62
  frame #44: 0x00000000004bc9bc python`PyEval_EvalFrameEx + 11708
  frame #45: 0x00000000004b7986 python`PyEvalEvalCodeEx + 1046
  frame #46: 0x00000000004d3bb9 python`_\_lldb_unnamedsymbol3329$$python + 73
  frame #47: 0x00000000004ece7e python`_\_lldb_unnamed_symbol3444$$python + 446
  frame #48: 0x00000000004a45de python`PyObjectCall + 62
  frame #49: 0x0000000000546a93 python`_\_lldb_unnamed_symbol3973$$python + 51
  frame #50: 0x00000000004bf3cc python`PyEval_EvalFrameEx + 22476
  frame #51: 0x00000000004b7986 python`PyEval_EvalCodeEx + 1046
  frame #52: 0x00000000004bf4e9 python`PyEval_EvalFrameEx + 22761
  frame #53: 0x00000000004b7986 python`PyEval_EvalCodeEx + 1046
  frame #54: 0x00000000004bf4e9 python`PyEval_EvalFrameEx + 22761
  frame #55: 0x00000000004b7986 python`PyEvalEvalCodeEx + 1046
  frame #56: 0x00000000004d3cf3 python`_\_lldb_unnamed_symbol3329$$python + 387
  frame #57: 0x00000000004a45de python`PyObject_Call + 62
  frame #58: 0x00000000004bc9bc python`PyEval_EvalFrameEx + 11708
  frame #59: 0x00000000004b7986 python`PyEvalEvalCodeEx + 1046
  frame #60: 0x00000000004d3bb9 python`_\_lldb_unnamedsymbol3329$$python + 73
  frame #61: 0x00000000004ece7e python`_\_lldb_unnamed_symbol3444$$python + 446
  frame #62: 0x00000000004a45de python`PyObjectCall + 62
  frame #63: 0x0000000000546a93 python`_\_lldb_unnamed_symbol3973$$python + 51
  frame #64: 0x00000000004bf3cc python`PyEval_EvalFrameEx + 22476
  frame #65: 0x00000000004bf10f python`PyEval_EvalFrameEx + 21775
  frame #66: 0x00000000004b7986 python`PyEval_EvalCodeEx + 1046
  frame #67: 0x00000000004bfc61 python`PyEval_EvalFrameEx + 24673
  frame #68: 0x00000000004b7986 python`PyEvalEvalCodeEx + 1046
  frame #69: 0x00000000004e8f3f python`_\_lldb_unnamed_symbol3424$$python + 63
  frame #70: 0x00000000004e3b02 python`PyRun_FileExFlags + 130
  frame #71: 0x00000000004e22e6 python`PyRun_SimpleFileExFlags + 390
  frame #72: 0x0000000000490fe1 python`PyMain + 1505
  frame #73: 0x00007f29bbaf1a40 libc.so.6`\_libc_start_main(main=(python`main), argc=11, argv=0x00007ffe6e259a98, init=\<unavailable>, fini=\<unavailable>, rtld_fini=\<unavailable>, stack_end=0x00007ffe6e259a88) + 240 at libc-start.c:289
  frame #74: 0x0000000000490919 python`_start + 41

The issue happens when the test is trying to resolve a breakpoint address by location, and in the process enumerates the swift modules, which creates the SwiftASTContexts, each of which creates a clang module importer, which then gets angry because we haven't told it where to find glibc.

[a0332500-857b-4d6a-874e-50b63b66b387]

The LLDB test suite uses LLDB and Swift out of the build dir configuration — i.e. not the final install layout. I'm checking with a full package build install layout to see if the behavior is different. This might end up being the same root cause that led to me turning off the in-tree REPL tests on Linux several weeks ago. If they are the same, it would be good to fix that.

[a0332500-857b-4d6a-874e-50b63b66b387]

> The LLDB test suite uses LLDB and Swift out of the build dir configuration — i.e. not the final install layout. I'm checking with a full package build install layout to see if the behavior is different.

That hunch was correct. The LLDB Swift tests all pass when using the install-layout version of lldb. This seems to imply our resource dir lookup logic is not adequate on Linux when attempting to run lldb out of the build dir to either (1) use the repl or (2) debug a Swift program. Our test suite really depends on running in a build-style layout, so this is going to be important to address.

[a0332500-857b-4d6a-874e-50b63b66b387]

This issue seems to have the same underlying cause as SR-1300.

[a0332500-857b-4d6a-874e-50b63b66b387]

The work-around here is to use an install-layout lldb and not try to use it directly out of the build dir.

Meanwhile, I'm going to see if I can get a real solution to this.

[a0332500-857b-4d6a-874e-50b63b66b387]

I managed to make a fix for this. I'm going to run it through a PR after I do a bit more local testing.

[a0332500-857b-4d6a-874e-50b63b66b387]

I have a PR testing this fix on swift-lldb/master-next now:
https://github.com/apple/swift-lldb/pull/36

Once I have that tested and submitted successfully, I'll pull this back into swift-lldb/master, and possibly further.

[a0332500-857b-4d6a-874e-50b63b66b387]

This is working in the swift-lldb/master-next branch, as witnessed in this Ubuntu 15.10 x86_64 build:
https://ci.swift.org/view/swift-master-next/job/oss-lldb-incremental-linux-ubuntu-15_10-master-next/30/

It was developed on Ubuntu 16.04 x86_64, so works there as well. I'm waiting on the Ubuntu 14.04 x86_64 builder to work through it. Once those are clear, I'll get this in swift-lldb/master.

[a0332500-857b-4d6a-874e-50b63b66b387]

This was fixed a while back.