Inliner sometimes produces invalid debug info

[Quuxplusone]

Bugzilla Link	PR23874
Status	NEW
Importance	P normal
Reported by	Derek Schuff (dschuff@google.com)
Reported on	2015-06-17 13:30:51 -0700
Last modified on	2015-06-18 16:00:30 -0700
Version	trunk
Hardware	PC Linux
CC	dblaikie@gmail.com, dexonsmith@apple.com, echristo@gmail.com, llvm-bugs@lists.llvm.org, paul_robinson@playstation.sony.com, rafael@espindo.la
Fixed by commit(s)
Attachments
Blocks
Blocked by	PR23367
See also

I have an IR file with an approximate structure as follows. It has an inner and
outer function with debug info and a middle function without debug info.
The inner is inlined into the middle, and then the middle is inlined into the
outer. After inlining when the module verifier visits the DISubprogram for the
middle function, it checks the first instruction in middle (which originally
came from inner), gets the inlinedAtScope, finds that it points to inner, and
fails verification.

; Function Attrs: nounwind uwtable
define void @inner() #0 {
entry:
  <some random instruction> !dbg !13
  ret void, !dbg !14
}

; Function Attrs: nounwind uwtable
define void @middle() #0 {
entry:
  call void @inner()
  ret void
}

; Function Attrs: nounwind uwtable
define void @outer() #0 {
entry:
  call void @middle(), !dbg !21
  ret void, !dbg !18
}

I'm trying to figure out what the cause is; I tried building a simple
reproducer from scratch similar to the above, but I can't reproduce it.
I have attached a reproducer which is reduced by bugpoint; bugpoint did a
pretty good job reducing the code but did nothing to the debug info, which is
still huge, so I'm looking for some suggestions for making that smaller. I
tried removing the !llvm.dbg.cu from the module; this had mostly the effect I
was looking for, in causing most of the metadata to go away, but it also caused
the problem to no longer reproduce.

Any suggestions?

notes on the reproducer:
In the attached file, inner is
_ZNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEaSERKS5_,
middle is
_ZNSt3__115basic_stringbufIcNS_11char_traitsIcEENS_9allocatorIcEEE3strERKNS_12basic_stringIcS2_S4_EE,
and outer is
_ZN7testing7MessageC2Ev

To reproduce just run opt -inline bugpoint-reduced-simplified.bc
The actual error is
!dbg attachment points at wrong subprogram for function
!22062 = !DISubprogram(name: "str", linkageName:
"_ZNSt3__115basic_stringbufIcNS_11char_traitsIcEENS_9allocatorIcEEE3strERKNS_12basic_stringIcS2_S4_EE",
scope:
!"_ZTSNSt3__115basic_stringbufIcNS_11char_traitsIcEENS_9allocatorIcEEEE", file:
!519, line: 453, type: !15385, isLocal: false, isDefinition: true, scopeLine:
454, flags: DIFlagPrototyped, isOptimized: true, function: void
(%"class.std::__1::basic_stringbuf.130.422.2573.3290.4485.5202.6875.8070.8787.10460.10699.10938.11416.12372.17258"*,
%"class.std::__1::basic_string.5.297.2448.3165.4360.5077.6750.7945.8662.10335.10574.10813.11291.12247.17254"*)*
@_ZNSt3__115basic_stringbufIcNS_11char_traitsIcEENS_9allocatorIcEEE3strERKNS_12basic_stringIcS2_S4_EE,
declaration: !15384, variables: !22063)
void
(%"class.std::__1::basic_stringbuf.130.422.2573.3290.4485.5202.6875.8070.8787.10460.10699.10938.11416.12372.17258"*,
%"class.std::__1::basic_string.5.297.2448.3165.4360.5077.6750.7945.8662.10335.10574.10813.11291.12247.17254"*)*
@_ZNSt3__115basic_stringbufIcNS_11char_traitsIcEENS_9allocatorIcEEE3strERKNS_12basic_stringIcS2_S4_EE
  %cmp.i = icmp eq %"class.std::__1::basic_string.5.297.2448.3165.4360.5077.6750.7945.8662.10335.10574.10813.11291.12247.17254"* %__str_, %__s, !dbg !91052
!91057 = !DILocation(line: 2411, column: 14, scope: !91058)
!91058 = distinct !DILexicalBlock(scope: !59819, file: !32233, line: 2411,
column: 9)
!59819 = !DISubprogram(name: "operator=", linkageName:
"_ZNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEaSERKS5_",
scope: !"_ZTSNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEE",
file: !32233, line: 2409, type: !53554, isLocal: false, isDefinition: true,
scopeLine: 2410, flags: DIFlagPrototyped, isOptimized: true, function: void
(%"class.std::__1::basic_string.5.297.2448.3165.4360.5077.6750.7945.8662.10335.10574.10813.11291.12247.17254"*,
%"class.std::__1::basic_string.5.297.2448.3165.4360.5077.6750.7945.8662.10335.10574.10813.11291.12247.17254"*)*
@_ZNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEaSERKS5_,
declaration: !58684, variables: !59820)
LLVM ERROR: Broken module found, compilation aborted!

[Quuxplusone]

File is too bug to upload. get it at https://drive.google.com/file/d/0B_T-_5p_VNCtMmYwZG90eWpWU0k/view?usp=sharing

[Quuxplusone]

Derek - you describe the module verifier visiting the DISubprogram for the middle function. Yet you described the middle function as not having debug info. What's the deal there?

[Quuxplusone]

(In reply to comment #0)

I have attached a reproducer which is reduced by bugpoint; bugpoint did a pretty good job reducing the code but did nothing to the debug info, which is still huge, so I'm looking for some suggestions for making that smaller. I tried removing the !llvm.dbg.cu from the module; this had mostly the effect I was looking for, in causing most of the metadata to go away, but it also caused the problem to no longer reproduce.

Any suggestions?

Here's the rough algorithm I use.

1. Check if the original source will reproduce with -gline-tables-only. If so, this is a better starting point because it skips all the type info.
2. Try to reduce the preprocessed source with creduce and/or delta.
3. Hand-reduce the original source, simplifying constructs that creduce has trouble folding away (usually member function or template related), and go back to step 2.

This is often good enough. But if I really just have bitcode (e.g., from a crash in a huge LTO) or the repro is still large, I continue:

4. If there are multiple compile units in !llvm.dbg.cu, bisect which ones are necessary for the repro.
5. If there are multiple entries in the remaining compile unit's subprograms: array, bisect which ones are necessary.
6. Try to eliminate the rest of the arrays that the compile unit points at.

(Any time you make progress, you can round-trip to bitcode to garbage collect unreferenced metadata.)

[Quuxplusone]

another tip, once auto-reduction has stopped (& you've done all the obvious things that auto-reduction can't do - stripping templates and the like is one of the big ones creduce is still not very good at) you can try using alwaysinline attributes to force the inlining that the optimizer is doing anyway - then you can sometimes get back to an -O0 reproduction, jsut forcing the problematic inlining explicitly that way

[Quuxplusone]

Thanks for the tips. The actual functions are reasonably well reduced, i'll try bisecting down the CUs and see what I can come up with. If that fails i'll go back to the source.

@David: When the inner function gets inlined into middle, its instructions keep their debug info.

[Quuxplusone]

There shouldn't be a DISubprogram for 'middle' then - so I'm not sure what the Verifier's visiting.

Perhaps it visited the instructions of 'middle' even though there was no DISubprogram - then compared the Function on the DISubprogram it reached from the instructions' debuglocs to the function the instructions were in and failed? That would be a bug in the verifier - it shouldn't inspect functions without DISubprograms.

[Quuxplusone]

Hm, interesting. in my repro there is a DISubprogram for the middle function, but the function itself has no dbg attachments. Maybe that is the underlying unexpected thing?

[Quuxplusone]

(In reply to comment #7)
> Hm, interesting. in my repro there is a DISubprogram for the middle
> function, but the function itself has no dbg attachments. Maybe that is the
> underlying unexpected thing?

Yes, that's likely to be a bad thing. The current requirement is that the call
instruction must have a debug location if the caller and callee have debug info
and the call site is inlinable.

If that requirement is violated => badness.

If you can create a small test case where that happens, that'd be great.

[Quuxplusone]

This file is the result of a bitcode link of some files which have debug info (libc++) and some which do not (all the user files).

I'll try to see if the invariant you mentioned is getting violated in that process. It's also possible that one of the passes we run is dropping something it shouldn't.

[Quuxplusone]

In my linked bitcode file, there are some calls to middle which have dbg
attachments (in this case from libgtest, which is built with debug info), and
some calls which do not (other user files built without debug info).
Middle is a comdat function, and the one included in the link output has no dbg
attachments (presumably the copies from those files that were built with debug
info had the attachments). There is also a DISubprogram for middle. I guess if
the callee (middle) has no dbg attachements it should not have a DISubprogram?
I guess I should look at the behavior of the linker in that case?

[Quuxplusone]

Ah, that does sound like it could be an LTO bug:

Build one object (bitcode) file without debug info
Build one object (bitcode) file with debug info.

Have the same comdat function in both.

Choose the non-debug-having implementation of the function, while keeping the
subprogram debug info from the debug-having implementation.

Duncan, does that sound like a thing that can/would happen?

Once we put subprogram debug info as a function metadata attachment, would that
fix this bug?

[Quuxplusone]

(In reply to comment #11)
> Ah, that does sound like it could be an LTO bug:
>
> Build one object (bitcode) file without debug info
> Build one object (bitcode) file with debug info.
>
> Have the same comdat function in both.
>
> Choose the non-debug-having implementation of the function, while keeping
> the subprogram debug info from the debug-having implementation.
>
> Duncan, does that sound like a thing that can/would happen?

Yes, that sounds possible.

> Once we put subprogram debug info as a function metadata attachment, would
> that fix this bug?

Yes, I think so.  That's tracked by bug 23367, which I'm hoping to get back to
soon (along with a bunch of follow-ups).