Fix Debug Information On Modern LLVM

[ErikMcClure]

This fixes source line debug information for all LLVM versions. However, the method used to generate a stacktrace stops working past LLVM 8 and has no equivalent in modern LLVM (that I can find), so it has been conditionally removed.

While the stack traces exist and breakpoints can be set, Terra never emitted debug info for variables. As a result I'm pretty sure no significant information is being lost, but there isn't a lot of debug information in the first place and I have no way of testing this because getting LLVM3.5 to build with terra is a nightmare even on linux.

A hack was used to preserve metadata information for deferred functions, which may result in incorrect debug information, but there are no tests for debug information, so I have no idea if this works reliably. The only way to test this is for people who currently use the debug information (although there doesn't seem to be much of it) to test this change and report if it is correctly preserved.

[elliottslaughter]

Thanks for working on this! This is definitely something that will be useful to a lot of people.

Just FYI, I get a link error on Linux when I attempt to build against LLVM 9:

[100%] Built target TerraLibraryShared
../lib/libterra_s.a(tcompiler.cpp.o): In function `llvm::MetadataTracking::track(llvm::Metadata*&)':
tcompiler.cpp:(.text._ZN4llvm16MetadataTracking5trackERPNS_8MetadataE[_ZN4llvm16MetadataTracking5trackERPNS_8MetadataE]+0x33): undefined reference to `llvm::MetadataTracking::track(void*, llvm::Metadata&, llvm::PointerUnion<llvm::MetadataAsValue*, llvm::Metadata*>)'
collect2: error: ld returned 1 exit status

I don't see this with LLVM 6 so it seems to be a purely LLVM 9 issue.

I'll try to actually test the debug info next....

[elliottslaughter]

Seems to be working on Linux with LLVM 6:

local c = terralib.includec("stdio.h")

terra main()
  var x = 1
  var y = 2
  var z = x + y
  c.printf("x %d y %d z %d\n", x, y, z)
end

terralib.saveobj("test-terra-debug-info", {main=main})

$ ./bin/terra -g ../../test-terra-debug-info.t
$ gdb --args ./test-terra-debug-info
(gdb) b main
Breakpoint 1 at 0x400534: file ../../test-terra-debug-info.t, line 7.
(gdb) r
Starting program: /scratch2/eslaught/terra-debug-info/build/test-terra-debug-info 

Breakpoint 1, $main () at ../../test-terra-debug-info.t:7
7         c.printf("x %d y %d z %d\n", x, y, z)
(gdb) list
2
3       terra main()
4         var x = 1
5         var y = 2
6         var z = x + y
7         c.printf("x %d y %d z %d\n", x, y, z)
8       end
9
10      terralib.saveobj("test-terra-debug-info", {main=main})

To a first approximation, Terra (master branch) with LLVM 3.5 seems to behave the same:

$ ./terra -mg ../test-terra-debug-info.t
$ gdb --args ./test-terra-debug-info
(gdb) b main
Breakpoint 1 at 0x400534: file ../test-terra-debug-info.t, line 7.
(gdb) r
Starting program: /scratch2/eslaught/terra-debug-info/test-terra-debug-info 

Breakpoint 1, $main () at ../test-terra-debug-info.t:7
7         c.printf("x %d y %d z %d\n", x, y, z)
(gdb) list
2
3       terra main()
4         var x = 1
5         var y = 2
6         var z = x + y
7         c.printf("x %d y %d z %d\n", x, y, z)
8       end
9
10      terralib.saveobj("test-terra-debug-info", {main=main})

I agree we would ideally like to have automated tests. If you have ideas on how to do that, I'm all ears.

[ErikMcClure]

The LLVM 9 automated tests passed on linux, which is... weird. I am in the middle of building LLVM 9 on windows to test with (and I also need to fix a mistake that broke LLVM 3.5), but if I can't reproduce the linker error I'll have to dig around for where that function is supposed to be defined. It's possible certain LLVM 9 builds aren't linking a required .lib file.

As for an automated debug test, why not just do exactly what you tested up there and do an exact string comparison on the result? It'd be a bit obnoxious to maintain if someone changes the structure of the emitted text, but as long as there's only one test like that it shouldn't be too difficult.

[elliottslaughter]

I suppose it could be something about the old Linux box I was testing on. I don't have my normal Linux box available at the moment. The above was on Ubuntu 14.04.

For the test, that seems fine. If you want to code it up, go ahead, otherwise I may do so if I get the time. There is also test/testdebug.t which seems to rely on the built-in stack trace capability, but I don't know how brittle it is.

[ErikMcClure]

So, I went and looked at testdebug.t, but it relies on the JIT debugging part that had to be removed. Without it, I'm not sure how to extract the debug information programmatically in a platform-independent manner. Assuming this passes all tests, I think we may just have to accept this pull request as is and try to recover additional debug information later.

[elliottslaughter]

Thanks!