[GR-38416] Implement debug location info

[adinn]

Feature request

Extend debug info generation to support information about the location of parameter and local variables

[vvlevchenko]

For IntelIJ Graal VM debugger plugin needs, we interested in sooner applying this PR and can provide any required assistent.

[olpaw]

@vvlevchenko love to hear that you guys are working on supporting Graal VM native-image debugging :grin:

I will start reviewing @adinn's PR this week.

[adinn]

@vvlevchenko I am extremely sceptical that there will be much benefit using this capability from a Java IDE that is oriented around a source line view of execution. That works with the JVM and JDWP because code executed under a debugger is deoptimized and run interpreted. This means that when you stop at a line and/or step to a new line you can be very sure that what gets executed is the precise code contained on that line and nothing else.

This is going to be very far from the case when you are executing compiled native image code, even code that is compiled using the economy compilation suite. Infopoints output by the compiler that are used to build debug info have a very approximate relationship to the location in compiled code that they are attached to.

So, when you are looking at some expression on line N of a method where the debugger has stopped you may find yourself in one of 4 main cases

1) none of the code on that line has actually been executed 2) some but not all of the code on that line has been executed (but not necessarily in some easy to follow depth first left to right order of evaluation) 3) all of the code on that line has already been executed 4) there is no code from that line to execute because it has all been optimized away (even though the compiler has still inserted that line number into the infopoint output)

Actually, that belies the true complexity of the case because what you will frequently also find is that code from previous or succeeding lines may or may not already have been executed too. That can also include code that belongs to inlined methods which has been promoted before or delayed until after the current line's code. Clearly, the problem will be worse with full optimization but all the above things can happen even with the economy optimization suite.

You might wonder what is the point of the debug info then? Well, if a) you can view and step through the machine code and b) you understand something about how the compiler operates ,then the debug info view of the code does help enormously to work out how the machine code was generated and, as a result, isolate some potential native image bugs. Of course, that is only really possible if you debug using a low level debugger like gdb on Linux. However, it's of limited use and the pre-conditions I outlined above mean it is not an option for most developers.

It certainly does not provide a comparable alternative for debugging problems with a Java app under development when compared with running and debugging on the JVM. So, the idea that devs will develop, debug and deploy on native image alone is, in my view, highly misguided. My belief has always been that debug info will only be of help in certain very specific use cases. Here are the two I have come up with so far:

1) Your app displays different behaviour when run as a native image vs when run on the JVM because of a bug in the GraalVM native toolchain. 2) Your app displays different behaviour when run as a native image vs when run on the JVM because you have introduced substitutions and you need to check that they have been applied and are doing what you expect.

Both of these can only be resolved by debugging of the generated code because the behaviours in question will be specific to GraalVM. Also, in both cases you are almost always going to have to debug with full optimization because a fully optimized app is the app you are going to deploy and hence is the one you need to be sure is correct. You might be lucky that you can spot and remedy the bug in an unoptimized native image but that is probably not the most likely case -- especially for use case 1.

So, to sum up: unless you plan to add machine instruction level debugging to IntelliJ I think you are not going to be able to use the debug info capability to offer any real benefit to developers, expert or otherwise. My opinion is that at best you will have a nice toy feature. Your mileage and your willingness to invest effort may vary, of course.

[vvlevchenko]

@olpaw thank you @adinn thank you for such deep detailed explanation. I totally understand your scepticism at this stage, but please consider our activity as early adoption. Before joining IDE team I was participated in Kotlin/native project and was responsible for debug info generation and debugger support in term of code generation (not IDE support, so in this project I'm going to gain experience from other side of the world :) and compose both words possibilities: language support from IDE side and rich dwarf debug info description) . The last point gives me some confidence in in our offer of assistance. Of course Kotlin Native case is much easier one because of its LLVM nature and easy LLVM support for code and debug info generation.

[fniephaus]

Thanks for your work, @adinn! This will greatly improve the debugging experience.

@vvlevchenko feel free to reach out to me if you like to discuss how GraalVM native-image debugging could be integrated into IntelliJ. :smiley: