No IN-VM options for the ghidra debugger

[XdotCore]

Describe the bug I have no IN-VM options in the ghidra debugger

To Reproduce I do not know it just happened between closing and opening it over night.

Expected behavior IN-VM options to appear in the debugger options

Screenshots

Environment (please complete the following information):

• OS: Windows 11 x64
• Java Version: Eclipse Adoptium 17.0.11.9
• Ghidra Version: 11.1.2
• Ghidra Origin: official GitHub distro

Additional context I'm not sure what else I can add, I just hope it's just user error or something,

[d-millar]

@XdotCore Hi and sorry for the confusion! Am not sure what exactly triggered the change (importing a new tool maybe?) but basically looks like you're looking at the new python-based implementation of the debugger. We are migrating away from the old Java-to-java version to a python-server-java-client version. That said, right now, both are still available if you'd like to revert to the model you're most comfortable with or if you'd like to compare and contrast the two models.

In the old version, the connections were handled by the DebuggerTargetsPlugin and the objects were displayed as a tree using the DebuggerObjectsPlugin. The IN-VM and GADP connections were added to the tool bar under Debugger->Debug by the former. In the new version, the connections are handled by the TraceRmiPlugin and the objects are displayed, again in tree-form, by the DebuggerModelPlugin. The python-based connections are added to the tool bar via the former under Debugger->Configure and Launch.

There are a number of differences between the two versions, most obviously you'll need Python 3 installed for the new versions. For gdb- and lldb-based debugging, the version of python defined by the PYTHONPATH (via the initila dialog) needs to be consistent with that compiled into those executables. You can check this in gdb by issuing "pi" to switch to the python interactive shell, running "import sys", and then "sys.version". The lldb version of "pi" is "script". There are a number of more subtle differences between the two configurations, which I'm happy to discuess, and, of course, a number of reasons why we're switching.

Let me know which direction you decide to go, and happy to assist whatever you decide.

[XdotCore]

Thank you for the response. I had reimported the debugger plugin due to issues with it, so that is probably what happened.

I am currently using the dbgeng option because I assume the application was compiled with msvc since it is a windows application using directx and WinMain. Is this a correct assumption?

When I use the dbgeng option, I have the application run and attach to the debugger, however it keeps breaking until it stops running entirely. Just to see how it ends takes several minutes of using the g command in windbg, and the debugged application freezes entirely with just a black window.

I initially made this issue because I wanted to try and see if I could get it working with the IN-VM option or if it would still have the same issue. Is it possible to revert to this version of the tool to test it out?

[d-millar]

@XdotCore Yep, apologies for not being a little clearer: If you configure the Debugger tool (File->Configure) to include the DebuggerTargetsPlugin & DebuggerObjectsPlugin, the original functionality will be there. (It may be a little confusing to have both - I might remove the TraceRmiPlugin and DebuggerModelPlugin, as well.) Dbgeng is the right choice. Does the application behave the same way if you debug it with windbg proper? What is the specific reason for the break? You may want to disable various exceptions that are caught by default.

[XdotCore]

Adding the DebuggerTargetsPlugin and DebugerObjectsPlugin got me closer. This game has steam drm so running not through steam will have it restart, meaning that I must attach to it at runtime, and I don't know how to do that with the new python method. However, when I attach it, the debugger gets all the dlls and stuff, and it looks good until I see the breakpoints are inneffective, even with the exe mapped, and looking in the memory views shows all 00's. I am using the Debugger Targets tab to create a dbgeng.dll instance for this, however dbgmodel.dll does not bring up an interpreter.

When I use windbg proper, both the windows kits and the microsoft store one, I can view the memory and add breakpoints after attaching to the process. When I use the IN-VM options, ignoring the eventual steam restart, it shows that the errors are visual c++ exceptions.

[d-millar]

OK, after a lot of searching, I have to admit with some dismay that we never implemented a batch file for attaching to a running process for the python version. I am adding a ticket right now - my apologies.

Leaving the python versions aside, am guessing the issue with the dbgmodel w/o an interpreter window is probably a DLL mismatch. This is actually one of the reasons we moved to the python version, although it doesn't completely solve the problem. The "easiest" thing to do is to figure out which version of the DLLs, i.e. dbgeng/dbgmodel/dbgcore/dbghelp, is being run (say, with SysInternals' ProcExp64),. Then grab the contents of the directory that contains them and drop them in the active JDK's bin directory. (I know - that seems marginally insane, but it's the only solution that works consistently.)

Ignoring dbgmodel for the moment, however, your real problems, as I understand them, are missing dynamic memory and a seemingly infinite string of Visual C++ exceptions. The latter is pretty easy to fix. You can either issue the normal windbg commands to disable breaking on specific exceptions from the Interpreter, or, in the Objects Tree, open the process node, then Debug->Exceptions->Visual C++ Exception. Highlight "Execute" and use "T" to toggle it to "Ignore". Maybe also, highlight "Continue" and toggle it to "Handled".

On the memory front, a couple of questions. Am guessing from what you said above that you have already used the Modules view to "Map Module to " whatever your static executable is called, yes? Does the Regions view have a full list of objects? If not, I might also try using "Force Full View", although I would think that, if this is empty, you wouldn't see any bytes at all (versus zeroes). There may be some other helpful hints in the Debugger Help section, but I may have to think about this one a little more if you've already done the things mentioned. As you may have already guessed, the breakpoint issue is almost certainly a result of the memory problem.

[d-millar]

tmp.zip

OK, should you wish to try the python versions, have attached a zip file with three files: local-dbgeng-attach.bat should be copied to Ghidra\Debug\Debugger-agent-dbgeng\data\debugger-launchers local-dbgeng-attach.py should be copied to Ghidra\Debug\Debugger-agent-dbgeng\data\support commands.py should be copied to Ghidra\Debug\Debugger-agent-dbgeng\src\main\py\src\dbgeng

Then, run "gradle assemblePyPackage" from the Ghidra root dir. Again, apologies for the oversight - these will be included in 11.2.

[XdotCore]

I tried the new python attach script and it gets an uncaught exception about the start address needing to be less than the end address, where the end address is 0.

The dynamic listing view and memory viewboth show all 00's. Scrolling in the dynamic listing view gives the below uncaught exception. Trying to go to the base address of the exe in the memory view gives "Address [base address] not in trace".

Uncaught exception

Start address must be less than or equal to end address: Start ram5068::621c0000 end = ram5068::00000000 java.lang.IllegalArgumentException: Start address must be less than or equal to end address: Start ram5068::621c0000 end = ram5068::00000000 at ghidra.program.model.address.AddressSet.intersectRange(AddressSet.java:757) at ghidra.program.model.address.AddressSet.intersectRange(AddressSet.java:486) at ghidra.trace.database.program.AbstractDBTraceProgramViewMemory.intersectRange(AbstractDBTraceProgramViewMemory.java:484) at ghidra.app.util.viewer.util.VerticalPixelAddressMapImpl.getAddressSet(VerticalPixelAddressMapImpl.java:142) at ghidra.app.util.viewer.listingpanel.ListingPanel.notifyDisplayListener(ListingPanel.java:509) at ghidra.app.util.viewer.listingpanel.ListingPanel.layoutsChanged(ListingPanel.java:504) at docking.widgets.fieldpanel.FieldPanel.notifyScrollListenerViewChangedAndRepaint(FieldPanel.java:1153) at docking.widgets.fieldpanel.FieldPanel.scrollView(FieldPanel.java:149) at docking.widgets.fieldpanel.FieldPanel.mouseWheelMoved(FieldPanel.java:1420) at docking.widgets.fieldpanel.FieldPanel.lambda$new$0(FieldPanel.java:120) at java.desktop/java.awt.Component.processMouseWheelEvent(Component.java:6709) at java.desktop/java.awt.Component.processEvent(Component.java:6393) at java.desktop/java.awt.Container.processEvent(Container.java:2266) at java.desktop/java.awt.Component.dispatchEventImpl(Component.java:4996) at java.desktop/java.awt.Container.dispatchEventImpl(Container.java:2324) at java.desktop/java.awt.Component.dispatchEvent(Component.java:4828) at java.desktop/java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4948) at java.desktop/java.awt.LightweightDispatcher.processMouseEvent(Container.java:4604) at java.desktop/java.awt.LightweightDispatcher.dispatchEvent(Container.java:4516) at java.desktop/java.awt.Container.dispatchEventImpl(Container.java:2310) at java.desktop/java.awt.Window.dispatchEventImpl(Window.java:2780) at java.desktop/java.awt.Component.dispatchEvent(Component.java:4828) at java.desktop/java.awt.EventQueue.dispatchEventImpl(EventQueue.java:775) at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:720) at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:714) at java.base/java.security.AccessController.doPrivileged(AccessController.java:400) at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:87) at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:98) at java.desktop/java.awt.EventQueue$5.run(EventQueue.java:747) at java.desktop/java.awt.EventQueue$5.run(EventQueue.java:745) at java.base/java.security.AccessController.doPrivileged(AccessController.java:400) at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:87) at java.desktop/java.awt.EventQueue.dispatchEvent(EventQueue.java:744) at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203) at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124) at java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113) at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109) at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101) at java.desktop/java.awt.EventDispatchThread.run(EventDispatchThread.java:90) --------------------------------------------------- Build Date: 2024-Jul-09 1157 EDT Ghidra Version: 11.1.2 Java Home: C:\Program Files\Eclipse Adoptium\jdk-21.0.4.7-hotspot JVM Version: Eclipse Adoptium 21.0.4 OS: Windows 11 10.0 amd64

Dynamic listing view screenshot at exe base

The regions view is full of objects. Toggling full view gives a similar error to above.

Regions view screenshot

[d-millar]

@XdotCore The first error I'm aware of - will take a bit more time to fix but can safely be ignored. You said the dynamic listing view shows all 00's but you screenshot show a non-zero value, so I might need some clarification on that. Appears to be collapsed in the listing - can you expand the stuff at 0xb2000?

[XdotCore]

I apologize, it seems it works more than I had found. I tried again, and set a new breakpoint and it is actually able to break, map, and step through the code, although using the watch gives only 0/\/??. The location in the screenshot (you are right, it is not 0, my mistake) cannot be expanded. However, it does read there and where it breaks, but only that byte of memory or the bytes of memory for the instruction and the ones above it are able to be read, but the ones below are shown as addressed to 0x0, and the only way to read them is to move to that location or below in the mapped file. Additionally, because of the unread memory appearing, the known exception appears every time I move to a different location.

Example screenshot of the breakpoint with below memory not being read

[d-millar]

No problem - "works more" is something I like to hear. Regarding the memory issue, I must admit I've never seen this before and, honestly, I'm not sure what's going on. Is there any chance the target is decrypting instructions on-the-fly or reading in that memory on-demand? (I've put a ticket in for the address exception thing, but may be a while before we have a solution...)

[XdotCore]

I assume it is neither decrypted or on-demand. The executable is not encrypted, I am able to view the assembly within the CodeBrowser without issue. Additionally, I am able to view the memory with the program stopped and not stepping, just clicking on different parts of the assembly/c++ loads the associated memory, suggesting to me it is not being loaded on-demand by the program.

Regardless, I am able to view the memory in both versions of windbg proper without issue.

[d-millar]

FYI, we've run down (almost) the "intersect range" address error. Unfortunately, the only guy who understands the logic surrounding the bug well enough to fix it @nsadeveloper789) is out-of-town until next Friday. Will keep you posted. Thanks again for pointing it out - definitely a serious bug.

[d-millar]

Good points re memory, am trying to see if I can re-create the issue. Am guessing this only happens with this target, though, yes? i.e. you don't see the issue if you attach to, say, notepad.exe?

[d-millar]

OH, interesting - good news, I AM able to re-create your memory issue! And, bad news, it's directly related to the address issue! Will dig in some more, but, as noted, may be out of my wheelhouse....

[XdotCore]

Yep, the issue also happens in notepad (the windows 11 WindowsApps one), firefox, and a flash player with embedded flash game.

[d-millar]

OK, I understand the issue - working on a fix right now.

[d-millar]

Very short term solution: when you bring up the local-dbgeng-attach dialog, uncheck dbgmodel.

[d-millar]

arch.zip

OK, replace the existing arch.py in Ghidra\Debug\Debugger-agent-dbgeng\src\main\py\src\ghidradbg with the zipped copy, run "gradle assemblePyPackage", cross your fingers and re-run. With a little luck, this should fix the main problem and allow you to run with dbgmodel checked (or unchecked).

[XdotCore]

It works! I haven't ran into a single error, the memory is mapped and read, breakpoints and watches work, thanks so much.

One last thing, how do I get classes to work in the watch so that I can debug the values of the member variables?

[d-millar]

Excellent! Re watches and class member variables, I will probably have to take that to the team. Don’t have a ready answer for you. @nsadeveloper789 is definitely your best bet in this, but, as mentioned, he’s out next week. In the meantime, will ask around / see if anyone else in the team has some insight.

[nsadeveloper789]

Oof. A lot can happen in two weeks! Okay, so if I understand things correctly up to this point, we've switched over to using the Python-based dbgeng, using a new/custom script to attach to an existing process. Two issues remain at hand?

1. There's a pervasive memory address-range issue that needs immediate resolution.
2. You have a question about class member variables in the Watches panel.

Not sure exactly when I'll get to 1, but it certainly has very high priority. I haven't yet made my way through the 2-week backlog.

Re 2/ There's no direct support for C++ class members, but I think it can be done, albeit not automatically, using Sleigh and your knowledge of the underlying structures / memory layout. I'll provide this example for a simple C struct, and you should be able to extrapolate to your C++ class member case. Take the following C struct:

struct my_struct {
    int a;
    int b;
}

Assuming an int is 4 bytes in your ABI, and you have a pointer to a my_struct in RCX, you could display b as follows:

*:4 (RCX+4)

The *:4 means to deref a 4-byte value, because int b is 4 bytes. The (RCX+4) part denotes the address of b which is the base address of the struct RCX plus the number of preceding bytes (offset). There is just the 4-byte int a preceding. Sleigh supports nesting these expressions, too, so supposing the pointer to the my_struct were at stack offset 8, you could deref field b as follows:

*:4 ((*:8 (RSP+8)) + 4)

I've contemplated a C/C++-like syntax for Watches as an alternative to Sleigh when the appropriate types are marked up in the Program database, but I've not been sufficiently motivated to pursue it. Who really wants to create yet another C/C++ parser for Ghidra?

[XdotCore]

Thanks for the response. For 1, @d-millar's fix for the memory range issue has fixed it for me, I have been using the debugger a lot and haven't found an issue related to it since the fix. For 2, I figured that out myself, and have simply set that memory in the dynamic listing to the class type just like you would in the program disassembler.