Debugger - The resulting target process has no mapping to the static image

[Triangle345]

Describe the bug Attempting to launch a 32 bit Windows binary in the debugger under dbgeng locally IN-VM causes this error:

Breakpoints do not work as expected.

To Reproduce Steps to reproduce the behavior:

1. Go to https://github.com/albertzsigovits/flare-on-challenges/tree/master/2016/Flare-On3_Challenges/1
2. Download Challenge1.exe and import into Ghidra, analyze, etc.
3. Open Challenge1.exe (after analysis) in the Ghidra debugger by dragging it from project view to debug icon.
4. I have tried several debug varieties but to be specific, lets go with "dbgeng locally IN-VM" for this example.
5. The above error will pop up at around 83%. Click ok to close the pop up.
6. Set a breakpoint (e.g. right before the password prompt @ offset 0x0143f ).
7. Click the green "continue/resume" button and notice that no breakpoint ever gets hit.

Expected behavior Setting a breakpoint before the password prompt at offset 0x0143f should hit inside Ghidra while debugging this binary.

Screenshots Here is a screenshot right after the error message:

Attachments See above link

Environment (please complete the following information):

• OS: Win 11 22H2
• Java Version: Adopt JDK 17.0.7
• Ghidra Version:10.3
• Ghidra Origin: official github

[d-millar]

@Triangle345 OK, so the error message (and the fact that breakpoints don't work) is telling you that the debugger has no means of mapping the static memory associated with your imported Ghidra program with the dynamic memory in the executing process if their names do not match. Why is this? Well, could be a number of reasons, e.g. the name of the executable was changed after compilation, the compiler was instructed not to include the name, the name of the executable was changed on disk, the executable was imported into Ghidra using a different name, the name of the program was changed after import, etc. Probably the exact reason is unimportant. In your case, challenge1.exe is known only as "image00000000_00e00000", as indicated in the Module list during execution. As a result, setting a breakpoint in the Static Listing or setting the breakpoint through any means at the static address, e.g. 0x4000143f, simply tells the debugger, "when this address gets resolved, add this breakpoint". (In the Breakpoint listing, this is indicated by the fact that the breakpoint only appears in the top half of the view and the icon is grayed out.) For better or worse, resolving the address is, however, on you.

The easiest way to do this is to right-click on the entry in the Modules list (while challenge1.exe has focus in the Static Listing) and select "Map Module to challenge1.exe". Alternatively, you could have selected "Map the current trace to various programs manually" from the Modules tool bar and entered the relevant address ranges. There are a few other ways to do this, which are described in the new Debugger Course documentation. If you haven't already perused this, I recommend it highly. Once you have reolved the memory mapping for challenge1.exe, setting the breakpoint works as you'd expect. You could also have enabled the breakpoint if you set it before resolving things.

Hope this helps - let us know if you have further questions.

[Triangle345]

@d-millar thanks for taking the time to explain this. Using "Map module to challenge1.exe" did work and my breakpoint was hit. However; I notice that the name is still image000... instead of challenge1.exe. After mapping it, I'd assume that the name would be updated to challenge1.exe?

[d-millar]

That would be possible, I suppose, if that were under our control, but we can’t easily modify the OS’s name for the running executable, nor the OS structures behind the module list. At best, we would just be masking what’s actually happening to improve the user’s view, but that creates its own set of problems and overall we’ve decided isn’t the best thing to do.

[Triangle345]

Thanks for the info. I understand that modifying the OS module list and executable name(s) is out of scope for Ghidra. On the contrary, I'm not seeing how Ghidra represents the newly mapped module to the user. It seems that after I click "Map module to challenge1.exe" something changes but there is no status or visible flag of this in the Ghidra UI? Maybe I'm not looking in the right place? Is there anywhere in the UI that shows manual module mappings?

[d-millar]

Ah, well, that’s a valid point. Debugger->Static mappings list the current set of mapping, but it’s not among the windows visible by default. It’s also in a sense implicit in the fact that navigation in the Static listing will move the Dynamic listing to addresses associated with the dynamic name, but maybe that’s not obvious enough. We could perhaps add a field to the Modules display. I will bring it up for discussion. I think so far we’ve assumed that (a) this was a somewhat unusual case, and (b) the user would remember having taken the action, but I will concede it’s nice to have visual confirmation that actions you take have an actual effect.

[Triangle345]

I agree with you that it is not at all obvious that there was a static module mapping. Another option would be to append the name field with "(module_name)" such as "image000_00e (challenge1.exe)". If both the object and modules view were updated to include this it might be more clear as to what happened after static mapping.

Figured that after seeing this error message I would want to know if a module was successfully mapped and that it fixed the issue. Currently, there is poor indication of these things (we only have the static view to see what is mapped and no indication at all to show that we are now correctly synchronized.

Anyways, I appreciate the attention given to this issue. My problem has been resolved with the guidance given. I can close this issue or leave it open if the admins wish to follow through with improvements discussed.

[nsadeveloper789]

I like this idea, though there's a bit of nuance I'll have to work out. Technically, the Static Mappings do not have to align with or cover entire modules. Still, I think that'll be easy enough, e.g., an asterisk or some color to indicate partial mapping (this should be a rare occurrence). I'm also considering a separate column for the static mapping info, rather than appending to the Name column.... Thoughts?

[Triangle345]

You could always do both. Add another column and append to the name field. I figure that its justified to modify the module name since that is the original intent (you know, to remap the module). Appending the name field will also allow it to be noticeable in the objects view whereas if it were just a column, it would not display in objects.

[nsadeveloper789]

Appending it to the Objects view is a whole different ball game. Admittedly, I hadn't considered mappings in the Objects view; however, the names there are currently controlled by the back-end debugger (dbgeng in your case). There is no mechanism for communicating module mappings from Ghidra to that back-end, and adding it would be no simple feat. Aside from control commands and memory/register requests, the path from the back-end to the GUI is mostly unidirectional.

In my mind, the question was purely of presentation in the Modules panel. The underlying model/back-end and database would be unaffected. Either the "Name" column would draw data from both the Modules and StaticMappings portions of the database, or there would be a new "Mapped" (or something) column that draws data from the StaticMappings portion. So far, our internal discussions favor the separate column.

I usually hesitate to "do both." While it seems an obvious compromise, it really isn't so simple. It often results in toggles to turn one or the other off, which just leads to multiplication of state and test cases.

[gscalise]

I'm having the same issue on MacOS 12.6.5 and LLDB 14. Tried it on an M1 Pro Mac and on an Intel Mac. Tried it on Ghidra 10.1, 10.2, 10.3 and building the 10.4 myself, and I've had the exact same result every time.

In my case, the binary I'm trying to work with launches and appears in Ghidra along with /usr/lib/dyld. The PC is in /usr/lib/dyld memory space, and no modules or sections appear on the Modules window. Entering image dump sections in the Interpreter window does show the sections for the two modules. Entering image list also shows the two modules.

I've tried doing the workaround that @d-millar mentioned a few times (refreshing the modules in the Objects window), but to no avail. I'm not sure if I'm doing something wrong.

One thing to note is that the logs do show messages like these:

Object <LldbModelTargetModuleImpl: path=Sessions[918c].Modules[easy_one] model=agent.lldb.model.impl.LldbModelImpl@21827349 schema=Module> is missing required attributes [_range] of schema schema Module {
  ifaces = [Module ]
  elements(resync NEVER) = {} default VOID
  attributes(resync NEVER) = {_range=<attr name=_range schema=RANGE required=true fixed=false hidden=true>\, _module_name=<attr name=_module_name schema=STRING required=true fixed=false hidden=true>\, _modified=<attr name=_modified schema=BOOL required=false fixed=false hidden=true>\, _short_display=<attr name=_short_display schema=STRING required=false fixed=false hidden=true>\, _order=<attr name=_order schema=INT required=false fixed=false hidden=true>\, _kind=<attr name=_kind schema=STRING required=false fixed=true hidden=true>\, _display=<attr name=_display schema=STRING required=false fixed=false hidden=true>\, _value=<attr name=_value schema=ANY required=false fixed=false hidden=true>\, _type=<attr name=_type schema=STRING required=false fixed=false hidden=true>\, Sections=<attr name=Sections schema=SectionContainer required=true fixed=true hidden=false>\, Symbols=<attr name=Symbols schema=SymbolContainer required=true fixed=true hidden=false>\, BaseAddress=<attr name=BaseAddress schema=ADDRESS required=false fixed=false hidden=false>\, ImageName=<attr name=ImageName schema=STRING required=false fixed=false hidden=false>\, UUID=<attr name=UUID schema=STRING required=false fixed=false hidden=false>\, Len=<attr name=Len schema=STRING required=false fixed=false hidden=false>} default <attr name= schema=VOID required=false fixed=false hidden=false>
}

I've tried debugging this a few times and noted it seems to be a race condition between the UI and the LLDB module, but I don't know if these two issues are related.

[nsadeveloper789]

@gscalise, when you expand the Modules node in the Objects pane, do you see the modules at all, or is the list empty? With that error in the log, I'd expect to see the modules, but they'd be missing their address ranges.

[gscalise]

@gscalise, when you expand the Modules node in the Objects pane, do you see the modules at all, or is the list empty? With that error in the log, I'd expect to see the modules, but they'd be missing their address ranges.

The modules do appear in the Objects pane, yes. It's the Modules pane that doesn't show anything.

[d-millar]

Just to clarify, when you saying you're "having the same issue," do you mean you have bytes in both the Dynamic and Static listings, but they are not tracking execution together? An empty Modules pane would suggest to me something completely different. Are there other windows that are empty, e.g. Registers, Regions, etc.? Also, when you set up the LLDB agent, did you use the macos_debugger_lldb_build_from_brew.sh or did you follow the instructions in InstructionsForBuildingLLDBInterface.txt? If the latter, did each step succeed? Do you have the same issues debugging something relatively mundane, e.g. HelloWorld example?

[gscalise]

Just to clarify, when you saying you're "having the same issue," do you mean you have bytes in both the Dynamic and Static listings, but they are not tracking execution together?

Yes, exactly.

An empty Modules pane would suggest to me something completely different. Are there other windows that are empty, e.g. Registers, Regions, etc.?

The other windows are fine. Registers, Interpreter, etc.

Also, when you set up the LLDB agent, did you use the macos_debugger_lldb_build_from_brew.sh or did you follow the instructions in InstructionsForBuildingLLDBInterface.txt? If the latter, did each step succeed?

I tried with both.

Do you have the same issues debugging something relatively mundane, e.g. HelloWorld example?

Yes.

[gscalise]

I've been able to make it work by refactoring the bindings in Debugger-swig-lldb to use the latest lldb that comes in my Mac instead of installing llvm@14 with Homebrew.

The version I have is

lldb --version
lldb-1400.0.38.17
Apple Swift version 5.7.2 (swiftlang-5.7.2.135.5 clang-1400.0.29.51)

I've had to make a few changes to support the refactoring of TraceInstructionControlFlowType -which used to be a FLAG_ENUM into InstructionControlFlowKind that was merged to llvm in this commit. These seem like fairly straightforward changes, and I can put them in a branch, but I suppose that merging such a breaking change would need a release strategy from your side.

[d-millar]

Hmmmmm. Maybe try the following: in the Objects window, in the Menu pull-down on the far right (top corner), unselect Maintenance->Hide Intrinsic Attributes. Then reopen Session[xxx].Modules[easy_one] and look for an attribute called "_range". Should look like "_range : [large_hex_number, larger_hex_number]". I'm not sure if the error you mentioned is significant or not - this might provide a hint.

[d-millar]

Also, not sure if you are comfortable debugging the source build in Eclipse, but you could try breaking in the "invoke" method for LldbListModuleSectionsCommand.java to see if module.GetNumSections is returning something sane.

[d-millar]

@gscalise Wow, apologies - I missed the message above. Very glad to hear you were able to fix the issue, and anything you care to share regarding the patch would be greatly appreciated. Upgrading our SWIG files to the latest stable release has been on my TODO list for some time, so pointers including the reference above are a definite incentive for me to bump the priority up. Thanks!

[gscalise]

anything you care to share regarding the patch would be greatly appreciated.

Will do! :)

you could try breaking in the "invoke" method for LldbListModuleSectionsCommand.java to see if module.GetNumSections is returning something sane.

It's returning 7, and it's inserting the sections correctly in the result. The completion of the future in agent.lldb.model.impl.LldbModelTargetModuleSectionContainerImpl.setElements(...) also maps the SBSection objects into LldbModelTargetModuleSection objects with the ranges properly set.

[gscalise]

More debugging insights... The ghidra.app.services.DebuggerStaticMappingChangeListener created here is not being invoked by mappingService, even with a 1000 times longer timeout, so the listenForMapping() completable future never completes. The first call to check() made here returns without completing, since the call to mappingService.getOpenMappedLocation() returns null.

[d-millar]

@gscalise Can I ask a favor? Would you mind making a separate ticket for the DebuggerStaticMappingChangeListener issue? We try to let folks know when we've fixed something by closing the issue, but right now we have several issues going on in this channel.

[gscalise]

@d-millar sure, what I meant by posting the insights here is that they seem to be related to this issue, and it only happens when the The resulting target process has no mapping to the static image ends up appearing.

It's not an issue per-se, but something that happens only in the context of this issue. Should I still open it as a separate issue?

[nsadeveloper789]

Yes, please. The mapping system is a bit of a house of cards. That message box appears when the launcher detects there's been no mapping after a set timeout. Essentially, it's just a canary that something has gone wrong, but it has no idea what. This conversation has now covered 3 different possible causes: 1) The names mismatch, 2) The back-end fails to present modules at all, and now 3) The mapping infrastructure has failed to notify a front-end listener. We sometimes let things meander a bit within an issue if it's quick enough, but this has started to get broad. Let's start tracking each one separately.