Request: add N64 debugging support

[n3rdswithgame]

I am currently trying to get an understanding of a few glitches and soft locks in Majora's Mask as I mentioned in the tasvideos thread (http://tasvideos.org/forum/viewtopic.php?p=438360 ). I'm currently using nemu as it is the only n64 emulator to my knowledge that has a debugger, but I would much prefer to be able to use bizhawk to debug the game.

[vadosnaprimer]

Try setting breakpoints to read or write through lua. Execution callbacks don't look supported. @pjgat09 mentioned m64p core has some debug features, but no one has got around to wiring them up. Whoever feels able to do that should send a pull request though.

[n3rdswithgame]

Is there any documentation on how to set and use breakpoints with lua? The TASvideos page on lua (http://tasvideos.org/Bizhawk/LuaFunctions.html) has no mention of breakpoints

[vadosnaprimer]

event.onmemorywrite() event.onmemoryread()

[n3rdswithgame]

thank you

[vadosnaprimer]

Did you get them to work?

[vadosnaprimer]

Just a question, why not nemu?

[Wyst3r]

While the breakpoints seem to work now (thanks!), they also appear to cause extreme slowdowns. Adding a single execute or memory breakpoint will drop the FPS to near 0.

I did some tests running the UI console that comes with the mupen64plus lib, and adding breakpoints there using the built-in debugging features doesn't seem to add any significant overhead. Looking at the BizHawk code, am i correct in assuming it makes a callback to C# for every single instruction?

[vadosnaprimer]

Yes, it's obviously due to the very nature of client breakpoints: emuhawk has to handle them in the end. And to detect the address you have a breakpoint at, it just checks them all. With insane amounts of instructions executed per frame, like with N64, PSX and alike, it might make sense to implement the breakpoint list internally, or use the existing one, resulting in no overhead for anything that has no breakpoint, but it requires a person who actually does it. I have other projects atm.

[Wyst3r]

I see, thanks for clarifying. I'll probably take a swing at it myself, since I have alot of breakpoint + Lua scripting ideas I want to try out. My BizHawk code experience is quite limited though so I guess we'll see what happens.

[vadosnaprimer]

[ot]Aren't you planning to return to Perfect Dark now, when we've finally fixed it in m64p core?[/ot]

[Wyst3r]

Hehe, yeah I noticed it had been fixed. I'm not really planning to go back to TASing for a while though. Also I need to port the Lua map I made for Goldeneye, since it'd be crazy to TAS without it.

I have some other tools in mind that would be very useful as well. I began working on a debugger for the internal scripts (action blocks) used by GE/PD as soon as I saw that execute breakpoints had been implemented (which is how I ended up here!). I'm also seriously considering making an advanced TAS bot that can do most of the heavy work automatically (alot of GE/PD TASing is just brute forcing after all).

[vadosnaprimer]

Right now, no core knows how to pause itself once the breakpoint is hit. However, some cores implement Step Into, which can give the idea of how to make the core pause mid-frame (presumably; I've never looked at it myself). If this is made to work for N64, you're almost there, but the debugger should already work. The speedup part just requires sending the proper parameters when setting the hooks. I'd really wish to work on this as long as there's someone who actually needs it, but that way I'll just never return to my own tas.

[Wyst3r]

Alright, so I think I managed to implement the changes I needed. It seems to work really well, although some parts are a bit hacky and some features are unsupported. For these reasons, I don't plan to submit a pull request, though I will include the patch here since I think it's a good reference for a "real" implementation (there's alot in there that can and should be straight-up copied). The patch can be found here (for some reason I couldn't attach a zip file to this post):

https://www.dropbox.com/s/dllejhtbs0x1kbj/m64p_native_debugger.patch?dl=0

Summary of the changes:

1. Added CallbackAdded/CallbackRemoved events to MemoryCallbackSystem, since we need to update the core with every breakpoint change.
2. Added DBG symbol to mupen64plus-core project + Set "EnableDebugger" flag in mupen64plus config.
3. Changed mupen64plus lib update_debugger to return a breakpoint index instead of a pc address, since read/write breakpoints need to know which address was read/written. An alternative would be to return a breakpoint pointer.
4. Added M64P_BKP_CMD_GET_STRUCT so that we can get the actual breakpoint information from the returned breakpoint index mentioned in the previous point.
5. Removed old READCB()/WRITECB() calls.
6. Added several new enums/structs/functions pointers to mupen64plusCoreApi to allow calling the Debug*() functions in the mupen64plus lib.
7. Added several functions for handling breakpoint added/removed events as well as the breakpoint hit event.
8. A couple of changes to frame_advance() function and OnBreakpoint(), to deal with the fact that the core can now pause itself.
9. Added StepInto() support for N64.
10. Added a IDebuggable::Resume() function for unpausing the core, and made sure this function was called when clicking "Run" in the debugger or client.unpause() in Lua. I have no idea if this is the correct way to do it. This way, the core being paused and bizhawk being paused is two separate things. It might be better to always unpause the core when Bizhawk is unpaused, though I couldn't find any good place to do this (the GenericDebugger class is the closest I could find, but it requires the debugger window to be open). It's possible to do it in MainForm, but that feels a bit ugly?

Unsupported features:

1. Step over/Step out - Haven't looked at these at all.
2. Address masks - From what I can tell, this is a somewhat difficult feature to implement. The idea is to change the breakpoint struct in mupen64plus, so that "endaddr" instead becomes "mask". You then need to mod lookup_breakpoint() and enable/disable_breakpoint(). The problem is that in lookup_breakpoint(), you're given an address range that is being read/written, and need to check if any of the breakpoints get hit by this. I spent several days trying to figure out a way to check if any of the addresses in the range matches the breakpoint address when masked, without looping through the whole range, but it's either very complicated to figure out or impossible. Since the ranges are normally <= 8 in length, it might be possible to simply brute force this, but i don't know how performance will be affected. Also, enable/disable_breakpoint() currently uses lookup_breakpoint to look through 65536 byte blocks for any enabled breakpoints, so brute force wouldn't work there. A rewrite of enable/disable_breakpoint() is probably necessary.

That's it. Hopefully this can be incorporated into a future BizHawk version without too much additional work. The challenges in making this patch release-worthy is mostly related to making the code consistent with other cores and the overall design philosophy of BizHawk (plus, the address mask stuff i mentioned). For now, I have enough to get started on my Lua scripts.

[vadosnaprimer]

OMG, this sounds phenomenal!

@parasyte and @pjgat09, we need your input here, since in fact it's probably PR worthy.

[adelikat]

Please submit this as a pull request :)

[Wyst3r]

Alright then :) I'm making some minor improvements to the mid-frame pausing/unpausing stuff, will PR as soon as it's finished.

[parasyte]

Interesting! I haven't looked over it yet (definitely prefer a PR). I have a mupen64 patch that adds breakpoints to each of the cores. If nothing else, it could be useful for contrast.

[Wyst3r]

https://github.com/TASVideos/BizHawk/pull/731

[DerekTurtleRoe]

@n3rdswithgame @vadosnaprimer @Wyst3r @adelikat @parasyte Is it OK if I close this since the debugger has been implemented?

[vadosnaprimer]

It hasn't.

[DerekTurtleRoe]

@vadosnaprimer Oh, I thought that was what #731 was, sorry I misread.

[vadosnaprimer]

The PR only contains breakpoints going through internal m64p debugger, instead of entirely through c# each time they're usually checked. Emuhawk debugger itself won't be usable anytime soon.

[DerekTurtleRoe]

@vadosnaprimer OK, thanks for the clarification.

[vadosnaprimer]

@Wyst3r do you still have the patch? Or the build? I guess since you're using it locally, you should have it.

[Wyst3r]

I don't know why I removed the original patch, but I made new ones from my local repository (they are based on the commit with the 1.11.7 tag): https://www.dropbox.com/s/8yeqh2eco29cj0c/0001-Use-mupen64plus-native-debugging-for-improved-breakp.patch?dl=1 https://www.dropbox.com/s/zviraeyvjsc02bb/0002-Improvements-to-mid-frame-pausing-Fixed-wrong-PC-bei.patch?dl=1

And here's the build: https://www.dropbox.com/s/j8spnxm516h6hr8/BizHawk%201.11.7%20-%20Native%20N64%20Debugger.zip?dl=1

Edit by feos: Attached the patches. patches.zip

[vadosnaprimer]

Thank you! Why don't you commit those to your fork btw?

[Wyst3r]

Dunno, hadn't really crossed my mind. Never thought these hacks would be used by anyone else :p

[vadosnaprimer]

After seeing this article, it's pretty obvious you're not the only one needing this :) https://aldelaro5.wordpress.com/2018/07/20/the-pain-of-researching-games-with-an-nintendo-64-emulator/

You might enjoy it too.

[aldelaro5]

@vadosnaprimer unfortunately, bizhawk is using .net which is making it impossible to work on linux (even with wine or mono) without a vm. Plus, as I mention in the article, I am really not into trusting current n64 emulator and mupen is no exception despite it being the least bad one. That is a much bigger problem, but still, nice to hear that debugging tools might get good on bizhawk :)

[parasyte]

@aldelaro5 You probably want an N64 with an EverDrive64 or 64Drive.

I've experimented with a remote debugger interface using the USB hardware on these devices, and it is simply excellent, even in rudimentary form. With some extra polish, it could be the one-stop solution for all your N64 reverse engineering needs. (cen64 is nice in theory, but way too complex to be realized in the short term, unlike adding debugger facilities to actual hardware.)

The only problem is finding the time to work on such fun projects while having a life.

[aldelaro5]

@parasyte this is a bit annoying to do because the everdrive cost quite a lot and an emulator allows me to do things you can't on hardware that i badly need like savestates or frame advance.

however, tbh, considering nothing better is getting done as far as emulation go (cen64 will take years before being ready), if a nice gui debugger exist to interface with the everdrive, then i'll likely really consider it. i usually don't like to research on hardware, but this seems to be the only good solution.

[parasyte]

@aldelaro5 I think you underestimate the power of inline debuggers. Save states and frame stepping are exactly the kinds of things you can do with them. Cost is another matter, but I'll leave it up to you to decide how to spend your money (and time).

Anyway, let's take this chat off of the BizHawk issue tracker. I have some insights I can share with you about debugger developments on N64 hardware. You can reach me at jay@kodewerx.org

[micro500]

@aldelaro5 Another option that I use very often is the trace logger. Most of the time I find it to be a lot more useful than breakpoints and a debugger. Since I have the full list of executed instructions I can search for the spot that I am interested in and work backwards to find out how it got there (not possible in a lot of debuggers). I can also search for all instances of a store/fetch from a specific memory address and pick which instance I am interested in without having to step through multiple false breakpoint triggers. Plus I don't have to worry about misclicking and skipping the instance I want. I also combine this with static analysis in a tool like Binary Ninja. Using a combination of these I've had a lot of luck reverse engineering MK64 without the need for a debugger. I'm sure you could do the same with Mario Party.