emoon commented 9 years ago

Currently FS-UAE (or any UAE version) from what I know doesn't support any remote debugging.

I would like to implement this but I would like to have have some guidance on how to do this before I get started.

I would likely want to use the existing code in debug.cpp but perhaps split it up so on path could go to sending to a remove socket and one still print to the internal monitor.

I would very much prefer to have a binary structure to be sent over the socket instead of sending text.

Any suggestions on how to go about with this?

Cheers!

Rhialto commented 9 years ago

On Sat 04 Apr 2015 at 07:30:03 -0700, Daniel Collin wrote:

I would like to implement this but I would like to have have some guidance on how to do this before I get started.

gdb has some remote debugging protocol I think. Is that what you had in mind, or were you not aware of that?

-Olaf.

emoon commented 9 years ago

Hi,

I looked at the GDB protocol and it seems that it would work as a good base to implement. My question then is if I should try to have try to unify some of the code in debug.cpp and debug_remote.cpp (or something similar?

I just want to make sure there are no objects in adding this feature and I want to go about the correct way of doing it.

emoon commented 9 years ago

After looking over the code for a while I have some more questions.

Implementing the remote debugger seems pretty straightforward. The main problem is code duplication which I would like to avoid.

Pretty much all of the code uses console_out_f to write the data to the console. That makes total sense in the current implementation. The problem with the remote debugger is that it wants to send back the data in another format compared to what is being written to the current.

There are a few approaches to this:

Have console_out_f being a function pointer instead which defaults to the current code and the remote debugger can change this to “catch” the written text output by a command, parse the text and send the correct formated way.

Pros. Small amount of code duplication. Cons. Need to parse data that already exists in existing structures inside UAE. Seems wasteful.

All functions that should dump a state writes to a struct with all the values in it. Then there are two functions that takes that data in. One that writes to the console console and one for the remote debugger.

Pros. Keeps the current structure, no extra parsing is required for the remote debugger. Cons. Adds complexity in the current functions as data needs to be populated in a out struct which isn’t needed today.

Duplicate most of the code that remote debugger need from the current debugger code.

Pros. Allows the remote debugger to do things fully independent of current debugger code, very small parts of the current code needs to change Cons. Duplicated code is never good, bugs occur when people change one file and forgets the other.

Another issue is the GDB Protocol. While it might serve as a good basis it also doesn’t cover lots of the Amiga specific things that one wants to support (copper debugging, etc)

I’m thinking about using the GDB Protocol as a basis where it makes sense but also add extended commands where it also makes sense that regular GDB might not understand.

Suggestens and thoughts about this?

Cheers!

FrodeSolheim commented 9 years ago

Hi, I have little knowledge of the internals of the debugging module, and I do not have any strong preferences on this topic at the moment.

However, one thing I think is very important: I try to keep the changes between FS-UAE and WinUAE as small as possible (in common code), to keep the code bases from drifting apart and easy merging of WinUAE improvements.

So I would be skeptical to merging updates which changes too much of the existing code, if Toni is not also willing to merge it. But if Toni is not interested, and for example most of the new stuff can be put in new modules, and changes to existing code kept to a minimum, then I'm much more likely to accept the changes into the main branch :)

emoon commented 9 years ago

Thanks for the reply,

Yes the plan is to make the change as little intrusive as possible and there is nothing in the change that would be specific to either F-UAE or WinUAE so that should be fine.

That being said I will need to change some general files (debug.cpp needs to know if remote is being used or not, the new code needs to have a update call per and such) but I will try to keep it minimal and most code will be inside the new debug_remote.cpp/h files.

tonioni commented 9 years ago

What kind of remote debugging? Your post do not explain (or I simply don't "get it") what do you want to do, include some examples please.

emoon commented 9 years ago

@tonioni

Imagine having a regular debugger (Visual Studio, WinDBG, etc) connected to UAE but debugging the Amiga code running inside the emulator. The way to do this is to allow an external debugger to connect to UAE and send commands to it (like step instruction, show memory, etc) and UAE sends back info according to the request (if step: send back new PC, registers, etc, if memory: send back memory given range requested by the debugger)

Does that make it clear?

emoon commented 9 years ago

To show more what I mean. Here is a screenshot of my debugger connected to C64 VICE (http://vice-emu.sourceforge.net)

https://dl.dropboxusercontent.com/u/5205843/vice_prodbg.png

What I want to do is to implement something similar for UAE

tonioni commented 9 years ago

Ok. I think you shouldn't use existing debugger commands because they are not stable or even supposed to be stable (it is mostly my own playground, tweaking/changing when I need some temp option when debugging some specific demo or game). Of course you can use (mostly safely) functions that are not part of parsing or formatting debugger commands.

Best (and most boring) option probably is better separation of input parsing/output formatting from "stable" debugger operations. (like disassembler, memwatch/breakpoint detection)

emoon commented 9 years ago

Alright. Thanks for the reply.

So I think I will go for more of a hybrid approach then. My plan is then to:

Have the code in debug_remote.cpp/h
Add a command line parameter to say if the remote debugger should be active or not.
Inside debug.h/cpp I will need to do some minor changes so if the remote debugger is active it will call some of those functions (for example if an illegal happens) but the changes should be quite minimal and the if no remote debugger is active it will behave as it is today.
I plan to implement the debugger using the GDB protocol but will likely add some extra stuff for Amiga (like copper debugging and similar things)
Parse the output from the operations as you say. I will likely in some cases do something own (when getting memory for example)
In order to make the capturing of output I thought about changing the console_out/console_out_f to function pointers that defaults to the current implementation. As these are just for printing data the overhead shouldn't matter. This would allow me to set custom output that I can capture easily by invoking the stable commands as you write above.

Sounds good?

valentinomiazzo commented 9 years ago

@emoon I think adding a GDB interface to WinUae and Fs-uae is very important. Great you are working on it. Thanks!

emoon commented 9 years ago

@valentinomiazzo Thanks. Notice (as written above) I will likely add some extensions that may not work directly with GDB but requires custom support but hopefully standard operations should work fine.

emoon commented 9 years ago

Also notice that I intend to do most of this work during July when I'm on vacation. I just want to make sure I have a good direction before I start so really appreciate your input here!

valentinomiazzo commented 9 years ago

I think there is a good amount of people that would like to develop something for Amiga but are blocked by the difficulty of setting up a development environment. With a GDB interface people can develop on their platform of choice with the IDE they like. This reduces a lot the barrier to entry and would be beneficial for the whole Amiga community. So, if you want an advice, try to make it compatible with the most common debuggers like Eclipse, VS, etc...

emoon commented 9 years ago

I will but in some cases it will be hard because copper debugging is like debugging a different CPU at the same time as the regular CPU. I have no idea if GDB support that.

valentinomiazzo commented 9 years ago

On the 'custom' side I can suggest to expose Amiga internals in JSON form so people can develop plugins and use libraries like http://d3js.org/ to visualize these data .

emoon commented 9 years ago

But that wouldn't be compatible with the GDB api no?

valentinomiazzo commented 9 years ago

No, it would not be. I imagine people will start using the standard debuggers built-in in the IDE because used to them. But when they will start to bang the hardware and need to profile they will start to use dedicated more specific debuggers like your.

emoon commented 9 years ago

Ah yes true. Thanks for the input.

I will start with getting the basics up and running and the plan for my debugger in general is to allow people using write plugins and custom visualises in various languages. Currently C/C++ is supported but I intent to expand that in the future.

emoon commented 9 years ago

I will go on implementing this as outlined in my previous post above unless someone objects.

valentinomiazzo commented 9 years ago

@emoon I just want to clarify that I'm not one of the mantainers of winuae or fs-uae. Therefore please don't give to my posts any kind of official weight. I just expressed my opinion :)

tonioni commented 9 years ago

Looks good. Just keep debug.c changes minimal, even if it means duplication. When you have some proof of concept (or better) ready, then it is time to check which parts of original debugger are really reusable.

emoon commented 9 years ago

@tonioni Thanks for the comments. Sounds good.

Yes. The plan is to keep debug.c changes minimal.

If we want to have close to zero changes in debug.c I suggest that perhaps we should have a global debug "interface" (Interface meaning a struct with function pointers, like activate_debugger, update_debugger, close_debugger). So default we use the current debugger but if remote is activate that one is used otherwise the remote one but perhaps that is overkill.

Thoughts?

FrodeSolheim commented 9 years ago

If you just need to hook into a handful (or two) of functions in debug.c, perhaps just add something like this to the beginning of the functions:

ifdef WITH_REMOTE_DEBUGGER

if (use_remote_debugger) {
    call_remote_debugger_function();
    return;
}

endif

(this way, the changes are easily identified, but could get a bit messy unless you only need to add to relatively few places).

emoon commented 9 years ago

Yeah that was pretty much my idea. It shouldn't be that many changes really.

emoon commented 9 years ago

Just wanted to let everyone know that I will be starting to work on this now.

valentinomiazzo commented 9 years ago

Great! Il 25/lug/2015 04:38 PM, "Daniel Collin" notifications@github.com ha scritto:

Just wanted to let everyone know that I will be starting to work on this now.

— Reply to this email directly or view it on GitHub https://github.com/FrodeSolheim/fs-uae/issues/63#issuecomment-124851558.

emoon commented 9 years ago

I have one question. I need to place my

remote_debug_update ();

call somewhere (this is to keep the update the socket, check for incoming events, etc) this needs be called once per frame or something similar.

Right now I stuck it into this function in newcpu.cpp

static int do_specialties (int cycles)

Which is fine but once there is a connection there will be plenty of 'hammering' on the socket (as I guess this code is called once per CPU instruction) If the remote debugger hasn't been setup the code will just return directly and shouldn't have any issues perf wise but once connected this may or not be an issue (haven't actually tested yet)

I thought about using some cpu timer to check how much time that has elapsed since the last time the code was called which may work ok.

Any other suggestions?

tonioni commented 9 years ago

Adding function call to hsync or vsync handlers (devices_hsync or devices_vsync*) is usually the best way to do. Use of UAE timers is only needed if you want cycle-exact triggers. ("Execute my callback function exactly after 123 DMA clocks")

emoon commented 9 years ago

Will do. Thanks!

emoon commented 9 years ago

Just wanted to give a update:

gdb_session

As I decided to go for implementing the GDB protocol I can of course use GDB for testing. In this screenshot gdb is connected to FS-UAE, and can request registers, memory and so on.

valentinomiazzo commented 9 years ago

Wow! This is awesome.

One question: how this relates with running gdbserver on the emulated Amiga? I imagine this is completely orthogonal.

BTW, even just have a cross toolchain windows-68k with working gdb/gdbserver is quite a feature. Please, if you want, document how to you got it work, it is not well documented on the web.

emoon commented 9 years ago

Thanks!

So this gdbsever isn't actually running "on" the Amiga but as part of UAE. This allows for debugging things which can't be launched from easy (track loaded demos, OS stuff, etc)

What I do hope (not sure about this yet) is that I can load an executable within UAE and tell the Amiga to start it (this is because if there is debug symbols in the executable the debugger needs to match line with actual code loaded in memory)

Regarding working toolchain I really only used this: http://crosstool-ng.org and selected 68k target. I haven't actually tried to use this to compile anything for the Amiga, just use GDB.

When it comes to building stuff for Amiga I use vbcc + vasm for our demo stuff which can be found over here: http://sun.hasenbraten.de/vbcc/

emoon commented 9 years ago

So I have some questions that maybe @FrodeSolheim or @tonioni (or anyone else of course) perhaps can reply to.

What I want to do from the backend is to load a Amiga executable into memory and execute it. I don't know every single details of what that involves but here goes:

Can I allocate memory from the Amiga host in some way that the Amiga "knows" about it. As UAE holds all the memory for the Amiga this should be doable I hope.
Then I need to do a own exe loader (no issue here)
Then I need to start it. I would assume I need to do a bit more than just set the PC to the start of the executable and go or is that all to it?

The primary reason I want to do this (as I hinted at in an above post) is that I want to support debug info for the executables which means the debugger must know where in memory a certain line maps to given source file)

tonioni commented 9 years ago

That gets really tricky and it is going to be impossible without adding helper routines to UAE Boot ROM.

You can't safely allocate memory from emulator side (For example It is impossible to know if memory lists are currently valid). Starting a process is even more complex.

Only Amiga-side m68k code can call emulator C code (via trap system) and only trap called C code can call few Amiga library functions (mainly allocmem(), freemem() and signal() and few others).

AFAIK only working solution is to set some trigger from debugger that boot rom code detects (for example set bit in boot rom ram area from host-side that boot rom code vblank routine checks), when triggered, it starts a dos process which then calls your debugger C code via trap, C code uses allocmem() to allocate memory for exe and poke it to amiga memory in relocated format, then start address is returned back to m68k boot rom code which jumps to loaded exe's entry point, when call returns, another emulator C code function gets called (if you need it) and either your code or boot rom code frees exe's segments then "debug" process exits.

It sounds complex but it really isn't that complex, most of that stuff already exists. I can do the uae boot rom part if needed.

emoon commented 9 years ago

Thanks for the reply Toni,

It actually doesn't sound that bad. If you would be able to add the needed code in the UAE Boot ROM I would really appreciate that.

Next to allocating regular memory I would likely need to alloc chipmem as well and I'm not sure if that is currently exposed?

tonioni commented 9 years ago

I forgot that you can call any library function from trap routine with CallLib() function. You are not restricted to few special cases. bsdsocket.cpp is good example showing how it works.

emoon commented 9 years ago

Ah! Great, thanks again!

tonioni commented 9 years ago

Debugger trap stuff added. filesys.cpp has "debugger_boot()" that sends signal to uae boot rom code which then creates new dos process and calls debugger_helper() with mode parameter=1. Here you can inject your executable to Amiga RAM, when debugger_helper() returns, it will call dos/RunCommand() with values you left in m68k registers (if D1 = 0, do nothing), when RunCommand() returns, mode=2 will be called (d0=RunCommand() return code).

emoon commented 9 years ago

Sweet! Thanks. I have been a bit side tracked by some other stuff lately but I will take a look at it soon again.

emoon commented 9 years ago

@FrodeSolheim Hey do you have any plans to bring in the latest WinUAE code soonish?

FrodeSolheim commented 9 years ago

@emoon Hi, short version: you can work off the FS-UAE "future" branch if you want updated WinUAE code :)

Right now I am focusing on releasing a stable 2.6.0 version (based on WinUAE 3.1.0 code), so the master branch is still tracking this WinUAE version (both the development versions and the stable/beta versions of FS-UAE).

I maintain a parallel branch called "future" where I continue to merge updated WinUAE code (and where I also merge in updates from the "master" branch. This branch will be merged into "master" when I switch to WinUAE 3.2-based emulation for the development versions).

emoon commented 9 years ago

Ah, I see. :) I will do that. Thanks!

emoon commented 9 years ago

Please don't hijack this thread for this. If you think that this is a problem open a issue over at the WinUAE repro. Also FS-UAE is cross platform already and if you took the time to look at the code changes in both FS-UAE and WinUAE you would see that there is code sharing going between the two projects.

FrodeSolheim commented 9 years ago

The future branch is now updated with the latest WinUAE beta (3200b10) changes.

emoon commented 9 years ago

@FrodeSolheim Thanks! I will move over my code to the future branch.

lab313ru commented 9 years ago

How could I check remote debugging functionality? Which branch is it, and which defines are need?

emoon commented 9 years ago

@DrMefistO Hi, right now the the remote debugger is very basic. You can find it over at my branch here https://github.com/emoon/fs-uae/commits/master

The way it works is:

Start fs-uae with -remote_debugger=1
Start gdb (m68k version)
Do a remote connection to the target (localhost if you are running on the same machine)
Now you can step, break, read/write memory, registers, etc.

That is what is supported right now.

lab313ru commented 9 years ago

"Start gdb (m68k version)" - where could I find this version? What features are you going to add in the future?

That is great! I'll check it.

emoon commented 9 years ago

@DrMefistO I built it from source using http://crosstool-ng.org/ for Linux and I also built it for Mac by just grabbing the latest GDB source and compiling it from m68k-target.

Right now I have the bare minimum implemented. My plan in general is to have a bare minimum GDB implementation that I will then use for my debugger which should have some more features in it which GDB might not support (using extensions) like copper debugger, etc

FrodeSolheim / fs-uae

Remote Debugger Support #63

ifdef WITH_REMOTE_DEBUGGER

endif