Console debugging - Githubissues

mateoconlechuga commented 8 years ago

It would be especially handy if we could have the debugger open on a certain illegal instruction, or some way to print to the console from a running program. This would make debugging C programs a lot easier.

elfprince13 commented 8 years ago

Note: we discussed two different contexts in which this would be useful.

First is just catching assert() conditions without having to manually set breakpoints (and maybe display a dialog with source information)
Second is funneling debug printfs to the emulator std out.

Presumably this feature would be toggleable in the case where extreme emulator accuracy was needed.

debrouxl commented 8 years ago

These two use cases are a good start into the broader set of program / emulator cooperation use cases. Custom debugging and validation features, toggleable for emulation accuracy as elfprince mentions, are a 13+-year-old feature request for TI-68k emulators, which never came to be fulfilled by any publicly accessible emulator.

The API/ABI of such features can be a set of custom instruction sequences, ala Valgrind, and/or a range of custom MMIO. The latter has no impact in the emulator's fast path.

Digging into old TI-68k stuff:

from the public header in e.g. PreOS, I gather that JM's unfinished, unreleased WTI had a special MMIO range, a way to detect WTI by reading a port from that range, and a way to set and clear (code ?) breakpoints by reading and writing to another port.
a post of mine in a TIEmu feature request topic mentions starting, stopping, reading the emulator's processor cycle count, assuming that is reliable enough for being usable.

We can think of commands for:

enabling and disabling existing code breakpoints without removing them;
adding / removing / enabling / disabling data breakpoints;
triggering a MMIO dump under some form;
triggering the emulator save state procedure (a double-edged sword...);
... certainly a handful of other valid use cases: more brainstorming needed :)

drdnar commented 8 years ago

On the Z80, there are a number of opcodes in the ED range that do nothing useful at all on hardware, except waste 8 clock cycles. I proposed having those trigger special functions in an emulator, allowing the exact same code to be tested on hardware and in the emulator, without worrying that you changed the exact bytes in the program in some way that broke or fixed your program on the other platform.

And that won't work on the eZ80, which traps illegal instructions. You could still have the assembler change them to NOPs on assembly for hardware, but it's still slightly problematic, because sometimes assembly programmers write code that's fixed very closely to the exact byte values of some instructions.

I like Mateo's suggestion of parsing a .LST file. You can skim labels from it, and use those as a basis for the emulator to resolve physical breakpoint addresses. And the debugger could display labels for you, too.

I also think that having Lua scripting could be beneficial in a debugger console. For example, you could write Lua code that parses one of your data structures into human-readable form. Or, at the least, have a way for the label scraper to learn about relocatable structs, as well as how to parse bitfields.

As for C printf() output, I'd suggest using writing to FFxxxx and wrapping every 64 K. Alternatively, just have FF0000 be debug output port. I'm not sure whether it would be easier for eZ80 C to write to the same byte again and again (easier on the emulator, probably), or wrap every so often (can use LDIR). I guess there's no reason you can't support both methods by just having every write to that range, in any order, be a serial debug output.

mateoconlechuga commented 8 years ago

The best way I can see to do this for C programs is to build a statically linked library that uses sprintf to build the string, and then since the resulting string is stored in RAM, simply copy it to the unmapped FFxxxx IO range as DrDnar described. This would be a fairly easy task, and quite handy. More CE emulator specific functions could be added as well, such as abort(); which I don't believe is implemented in the current standard. An execution of a ld l,l (0x6D) could trigger the debugger to open, as that instruction is only present in data. In addition, it would be nice to have Lua scripting for the debugger console, but I would not consider that high on the to-do list.

As Lionel said:

enabling and disabling existing code breakpoints without removing them;
adding / removing / enabling / disabling data breakpoints;
triggering a MMIO dump under some form;
triggering the emulator save state procedure (a double-edged sword...);
... certainly a handful of other valid use cases: more brainstorming needed :)

Also, equate file parsing is already a thing. This means that you can assemble your program in spasm-ng and it can spit out a .lab file that you can use to see equates and labels in the disassembly @DrDnar

elfprince13 commented 8 years ago

Don't copy the whole string - just its address. This will make it somewhat more reliable to catch what's going on.

On Friday, February 5, 2016, Matt Waltz notifications@github.com wrote:

The best way I can see to do this for C programs is to build a statically linked library that uses sprintf to build the string, and then since the resulting string is stored in RAM, simply copy it to the unmapped FFxxxx IO range as DrDnar described. This would be a fairly easy task, and quite handy. More CE emulator specific functions could be added as well, such as abort(); which I don't believe is implemented in the current standard. An execution of a ld l,l (0x6D) could trigger the debugger to open, as that instruction is only present in data. In addition, it would be nice to have Lua scripting for the debugger console, but I would not consider that high on the to-do list.

As Lionel said:

enabling and disabling existing code breakpoints without removing them;

adding / removing / enabling / disabling data breakpoints;

triggering a MMIO dump under some form;

triggering the emulator save state procedure (a double-edged sword...);

... certainly a handful of other valid use cases: more brainstorming needed :)

— Reply to this email directly or view it on GitHub https://github.com/MateoConLechuga/CEmu/issues/15#issuecomment-180238369 .

~Thomas

elfprince13 commented 8 years ago

Actually - better yet. Use a single MMIO address to trigger debugger actions, by writing a 1-byte value from some enum of possible actions, and then let the debugger read any further information by inspecting the chunk of data following PC (and update PC as necessary to skip over it).

drdnar commented 8 years ago

Ah, sweet. I still prefer to have debugging data come more from the assembler than the instruction stream. Adding our own opcodes may mean that the software being debug can't be exactly the same binary used on hardware, which kind of bothers me.

Only copying the string's start address has the advantage of being slightly more thread-safe. Not that anybody's working on multithreaded eZ80 code yet. . . .

mateoconlechuga commented 8 years ago

Well, now you can write to the console using ports 0xFB0000-0xFFFEFF. Single characters are sent at the moment; I'll probably change it to use a pointer and a signal port with the next commit. Ports 0xFFFF00-0xFFFFFF can be used for sending debugging comands that open the debugger in some way, for a total of 65536 possible commands. However, I am working on the C library that supports these, but I don't know what commands are needed. Is there a debug C command reference somewhere?

elfprince13 commented 8 years ago

I've been thinking about this over the course of the evening. I actually think that a hybrid system is in order. Having the debugger trigger commands in response to only a single MMIO address seems like a pretty important design feature, but using those high ranges to send data also seems like a good idea (because, e.g. we could sprintf directly into them). Let's say we reserve 0xFF0000 as the address to trigger debug commands, but allocate a buffer to receive data in the rest of that range prior to triggering a command. I would recommend disabling interrupts around any debugger-specific code anyway, so the threading shouldn't be an issue.

On Friday, February 5, 2016, drdnar notifications@github.com wrote:

Ah, sweet. I still prefer to have debugging data come more from the assembler than the instruction stream. Adding our own opcodes may mean that the software being debug can't be exactly the same binary used on hardware, which kind of bothers me.

Only copying the string's start address has the advantage of being slightly more thread-safe. Not that anybody's working on multithreaded eZ80 code yet. . . .

— Reply to this email directly or view it on GitHub https://github.com/MateoConLechuga/CEmu/issues/15#issuecomment-180664942 .

~Thomas

elfprince13 commented 8 years ago

Oops, got ninja'd while typing that out on my phone. The reason that I now think a buffer is preferable to only sending a pointer is it saves having to restructure the memory layout of a program to add internal sprintf buffers if you want to add debugging features to it.

Mateo - the obvious things to me are debug-prints and asserts. But placing a new breakpoint could also be handy.

debrouxl commented 8 years ago

Agreed about a hybrid system, with a command+data area, where we'd write commands, metadata (see below) and their arguments by reference and/or by whole contents, depending on the enumerated command byte.

We should reserve 15 bytes (I'm less confident about 7 bytes being enough) for future extension, before the data bytes. For instance, while I agree that interrupts should be disabled around debugger-specific code anyway, we should nevertheless reserve a byte for locking purposes, and maybe a byte for thread identifier. The usage of neither byte would be mandatory, of course. I'm thinking of the following workflow:

(optionally, when using locking: check the lock byte. If it's nonzero, take it by writing e.g. 0x80);
(optionally: write thread identifier if necessary. Most programs will leave it untouched);
write command arguments;
write command byte last to trigger command execution;
the emulator will reset the (lock and) command bytes when it's done.

mateoconlechuga commented 8 years ago

When we have 327679 bytes to work with and 83885824 possible commands+data, I don't think that will be a problem ;) The commands I added to the toolchain recently do not disable interrupts, but that can be done from within CEmu and then they can be put back into their original state, which I highly prefer.

mateoconlechuga commented 8 years ago

The following functions are caught by CEmu and then printed to the console or the debugger is entered:

/* opens the debugger and quits the program */
void abort(void);

/* opens the debugger */
void debugger(void);

/* prints directly to the console without opening the debugger */
dbg_printf(dbgout, const char *form, ....);

/* prints the assertion, including line number and file name
 * and opens the debugger and exits the program
 */
assert(condition);

What more would be useful commands?

debrouxl commented 8 years ago

Other special emulator support that I thought about, for e.g. unit tests and streaming of the corresponding events to subscribers:

"program entry" and "program exit". That said, program entry could be detected by control flow change to the program load address, and therefore, program exit could be detected by control flow change to the return address; also, dbg_printf() and a non-aborting assert(true) can be used to achieve a weaker form of that;
"start section" and "end section". Again, dbg_printf() and a non-aborting assert(true) can be used to achieve a weaker form of that;
computing the hash of a memory block (e.g. the screen area, for automated testing of graphics routines), e.g. with CRC32c or a member of the CityHash family for a fast hash, and SHA-256 for a proper cryptographic hash. This can be done from the inside as well, but slowly and with a size penalty, relatively small for CRC32c but huge for a pure software implementation of SHA-256 on an 8-bit microcontroller.

The special debugging support should be used by some of the tutorials, examples accompanying the toolchain and standard library unit tests.

The rationale behind unit-testing-oriented proposals is to try and help avoid, in core infrastructure, the disaster I found and subsequently fixed in GCC4TI when trying to build and execute the set of examples inherited from GCC4TI's dead ancestor. Clearly, this task hadn't been executed in years, as multiple examples didn't build in the first place, and IIRC, I discovered at least one bug in the standard library. Additionally, the on-calc names for multiple sets of examples from the same family (different implementations of the same functionality) collided, but that's not a correctness issue.

CE-Programming / CEmu

Console debugging #15