Share fuzz testing code

[cavaliercoder]

Hey, great project thank you!

I'd love to reuse your fuzzing process on a Go M68k implementation. Would you considering sharing the sources you used to fuzz and interrogate the Musashi emulator?

[emoon]

Hi,

This code uses the QuickCheck crate from here https://github.com/BurntSushi/quickcheck and bunch of the code to set up the tests can be found here https://github.com/marhel/r68k/blob/master/emu/src/musashi.rs#L565

[cavaliercoder]

Thanks, I'll take a closer look.

[marhel]

Yes, there's no separate sources for that, this kind of testing is done by loading the Musashi lib into the process, and interacting with it via Rusts C-interop. Then the test cases uses QuickCheck to come up with a random configuration of registers, status flags and memory, and we try our best to set up both Musashi and r68k cores this way, and then run a single instruction, and compare the effects on both cores afterwards.

The code uses Rust macros heavily, such as;

qc!(MASK_OUT_Y, OP_CMPI_32_PD, qc_cmpi_32_pd);

This macro generate a test function called qc_cmpi_32_pd, to test a particular variation of the CMP instruction (32-bit immediate value with address register postdecrement). The mask MASK_OUT_Y, tells us which bits in the opcode can vary (in this case, the 'Y' register part may vary, which controls which address register is used), because we loop to cover all opcode variants explicitly instead of relying on randomization to hit all opcode variations.

Then we also run completeness tests that runs through all 64K possible opcodes, (1K per test) such as:

qc_allow_exception!(BLOCK_MASK, BLOCK_7K, qc_block7k);

Which does the same thing. The reason we differentiate between these cases, and don't randomize the exact opcode, are that during initial development with just a few opcodes implemented, it's not helpful to know that the cores differ when asked to execute, say, opcode 0x4313 with some values, because you'd need a disassembler to know which operation that was.

Much more helpful when implementing CMP, to see that the instruction effects appears to differ when using A7 postdecrement, say, or (an actual case) ADDA differs when the same register is used as both source and destination