See about adding a timing harness for single Forth words through py65

[scotws]

Possibly based on the code for testing -- send word / line of input with word to py65, starting the timer, and then stop it when we receive a result. Repeat x times and average. Would be very crude, but might enable at least comparisons of various code variants.

[SamCoVT]

py65mon has a "cycles" command for a measurement method that doesn't depend on emulation speed.
Counting cycles also seems to make the most sense for applying to homebrew hardware that runs at various speeds.

Using this would require breakpoints to be set at the entrance of a word and another at its exit (and some words have multiple exits) or at the point it returns to. Perhaps a breakpoint in find-name and another in accept (before input is checked) might be good locations to allow testing many of the words.

Unfortunately, the exact addresses to use may move when Tali is reassembled. We might be able to use py65mon-labelmap to make an automated reasonable guess about where to place the breakpoints. The other annoyance is that py65mon doesn't seem to have a "continue" where you can pick up where you left off. I've resorted to using "goto" with the current PC value to get it simulating at full speed again, but that will be more difficult to automate.

[SamCoVT]

Some notes about py65mon behavior:

Hitting CTRL-C when py65mon has been run from the command line causes the command line -m and -r options to be undone, switching back to a 6502 processor with blank memory.

If we want to get around this, we need to load py65mon with no arguments and then run:

mpu 65c02
load taliforth-py65mon.bin 8000
goto f006

This will start Tali2 as if it was started with command line options, but now a CTRL-C will just drop you back to the py65mon prompt without upsetting anything else.

[scotws]

Ah, I did not realize this. Does this actually qualify as a bug we should talk to the py65 people about? It is annoying ...

[SamCoVT]

Now that we run the cpu emulator directly, this should be significantly easier. The cpu has a "cycles" property that we just need to check at the right times. The issue is getting the test code to tell us when.

I propose we use some extra "I/O" addresses to grab the starting and stopping cycle counts. $f001 and $f004 are already used for actual I/O, but we could use $f002 to grab the starting cycles and $f002 to grab the ending cycles and compute the difference (and even subtract the overhead of the cycle testing)

A cycle test for a word might look like:

hex
: dup_cycles 5 f002 c@ dup f003 c@ drop drop;
dup_cycles ( the result shows up here in results.txt )

The 5 is just the item for DUP to duplicate. Obviously some words depend highly on their input, but we could do a simple case for each word. That would let us know if we unexpectedly sped up or slowed down one of the words.

I'll give this a go and see what I can get working.

[scotws]

Sounds great! Do we want this to be part of the test program or have a second, specialized program for benchmarks?

(BTW, it looks like starting this weekend, I'm going to be crazy busy until about the end of the month to the point where I might not even be able to update stuff here. Sorry -- I'll try to catch up ASAP.)

[SamCoVT]

I think this can be done with the existing test program, but I expect all of the timing tests would be put in their own separate forth source file. It's up to you if you want them to run by default, but I think they probably should. That way the diffs will show us how much faster/slower words got when we make changes. I'll give it a go and see if it makes sense to have a separate program or not. It might, especially if you want formatted output ( eg a table or .md type output)

PS: I've been forthing too much recently, and now I can't type ( without a space after it :)

PPS: Thanks for the heads up on your schedule. I'll do my best to keep any issues and pull request as separate as I can. I think I have the hang of git now.

[SamCoVT]

I've finished up the timing tests with #106 and it looks good.