Feature Request: Enable skipping specific number of cycles or a time interval in milliseconds

Very nice piece of software! For my testing application, timing is somewhat crucial. Is there a possibility to skip a number of scan cycles or even better, some milliseconds? How is timing handled in the simulation, anyway?

Re: skipping scan cycles.

This is definitely possible. In fact, right now, a blank line of input is treated as "no action for this scan cycle", either in interactive mode or in script mode. You can include as many blank lines as you want. One of the features on my mental todo list is to have a command that makes this easier, i.e. skips a specified number of scan cycles automatically. Or, more generally, just repeats a given action for a specified number of scan cycles. E.g., "R50" would skip 50 scan cycles, while "R50 c" would tap 'c' for 50 scan cycles in a row.

Re: timing.

Timing is emulated really, really stupidly right now - namely, each call to millis() from anywhere just returns 1 more than whatever was returned by the previous call to millis() from anywhere. I didn't spend a lot of time making a more accurate emulation because I didn't need a good millis() emulation for my purposes at the time, and I was focused on getting Hardware-Virtual to build in the first place. What do you have in mind that needs millis()?

Something like R50 would be very useful.

I need the millis() function to be at least somewhat meaningful as I intent to tests specific features of a plugin that involve timeouts. I know that it is almost impossible to get this right, but a rough estimation of the actual runtime behavior would be already a great improvement. One possible way to go would be to roughly assume a linear scaling between the simulated program execution time on the atmega32u4 and the elapsed wall time on the x86 CPU. Thus, we could easily scale up real time to elapsed simulated time. Of course, in reality this is way more complex as the two architectures have little in common. But for many testing applications the linear scaling hypothesis might be good enough.

To determine the scaling factor, I suggest two possible ways:

either an initial measurement of the performance of the x86 system, e.g. by running and timing a small sample procedure whose execution time on the atmega32u4 is known,
or by simply taking the clock frequency e.g. from /proc/cpuinfo (I am a Linux user only, but there must be a portable solution for this) and comparing it with the atmega32u4's.

We could thus have an additional simulator command that causes a number of cycles to be skipped that is roughly equivalent to the user defined time interval in milliseconds. This would be of great use for all those test cases where a given timeout has to elapse, e.g. to test some sort of exception handling procedure that is executed in case of a timeout.

Another thing that would be quite useful, by the way, is a command to specify matrix row and column indices additional to the already existing key-names, e.g. as matrix(1, 3) or key(1, 6). My plugin emits user defined keycodes to the keyboard report when certain keys are pressed in a well defined order. Those keys are mainly specified as matrix key positions, thereby partially omitting Kaleidoscope's keymap.

Some more words about adding commands... I really like your implementation of the Hardware-Virtual plugin. It is very nicely crafted and was easy to get in touch with. But having dealt with such things before, I know that extensions of a initially simple command parser can pretty soon reach a dimension where a handmade command parser becomes way too complex to handle the required grammar. It can be a perfect headache to maintain. More than once I designed a grammar that was not very extensible due to limited design in the first place, e.g. in a sense that I could not extend it later without breaking old application code. Because of this, it may be a good idea to delegate the command parser work over to a scripting engine. I had very good experience with boost-python as a C++ wrapper. Simulator input would be entered directly at the python console or through a python script. It would be pretty easy to wrap all existing commands in python code. The simulator would remain portable as python can be found on almost any platform. Boost and python can be installed as Ubuntu packages so building on travis VMs is not a big deal. Such a solution would also be nicer to maintain and extend.

I am aware, that this requires quite some changes. But as your Hardware-Virtual plugin will probably sooner or later be the core part of a Kaleidoscope test-bench it might be worth the effort. I am still thinking about a simavr based solution as an addition to the Hardware-Virtual plugin. Both tools could eventually share the same command frontend.

If you are interested, I would be happy to help you as much as possible with the boost-python part.

Wow, some great ideas here.

Re: timing. Assuming linear relationship between x86 and avr runtime is definitely very approximate, as you note - clock frequency ratio is not nearly the whole story, as the x86 is a sophisticated superscalar, out-of-order machine while the 32u4 is... probably not. An even bigger issue that the x86 performs lots of extra I/O for grabbing input and for logging output - and I/O operations are very slow. This will make the correlation between x86 and avr performance even worse.

An alternative idea, assuming that no one really needs timing at a within-scan-cycle granularity, is to just estimate how long a single loop() takes in Kaleidoscope on avr, and increment the millis() value by that much at the end of every loop(). This is probably a lot more accurate, easier, and sufficient for most purposes?

Re: specifying keys by coordinates, certainly that's something we can support. Feature request noted.

Re: adding commands, awesome! A scripting interface sounds like exactly what we need, and it's way more scalable. I'm very aware that scaling the current command parser is bound to be a headache... And, having a scripting interface would actually eliminate the need for an "R" or "R50" command - that could just be handled elegantly in the scripting layer, without cluttering the internal command processor.

If you're willing to help, I'd love to see a Python scripting interface for Hardware-Virtual.

Alright. I am going to be AFK for the next week. Upon my return I can work on the scripting interface.

Am 26. Oktober 2017 20:22:51 MESZ schrieb Craig Disselkoen notifications@github.com:

Wow, some great ideas here.

Re: timing. Assuming linear relationship between x86 and avr runtime is definitely very approximate, as you note - clock frequency ratio is not nearly the whole story, as the x86 is a sophisticated superscalar, out-of-order machine while the 32u4 is... probably not. An even bigger issue that the x86 performs lots of extra I/O for grabbing input and for logging output - and I/O operations are very slow. This will make the correlation between x86 and avr performance even worse.

An alternative idea, assuming that no one really needs timing at a within-scan-cycle granularity, is to just estimate how long a single loop() takes in Kaleidoscope on avr, and increment the millis() value by that much at the end of every loop(). This is probably a lot more accurate, easier, and sufficient for most purposes?

Re: specifying keys by coordinates, certainly that's something we can support. Feature request noted.

Re: adding commands, awesome! A scripting interface sounds like exactly what we need, and it's way more scalable. I'm very aware that scaling the current command parser is bound to be a headache... And, having a scripting interface would actually eliminate the need for an "R" or "R50" command - that could just be handled elegantly in the scripting layer, without cluttering the internal command processor.

If you're willing to help, I'd love to see a Python scripting interface for Hardware-Virtual.

-- You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: https://github.com/cdisselkoen/Kaleidoscope-Hardware-Virtual/issues/1#issuecomment-339755118

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

All right, so this discussion went a lot of places. Summarizing things up, and current status:

Automating the input scripting more, e.g. with an R (repeat) command. Possibly superseded by the planned Python testing interface. At any rate, if there is renewed interest in this command, the most appropriate place to discuss would be a new feature-request issue in keyboardio/Kaleidoscope-Hardware-Virtual.
Timing. This discussion is being continued in keyboardio/Kaleidoscope-Hardware-Virtual#4 (and somewhat in keyboardio/Kaleidoscope-Hardware-Virtual#2)
Specifying keys by coordinates. This feature request was addressed in commit 5ec8384.

With this in mind, I'm closing this issue, with discussions to be continued as noted above.

cdisselkoen / Kaleidoscope-Hardware-Virtual

Feature Request: Enable skipping specific number of cycles or a time interval in milliseconds #1