Add a command testing mode (Feature Request)

[gimpf]

For development/integration of larger configurations it would be great if one could test the invocation of commands w/o Talon actually performing any actions, but instead only writes out which command it would perform.

How I currently imagine this:

• Have a prepared audio file
• Set Talon input to output monitor
• Set Talon to "command testing mode"; this would replace command execution w/just printing to log what happens, and maybe even allow to set fake os/app/title/etc. scopes.
• Play the audio file, and capture the printout (ideally in a separate file, otherwise extract from log)
• Reset Talon settings
• Compare the printout w/the expected printout.

This would be like an integration test for the whole speech recognition engine, talon command engine with the specific command/capture/list definitions from the configuration.

Why this would be helpful: A wrong match, an incorrectly refactored capture, or simply an inadvertently conflicting definition in the wrong position can either unnecessarily destroy a whole config (but is easily seen), or may not be discovered until weeks later (if a specific os/app/etc. combination happens not to be used for a while).

I know it would have helped me already, and would also have helped to discover some issues in the community repositories.

[dwiel]

This would be extremely helpful even if it didn't include raw audio. Actually, I would probably prefer not to include raw audio. I would want something like: assert(generate_outputs("phrase hello world") == "hello world"). Obviously, it gets harder to mock clipboard actions and things like that, but I would want this kind of thing for testing edge cases like phrase hello slash world -> hello/world, keeper slash -> slash, slash -> /, etc. and these don't often involve any feedback from apps.

[crypdick]

I have a similar use-case. I am trying to systematically create an optimized vocabulary by looking at phoneme transitions. But testing my changes has been extremely tedious: swapping a word may improve accuracy in one context, but result in "sound clashes"/reduced accuracy in another context.

Things would be a much easier with a tool to "unit test" the vocabulary. For each word in the vocabulary, pre-record as many samples as you want. Then, as you edit the vocabulary, Talon can make predictions for each of the recordings and assert whether the predictions matches the ground truth label. This would make it very easy to test whether changes to the vocabulary causes side effects.

[dwiel]

Just another case that came up: the logic for deciding how to map from jetbrains window titles to filename extensions changed. I can change it back from my particular use case, but without knowing which use cases the new code was targeted at, I have no way of knowing if my change will break something for some one else.