search

dictation-toolbox / dragonfly

Speech recognition framework allowing powerful Python-based scripting and extension of Dragon NaturallySpeaking (DNS), Windows Speech Recognition (WSR), Kaldi and CMU Pocket Sphinx
GNU Lesser General Public License v3.0
387 stars 75 forks source link

Rules using command chaining and Dictation don't work for the Sphinx engine #4

Closed drmfinlay closed 6 years ago

drmfinlay commented 6 years ago

Command chaining (repetition) of rules using Dictation elements/extras doesn't work at the moment. This is due to such rules needing to be spoken in sequence with utterance pauses.

I have some work and tests on this using a Timer thread which processes the complete repetitions of rules so far after a timeout period, similar to the timeout period that starts after speaking part of a rule. This work is in the feature/repetition branch.

drmfinlay commented 6 years ago

I've been doing a little more investigation into this lately and fixing this completely is absurdly difficult to do IMO.

The best way of having this functionality is probably by using n-gram language models interspersed with grammar literals like Silvius does. I believe that CMU Pocket Sphinx and Kaldi (what Silvius uses) both support the same ARPA n-gram language model format. Having that would get rid of utterance breaks altogether and simplify the engine considerably. I'm not sure where to start with implementing that though.

For the moment, I think it would be best to have separate JSGF rules for mappings using Dictation by changing dragonfly2jsgf. Repeatable mappings that don't have Dictation can then still be repeated without the current performance issues, you'll just need to pause before starting mappings that use Dictation.

I've deleted the feature/repetition branch as it was very out of date and the approach is a dead end.