t4ngo / dragonfly

ARCHIVED! - Speech recognition framework allowing powerful Python-based scripting and extension of Dragon NaturallySpeaking (DNS) and Windows Speech Recognition (WSR)
GNU Lesser General Public License v3.0
364 stars 82 forks source link

Order of grammars / grammar precedence #16

Open codebold opened 9 years ago

codebold commented 9 years ago

I miss a feature to priorize one grammar/rule over another.

The thing is that I want to use the same command in multiple grammars. Things might become clear if you have a look at my dragonfly scripts repository (, or maybe not...). In those scripts I try to establish one general base grammar (_base.py) that gets ovewritten by more specific grammars (_others.py) based on the context. However i had to realize that there is no concept for overwriting grammars.

If there would be such a concept, I could reliable use the same command in different contexts. So for example the "find" command should be treated like this:

There are plenty of similar cases in which different programs execute the conceptually equal commands in different ways and therefore need individual treatment. The presence of contexts suggest that there might be the possibility to do so (use the same spoken command for executing different actions depending on the context).

However, the problem is that i did not find a deterministic behavior in which order NatSpeak recognizes commands. I tried adding/activating the more specific grammars/rules before the more general ones and vice versa. Sometimes it worked for a short time, but next time i reloaded the grammars or restarted NatSpeak it broke again.

I'm willing to put more effort into this myself, but right now I'm stuck. So any help or thoughts in this direction are much appreciated.

jgarvin commented 9 years ago

I ran into this with emacs having different bindings for everything compared to normal apps. I fixed it by making the general case a 'not emacs' context rather than global. Not as elegant but does the job. On Nov 27, 2014 12:15 PM, "codebold" notifications@github.com wrote:

I miss a feature to priorize one grammar/rule over another.

The thing is that I want to use the same command in multiple grammars. Things might become clear if you have a look at my dragonfly scripts repository https://github.com/codebold/hiccup (, or maybe not...). In those scripts I try to establish one general base grammar (_base.py) that gets ovewritten by more specific grammars (_others.py) based on the context. However i had to realize that there is no concept for overwriting grammars.

If there would be such a concept, I could reliable use the same command in different contexts. So for example the "find" command should be treated like this:

  • In general (AppContext=()) this command is mapped to 'Key("c-f")'.
  • On Emacs (AppContext=(executable="emacs")) this command should be mapped to 'Key("c-s")'.
  • And on Vim (AppContext=(executable="putty")) this command should invoke 'Key("/")'.

There are plenty of similar cases in which different programs execute the same concepts in different ways and therefore need individual treatment. And the concept of contexts suggest that there might be the possibility to do so.

However, the problem is that i did not find a deterministic behavior in which order NatSpeak recognizes commands. I tried adding/activating the more specific before the more general and vice versa. Sometimes it worked for a short time, but next time i reloaded the grammars or restarted NatSpeak it broke again.

I'm willing to put more effort into this myself, but right now I'm stuck. So any help or thoughts in this direction are much appreciated.

— Reply to this email directly or view it on GitHub https://github.com/t4ngo/dragonfly/issues/16.

codebold commented 9 years ago

Thanks! I thought about that, too. But i think that it will make things very complex as it has to be ensured that every command in the base grammar with the 'not specific' context has an equivalent in the 'specific' context's grammar. In that way i have to duplicate all the basic editing and navigating commands (like "one-word-forward", "select-one-word-forward", "delete-one-word-forward", and so on...) even if i will only treat the "find/search" command differently. Or am i missing something here? And what makes things even worse is that i have to keep those grammars more or less in sync.

So if there is no other way i will go with this workaround, but i really hope there is another way.

codebold commented 9 years ago

I put some thoughts into the "not specific" context again and came up with an idea. Why not use Python itself to establish the concept of overwriting commands. I still think of one base rule that defines the base commands and multiple more specific rules that execute different actions for commands that are already defined in the base rule. But instead of simply loading the stuff into the speech recognition engine, the more specific rules are inherited from the base rule and the commands in the more specific commands overwrite the more general inside the Python object instead of inside the speech recognition. There is still some manual bookkeeping required for taking care that the specific rules will not become active in the base context. However this seems to be a reasonable effort and may be automated with some bootstrapping code.

jgarvin, thank you for bringing up the 'not specific' thing! I will give it a try and report on the outcome.

If there is another way to implement order of grammars or grammar precedence I'm still very interested in it.

synkarius commented 9 years ago

I would be interested to see your implementation of this. If it is successful, please do post it.

codebold commented 9 years ago

After playing around a bit, I came up with this builder .

Besides simply building grammars from plain old rules, it provides a method for adding "competitive" mapping rules. The first submitted rule builds the base rule that is extended by the other rules being submitted in the call. This way it easily possible to use the same command/action in different contexts.

However the builder accesses internal properties of the rule classes and can only be used for mapping rules yet. So it's definitively a hack. But one that I can live with...

synkarius commented 9 years ago

Huh! I wouldn't have thought to combine mapping rules like that. Yes you used internal properties, but that's very useful. Interesting.