Mysterious text gets typed in this grammar

[dylan-chong]

EDIT: The rest of this description is outdated see the comments below

This following grammar I have below behaves really strangely.

As you can see in the spec in the code, I should either be able to say this or this should not get typed, and then some stuff gets printed to the nat link console. This behaviour works as expected.

Another weird thing is that I can say should not get typed and then the text should not get typed gets typed into my host OS. Why??? 😕

import aenea.config
import aenea.configuration

from aenea.proxy_contexts import ProxyAppContext

from dragonfly import (
    AppContext,
    CompoundRule,
    Grammar,
)

def load():
    global git_grammar
    context = aenea.wrappers.AeneaContext(
        ProxyAppContext(
            match='regex',
            app_id='.*',
        ),
        AppContext(title='git'),
    )
    git_grammar = Grammar('git', context=context)

    git_grammar.add_rule(GitRule())
    git_grammar.load()

def unload():
    global git_grammar
    if git_grammar:
        git_grammar.unload()
    git_grammar = None

class GitRule(CompoundRule):
    def __init__(self):
        super(GitRule, self).__init__(
            spec='this [should not get typed]',
            extras=[],
        )

    def _process_recognition(self, node, extras):
        print('_process_recognition', extras)

load()

[dylan-chong]

OK oops part of the problem was caused by the grammar is not getting reloaded for some reason. There goes several hours of my day!

But anyway, the problem is still there. but i have been able to replicate it with a different piece of code.

So this is the new code that reproduces the bug.

So when I say this should not get typed, some stuff should be printed to the nat link console and that's it. This behaves as expected.

But when I say should not get typed, my host os types out wtf this should not get typed. Why does this happen? The grammar spec says that I have to have first the word this before saying should not get typed. Where does the Text even get executed?

import aenea.config
import aenea.configuration

from aenea.proxy_contexts import ProxyAppContext

from dragonfly import (
    AppContext,
    CompoundRule,
    Grammar,
    MappingRule,
    RuleRef,
)

from aenea import (
    Text,
)

def load():
    global git_grammar
    context = aenea.wrappers.AeneaContext(
        ProxyAppContext(
            match='regex',
            app_id='(?i)(?:(?:DOS|CMD).*)|(?:.*(?:TERM|SHELL).*)',
        ),
        AppContext(title='git'),
    )
    git_grammar = Grammar('git', context=context)

    git_grammar.add_rule(GitRule())
    git_grammar.load()

def unload():
    global git_grammar
    if git_grammar:
        git_grammar.unload()
    git_grammar = None

class GitRule(CompoundRule):
    def __init__(self):
        super(GitRule, self).__init__(
            spec='this [<command_with_options>]',
            extras=[
                RuleRef(
                    name='command_with_options',
                    rule=MappingRule(
                        name='command',
                        mapping={
                            'should not get typed':
                            Text('wtf this should not get typed')
                        }
                    ),
                )
            ],
        )

    def _process_recognition(self, node, extras):
        print('_process_recognition', extras)

load()

[calmofthestorm]

I have a theory of what may be going on here, though I'm not confident in it.

As far as I can tell, based only on my experience rather than concrete cited resources, Dragon takes the grammar structure into account when determining what words you sad. That is to say, rather than first processing the audio into words (analogous to lexing), and then in a separate pass parsing those words according to the grammars in effect, Dragon uses information from the grammar to assist with the lexing.

So for example, suppose you were in command mode, with only a single grammar active. This grammar consisted of two rules: one is you saying "red" and the other "blue". In my experience, Dragon will try very, very hard to match pretty much everything you say to either "red" or "blue".

This is less pronounced when in Normal mode, since Dragon always has the "alternate" hypothesis that you were dictating normal words.

So what I think may be going on here, is that you have few rules active, are in command mode, and say "should not get typed" without "this". Dragon tries to match that to something, and the best guess it has is that it simply didn't hear the "this".

Just some thoughts, I could be way off base. Issues like this seem unlikely to be a bug in Aenea (or even Dragonfly), but I'm fine with making this issue tracker available for their discussion.

[dylan-chong]

Hey thanks fir the ideas . I've just tried it out again and I don't thinks it's the normal mode thing because I don't actually have a text field open - I can't say 'hello world' because that won't get typed.

It did give me another idea though . The Text in the mapping has an execute method, and dragonfly seems to execute the text (since keys get typed into my host OS) . When I change Text() to [Text()] in the mapping, the bug doesn't happen. When I say "should not get typed" there is a message and then there's a message in the console saying the grammar is not matched. Hmm

On 24/06/2018, at 8:34 AM, Alex Roper notifications@github.com wrote:

I have a theory of what may be going on here, though I'm not confident in it.

As far as I can tell, based only on my experience rather than concrete cited resources, Dragon takes the grammar structure into account when determining what words you sad. That is to say, rather than first processing the audio into words (analogous to lexing), and then in a separate pass parsing those words according to the grammars in effect, Dragon uses information from the grammar to assist with the lexing.

So for example, suppose you were in command mode, with only a single grammar active. This grammar consisted of two rules: one is you saying "red" and the other "blue". In my experience, Dragon will try very, very hard to match pretty much everything you say to either "red" or "blue".

This is less pronounced when in Normal mode, since Dragon always has the "alternate" hypothesis that you were dictating normal words.

So what I think may be going on here, is that you have few rules active, are in command mode, and say "should not get typed" without "this". Dragon tries to match that to something, and the best guess it has is that it simply didn't hear the "this".

Just some thoughts, I could be way off base. Issues like this seem unlikely to be a bug in Aenea (or even Dragonfly), but I'm fine with making this issue tracker available for their discussion.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[dylan-chong]

I'll try upgrading dragonfly to a newer version and have a look at their issue tracker sometime.

On 24/06/2018, at 8:34 AM, Alex Roper notifications@github.com wrote:

I have a theory of what may be going on here, though I'm not confident in it.

As far as I can tell, based only on my experience rather than concrete cited resources, Dragon takes the grammar structure into account when determining what words you sad. That is to say, rather than first processing the audio into words (analogous to lexing), and then in a separate pass parsing those words according to the grammars in effect, Dragon uses information from the grammar to assist with the lexing.

So for example, suppose you were in command mode, with only a single grammar active. This grammar consisted of two rules: one is you saying "red" and the other "blue". In my experience, Dragon will try very, very hard to match pretty much everything you say to either "red" or "blue".

This is less pronounced when in Normal mode, since Dragon always has the "alternate" hypothesis that you were dictating normal words.

So what I think may be going on here, is that you have few rules active, are in command mode, and say "should not get typed" without "this". Dragon tries to match that to something, and the best guess it has is that it simply didn't hear the "this".

Just some thoughts, I could be way off base. Issues like this seem unlikely to be a bug in Aenea (or even Dragonfly), but I'm fine with making this issue tracker available for their discussion.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[dylan-chong]

I'll report back with what happens over at the dragonfly repo

[dylan-chong]

Moved issue to here https://github.com/t4ngo/dragonfly/issues/71