Various thoughts about function context

mpourmpoulis commented 4 years ago

Disclaimer

The is in continuation of the chat in the gitter channel. If I have misunderstood something, please feel free to correct me!

Files Involved

the main files involved here are

Where Contexts Get Created

unlike previous versions of Custer, instead of creating the context of our selves, we instead provide rule details object and castor creates the context for us.

for non-CCR grammars, this is handled by the mapping_rule_maker with the following bit of code

        context = None
        if details.function_context is not None:
            context = FuncContext(function=details.function_context, executable=details.executable, title=details.title)
        else:
            if details.executable is not None or details.title is not None:
                context = AppContext(executable=details.executable, title=details.title)

For CCR grammars these happens right here

https://github.com/dictation-toolbox/Caster/blob/d8f696284cf3df7f5161a3ba4a64c6f12b55f608/castervoice/lib/merge/ccrmerging2/ccrmerger2.py#L136-L168

an important thing to note here is that all of these happens long after they rule details have been validated, so any restrictions imposed there will have no effect from now on!

The Lost Executable

one of the first things to notice is that the function context constructor does not only receive the callable function we specified, but also the executable and title parameters we included in the rule details. At first glance these can have users thinking that by including an executable, the function context really evaluates to true only if both their function evaluates to true and the executable matches the foreground window. So for instance,@davidlehub shared the following example

def get_rule():
    return notepad,RuleDetails( executable="notepad", function_context = f2, ccrtype=CCRType.APP)

which unfortunately does not work as expected! if we take a quick look at the part of the constructor

https://github.com/dictation-toolbox/dragonfly/blob/7b9118f66b667bc474c740ff4f0617f700ee97b1/dragonfly/grammar/context.py#L377-L391

we see that they are treated as double star keywords arguments, to be fed to our function as default values if it has a Keyword Parameter named executable or title. But they either way do not make their, as dragonfly overrides them with their respective values for the current foreground window(line 400)

https://github.com/dictation-toolbox/dragonfly/blob/7b9118f66b667bc474c740ff4f0617f700ee97b1/dragonfly/grammar/context.py#L398-L413

And as you can see, there are no more checks regarding whether the foreground executable matches the barometer we specified!

if we want to address these misleading problem without change in dragonfly, we probably should create function context in the following manner

context = AppContext(executable=details.executable, title=details.title) & FuncContext(function=details.function_context)

also due to the laziness

https://github.com/dictation-toolbox/dragonfly/blob/7b9118f66b667bc474c740ff4f0617f700ee97b1/dragonfly/grammar/context.py#L174-L186

Our provided function will only be evaluated if the app we care about is in the foreground!

The Two Function Calls

One further thing that was mentioned in the chat is that the function seems to be evaluated twice. Why is that?

https://github.com/dictation-toolbox/Caster/blob/d8f696284cf3df7f5161a3ba4a64c6f12b55f608/castervoice/lib/merge/ccrmerging2/ccrmerger2.py#L158-L161

In the above code snippet we can see that caster note only creates the context for our grammar but also

produces its negation
takes the logical and for those negations

Effectively producing the context that is only true when all the other ones are false. this context is supposed to be created for the global rule

        """
        Returns a list of contexts, AppContexts based on 'executable', FuncContext, one for each
        app rule, and if more than zero app rules, the negation context for the
        global ccr rule. (Global rule should be active when none of the other
        contexts are.) If there are zero app rules, [None] will be returned
        so the result can be zipped.

        :param app_crs: list of CompatibilityResult for app rules
        :param rcns_to_details: map of {rule class name: rule details}
        :return:
        """

and in order to evaluate it we must evaluate all of the other contexts, including our function context. As a consequence, our function is going to be called twice:

while evaluating the context for the grammar we made it for
While determining whether the global rule should be active or not

Which explains why stuff were printed twice in the original example. But what is this global rule?

A Small Detail

Though I believe for most practical purposes these will not be a problem, you should be aware that those two function calls are not guaranteed to return the same value in the general case(for example if there is some sort of global state they are manipulating, the second goal may have a different behavior)

Handling CCR

during the merging process, there are two very important things Caster needs to do

allow for CCR specs to be repeated
Allow for specs from different grammars to be combined

the first of those do challenges is handled by creating repeat rules

https://github.com/dictation-toolbox/Caster/blob/d8f696284cf3df7f5161a3ba4a64c6f12b55f608/castervoice/lib/merge/ccrmerging2/ccrmerger2.py#L180-L205

using the dragonfly Repetition element, which are going to be loaded instead of the user written rules!

https://github.com/dictation-toolbox/Caster/blob/d8f696284cf3df7f5161a3ba4a64c6f12b55f608/castervoice/lib/merge/ccrmerging2/ccrmerger2.py#L56-L60

But tackling the second problem is quite more tricky, because not all CCR grammars are going to be active all the time, so their combination is not necessarily going to be the same for every utterance! probably the most obvious solutions to these are

for every utterance, determine which are activeand recompute their combination on the spot if needed
or precompute the correspondent combination for every possible scenario and activate only their one matching the current context

Caster more or less, seems to implement the second solution

https://github.com/dictation-toolbox/Caster/blob/d8f696284cf3df7f5161a3ba4a64c6f12b55f608/castervoice/lib/merge/ccrmerging2/ccrmerger2.py#L125-L139

it takes all the application CCR's and for each_of_them it merges it with all of the global CCR's,producing a separate rule bound to their application context, and then loads all of them!

https://github.com/dictation-toolbox/Caster/blob/d8f696284cf3df7f5161a3ba4a64c6f12b55f608/castervoice/lib/merge/ccrmerging2/ccrmerger2.py#L130-L133

through this process we create multiple variants of somewhat similar grammars:

they all contain the shared global stuff, that is specs from rules the user has annotated as global CCR and which should be active regardless of which application is running
but but each of them has its own separate application-specific goodies, that should be active only when it's context matches

then by using contexts, at the start of utterance dragonfly determines which of those precomputed variants is appropriate for the current "scenario" and activates it!

but what if we have an application in the foreground for which we have no specific commands? for the scenario we create a big global rule, comprisingOf only the shared global staff

https://github.com/dictation-toolbox/Caster/blob/d8f696284cf3df7f5161a3ba4a64c6f12b55f608/castervoice/lib/merge/ccrmerging2/ccrmerger2.py#L129

which should be active all only when none of the other variants are! That is exactly what the negation context expresses why it is used!

Minor Note

The whole thing with repeat rules does create some complications back at #775

No Function context for global CCR

the thing is that function contexts introduce scenarios, in a way more or less similar to their application context counterparts.

I think this is probably the reason why they are not allowed

Rule Separation

Rule separation happens right before merging

https://github.com/dictation-toolbox/Caster/blob/d8f696284cf3df7f5161a3ba4a64c6f12b55f608/castervoice/lib/merge/ccrmerging2/ccrmerger2.py#L56-L57

In the relevant code is here

https://github.com/dictation-toolbox/Caster/blob/d8f696284cf3df7f5161a3ba4a64c6f12b55f608/castervoice/lib/merge/ccrmerging2/ccrmerger2.py#L109-L124

and combined with how merging works it has a few important consequences

In particular, during the merging process specs from application specific grammars are never combined together!and because grammars are still considered separate even if they have the exact same context, if you have two or more rules bound to an application, you can never speak specs both of them in the same utterance!

mpourmpoulis commented 4 years ago

Caster more or less, seems to implement the second solution

By the way it seems that Breathe Goes for the first one.

LexiconCode commented 4 years ago

Thank you for such an excellent write up and follow-up. Due to some life events going on, It'll be a week or so before I could seriously follow-up with the issue.

LexiconCode commented 4 years ago

Just as a note this should be resolved for Caster funcContext App. However funcContext Global as mentioned above still needs to be addressed.

dictation-toolbox / Caster