search

zsims / hunt-and-peck

Simple vimium/vimperator style navigation for Windows applications based on the UI Automation framework.
621 stars 54 forks source link

Add more hints: Toggle, Select, ExpandCollapse, Value, RangeValue #9

Closed extratype closed 7 years ago

extratype commented 7 years ago

3 Here it is!

zsims commented 7 years ago

Awesome! Thanks for the contribution. Let me have a play and then happy to merge!

I'd also like to setup CI at some point, but tbh there's not much that can easily be tested.

zsims commented 7 years ago

I've had a good play with this, and found some cases where it prevents any hints from working:

  1. As some controls support several hint patterns, they typically overlap so you end up being able to select something but not invoke it. A good example is Windows Exporer folders.

  2. The selection hint doesn't typically work (UIA throws an Operation not supported exception). For example when attempting to select Chrome tabs.

Have you found many cases where having these extra hints helps? Using the hap debug mode (Ctrl + Alt + ;) it seems that most places use the LegacyIAccessible control pattern -- such as Chrome. Do you think that select cases are worth supporting first? We probably need to think about how the other patterns fit into the model hunt and peck currently works around, or introduce alternative modes (different hot keys?)

extratype commented 7 years ago
  1. As some controls support several hint patterns, they typically overlap so you end up being able to select something but not invoke it. A good example is Windows Exporer folders.

Isn't it the other way around? I can invoke a folder in the list but not select. In the namespace tree at the left, I can select an item but not invoke. It doesn't implement invoke pattern. Then I can 'invoke' it by pressing enter key. Maybe choosing selection pattern first is better so that the user can decide whether to invoke it.

The selection hint doesn't typically work (UIA throws an Operation not supported exception). For example when attempting to select Chrome tabs.

Oh I didn't notice that, and LegacyIAccessible.DoDefaultAction works! I don't know why Chrome doesn't support SelectionItem.Select. I'd rather press T to choose tabs with Vimium.

I think LegacyIAccessible is too broad and don't find it useful. I added those patterns merely for completion so that I can access as many elements as possible while keeping the action simple.

I have been using this branch for about a month but I don't feel the need to support alternative hints. I would suggest typing at the prompt like >select [hint] (or >s [hint] for short) to specify the pattern.

zsims commented 7 years ago

Isn't it the other way around? I can invoke a folder in the list but not select.

Oops, yes you're right. Sorry 😨

Oh I didn't notice that, and LegacyIAccessible.DoDefaultAction works!

Interesting, I'll take a look at this!

I have been using this branch for about a month but I don't feel the need to support alternative hints.

I think the support is pretty useful where no other patterns are available. Would we be happy with that approach for now? If that's the case, then the precedence of patterns you have look good: Invoke > Toggle > Select > Expand/Collapse > Value > Range Value

Thanks again for the contribution/discussion :)

extratype commented 7 years ago

I think the support is pretty useful where no other patterns are available.

I agree, too. If you want to look for other patterns, see UI Automation Control Patterns Overview Control Types and Their Supported Control Patterns

zsims commented 7 years ago

Great, thanks for the link! This is a good start, although as discussed we may need to refine it in future to allow selecting specific patterns.

LGTM