search

Descolada / Acc-v2

Acc library for AHK v2
MIT License
41 stars 6 forks source link

Some ideas #4

Open sancarn opened 12 months ago

sancarn commented 12 months ago

Hey,

Found this library, cool stuff and glad my name made it into the commentry! haha

I don't really code in AHK anymore, but I did port everything to VBA

Some of your features are super cool, and I don't fully understand, and also you are missing some features too.

Missing Features

  1. Many Acc Elements implement IAccIdentity interface, which has a GetIdentityString(). This can be useful if you're building a treeview or something. Not all elements do implement this, but it still can be useful.
  2. FindElement could probably use a DepthFirstSearch vs BredthFirstSearch param.
  3. With FindElement and FindElements I find that sometimes you can discount whole trees worth of elements to increase the performance of a search. I'm not 100% sure how these functions work, and I can't remember if you have callbacks in AHK, but if you do it might be worth concidering the below:
;Excuse my ahk code, i am very rusty!

cbFindElement(element) {
  if(element.class=="Bollib"){
    ;If we know we don't have to search the descendents
    return Acc.FilterResult.SkipDescendents
  } else if element.name == "Something" {
    ;If we get so far that we can guarantee the element is not in the tree
    return Acc.FilterResult.NoMatchCancelSearch
  } else if element.name == "Whatever" {
    ;Found, don't search decendents
    return Acc.FilterResult.MatchFound
  } else if element.name == "Podrick" {
    ;Found, but child tree could also contain matches...
    return Acc.FilterResult.MatchFoundSearchDescendents
  }
}

el = acc.FindElements(cbFindElement)

This gives significantly finer control over the search process :) Here are my own implementations for FindAll and FindFirst if you wanted to add these to your own library 😄 .

Questions

Onto the things I don't understand. Your ActivateChromiumAccessibility function - Does this, rather, obtain the IAccessible of chromium if it is active? Or will it actually obtain IAccessible of chromium even if accessibility is not enabled?

At work we usually launch chrome with --force-renderer-accessibility command line flag, because the accessibility is disabled by default. But would sending WM_GETOBJECT actually enable it for any arbitrary chrome window?!

Descolada commented 12 months ago

Thank you for your feedback! :)

Missing features

  1. I vaguely remember trying to implement GetIdentityString at some point, but I didn't manage to get any human-readable output from it. Looking at your code it looks like you are simply converting bytes into hex values and concatenating them to create an ID, is that right?
  2. Yeah, FindElement currently doesn't support BFS, though it does support pre-order, post-order, and changing the search direction. Adding BFS shouldn't be too difficult, but since my focus has shifted to UIA, I'm not sure whether I'll implement it in this library.
  3. That is a wonderful idea and I'll definitely implement it. Although I don't care much for Acc performance after switching to UIA, which blows Acc out of the water in almost every way. Tree searches using UIA are so much faster that even the most optimized Acc search algorithm might not beat an un-optimized UIA search ;)
  4. There are of course other missing features as well. One big one is IAccessible2 which I have partially implemented but not pushed to Github yet, and I'm not sure if I'll ever will (development for it has been on pause for >6 months now). It gives very fine control over Chromium-based applications, and among other things implements unique ids for elements. You might want to take a look at it if you haven't already: https://accessibility.linuxfoundation.org/a11yspecs/ia2/docs/html/

Questions

  1. AFAIK it's not possible to determine whether accessibility is enabled in a Chromium app without actually trying to get access to the Chromium renderer element. So activating it involves sending WM_GETOBJECT, and waiting for Chrome_RenderWidgetHostHWND1 to have both a name and a value, which in turn involves getting the Acc element for the renderer. Acc.ElementFromHandle(cHwnd,,False).FindElement({Role:15}, 5) is done because some Chromium windows return an incorrect element, so this tries to work around that. ActivateChromiumAccessibility activates accessibility regardless of whether Chrome was launched with --force-renderer-accessibility or not, and does indeed enable it for any arbitrary Chromium window (unless accessibility was specifically disabled in the settings, in which case it doesn't work). This works for other Chromium windows such as Spotify or Discord as well.

By the way, I strongly recommend you try out UIAutomation instead if you haven't already. It can be used from VBA as well, and is superior to MSAA in almost every way...

sancarn commented 12 months ago

Looking at your code it looks like you are simply converting bytes into hex values

Yep. I also concatenate it with other data because maaany objects don't implement it... But that's application dependent really. I'm really not a fan of the code I'm using to create a unique ID... If there's a better way in Accessibility I'm not aware of it. I think in UIA there is a similar difficulty in getting some ID which isn't runtime specific.

Tree searches using UIA are so much faster

Hmmm, that's useful to know. I didn't like how verbose UIA was, and anticipated that UIA was merely built on top of IAccessible, so performance would be equal. But if there are real benefits to using UIA over IAccessible, maybe I should consider it. I definitely had major issues when building an inspect equivalent in VBA. Thanks for the tip :)

One big one is IAccessible2

Thanks! Didn't even know this was a thing. Imagine it probably isn't very universal but still anything which provides unique IDs and finer control of chromium-based apps is a must have!

activates accessibility regardless of whether Chrome was launched with --force-renderer-accessibility or not, and does indeed enable it for any arbitrary Chromium window (unless accessibility was specifically disabled in the settings, in which case it doesn't work). This works for other Chromium windows such as Spotify or Discord as well.

Wow that is insanely helpful information to know. Will endeavor to add this to stdAcc then.

Thanks again for all the knowledge download, library looks much better than the state it was in in Acc v1. Good job!