Hammerspoon / hammerspoon

Staggeringly powerful macOS desktop automation with Lua
http://www.hammerspoon.org
MIT License
12.01k stars 582 forks source link

hs.axuielement:searchPath gone missing #2536

Open jdtsmith opened 4 years ago

jdtsmith commented 4 years ago

I thought I should open a separate issue for this. With axuielement migrating into the core and its API changed to emphasize asynchronous operation, a terribly useful bit of functionality has apparently gone missing: searchPath. This method provided a very compact notation for "pruning" the breadth-first accessibility search layer by layer beneath a given accessibility object. An example from SplitView:

   local spaceAX=self.dockAX:searchPath({   -- a list of space controls for each screen
     {role="AXGroup",identifier="mc"},
     {role="AXGroup",identifier="mc.display"},
     {role="AXGroup",identifier="mc.spaces"}})

A couple questions:

  1. Is there an update to the Queries guide (where searchPath is described and its performance advantages touted)?
  2. Is there a reason searchPath was deprecated, or is it still to be implemented in the new API?

I presume one could "roll their own" searchPath replacement as a driver of a coroutine which iterates through a path criteria table like the above, forms a depth 1 elementSearch, whose callback continues the coroutine with the element(s) found. The coroutine then moves on to the next criterion in the path, yields a new elementSearch for each of the elements found at that layer, and so on, until the criteria table is exhausted and a final element is returned, or no element is found and an error is signaled.

latenitefilms commented 4 years ago

I'll let @asmagill explain his logic behind it, but I believe that searchPath has basically been replaced by elementSearch, which is a lot faster, and also asynchronous as you note.

I'm pretty sure searchPath was all done in Lua-land, so you could bring it back yourself in your own code if you wanted:

https://github.com/asmagill/hs._asm.axuielement/blob/3892477d44bfe91bcf5cc29c6bf6910a75de3bca/init.lua#L748

However, you should be able to achieve the same thing as what you want to do in the SplitView example using elementSearch. If you have a look at the matchesCriteria documentation you'll see you can do a table of multiple criteria's, in a similar way that you have multiple search path roles above.

For anyone who stumbles across this issue, here's the example I posted in #2510:

hs.openConsole() -- Open the Hammerspoon Console
hammerspoonPID = hs.processInfo["processID"] -- Get the Hammerspoon Process Identifier
element = hs.axuielement.applicationElementForPID(hammerspoonPID) -- Get the AXUIelement for the Hammerspoon application
criteria = "AXWindow" -- Set a criteria - see: http://www.hammerspoon.org/docs/hs.axuielement.html#matchesCriteria
criteriaFunction = hs.axuielement.searchCriteriaFunction(criteria) -- Set a criteria function - see: http://www.hammerspoon.org/docs/hs.axuielement.html#searchCriteriaFunction
callback = function(message, results, numberOfItems) -- Setup a callback - see: http://www.hammerspoon.org/docs/hs.axuielement.html#elementSearch
    print(string.format("Message: %s", message)) -- Will return "completed"
    print(string.format("Results: %s", hs.inspect(results))) -- Will return a table with a single `AXWindow`
    print(string.format("Number of Items: %s", numberOfItems)) -- Will return the number 1 (as there's only one `AXWindow` in the table above)
end
searchObject = element:elementSearch(callback, criteriaFunction) -- Starts the element search based on the above callback and criteria
print(string.format("Is the Search Object Running: %s", searchObject:isRunning())) -- Demonstrates the search object is running

In the above example, I'm just setting the criteria to be any object with a role of "AXWindow", however, I could use a table in place here, and have a much more complicated criteria to match.

Does that help at all?

jdtsmith commented 4 years ago

I'll let @asmagill explain his logic behind it, but I believe that searchPath has basically been replaced by elementSearch, which is a lot faster, and also asynchronous as you note.

I'm pretty sure searchPath was all done in Lua-land, so you could bring it back yourself in your own code if you wanted:

https://github.com/asmagill/hs._asm.axuielement/blob/3892477d44bfe91bcf5cc29c6bf6910a75de3bca/init.lua#L748

However, you should be able to achieve the same thing as what you want to do in the SplitView example using elementSearch. If you have a look at the [matchesCriteria] [snip] In the above example, I'm just setting the criteria to be any object with a role of "AXWindow", however, I could use a table in place here, and have a much more complicated criteria to match.

My understanding is that the table input for elementSearch serves as a list of attributes an individual element must match (all of):

an array table of one or more key-value tables as described immediately above; the element must be a positive match for all of the individual criteria tables specified (logical AND).

rather than a path description which allows the search to descend the hierarchy ignoring irrelevant branches. So as far as I can tell, replicating this path search requires multiple nested elementSearch's, probably, for efficiency, driven by a coroutine. Perhaps this would take as input an array-table of criteriaFunction's (which could each express multiple criteria per element arbitrarily), and do a depth-first search one level at a time, e.g. if multiple elements match per level.

Looking at that Lua-code for the old searchPath makes me realize how many subtleties have to be dealt with: first criterion does or does not match the element from which the search commences, multiple elements may match the full criteria path, etc.

The shell equivalent of such a search (expressing only filename criteria):

% ls a*/b[0-9]x/data/foo?bar
asmagill commented 4 years ago

First, no the Queries document hasn't been updated yet... not sure yet if it will be better to update it or replace it entirely, but right now the best examples of module usage are in the examples directory at https://github.com/asmagill/hs._asm.axuielement/tree/master/examples.

As noted, most things have been rolled up into elementSearch and its results table. I had personally been disappointed in the speed and responsiveness of searchPath and so it wasn't a priority to me at the time of finalizing the module for inclusion in core Hammerspoon... I'll revisit it and see if I can determine the best way to reintroduce it.

asmagill commented 4 years ago

And just to clarify -- it should be noted that elementSearch moved to a criteria function rather than a table so it can use its own logic or criteria to judge an element rather than just a list of attributes. However, in @latenitefilms example, he shows the use of searchCriteriaFunction which does take a table like you described and returns a function that elementSearch can use. You are in no way required to use searchCriteriaFunction if you want to make a more complex matching algorithm.

asmagill commented 4 years ago

In fact that gives me an idea for replicating the behavior of searchPath... let me give it some thought and see what I can come up with later tonight...

jdtsmith commented 4 years ago

I had personally been disappointed in the speed and responsiveness of searchPath

To be fair I never used it on large trees, but the idea of pruning useless branches early in a tree search was compelling, as was the comfort of specifying "the whole path" to avoid spurious matches at the leaves of a wide and not-well-explored tree. To continue the shell analogy, a more powerful path search would abstract e.g. zsh's "as many directories as necessary" globs:

% ls a*/**/b[0-9]x/data/foo?bar

use of searchCriteriaFunction which does take a table like you described

I was envisioning a table of functions (one per level in the descending path hierarchy), which themselves could each encode a table of criteria, as a way to specify a pathSearch in the new paradigm. Something like ** would require a special "just keep matching anything with children" type of function that would operate at its level or below.