Observer breaks after resuming from sleep

[iliyang]

In my Hammerspoon script, I have an observer that watches an AXTextArea element of an application (the Spark email client; it's the unread messages counter) for AXSelectedTextChanged events. It all works fine, but when I put the computer to sleep and resume, the observer doesn't receive events anymore – I have to re-hook it (which I do by manually reloading my Hammerspoon config).

Inspecting the key of the sole entry in the my_observer:watching() table, I get:

<userdata 1> -- hs._asm.axuielement: *AXRole undefined*

After reloading the Hammerspoon config, the value of the key is as exepcted:

<userdata 1> -- hs._asm.axuielement: AXTextArea (0x60004041a878)

So something gets broken at some point. I think this issue also occurs without sleeping the computer, when using it for a prolonged time (keeping the application being watched open all the time). Conversely, putting the computer to sleep for only a few seconds, the issue doesn't occur. But at least I get a consistent broken behavior every morning when I open the Macbook: the counter in my custom menu bar is different from the value of Spark's UI element.

Could it be an issue somewhere in hs._asm.axuielement.observer?

[latenitefilms]

The AX object will have been destroyed when the Mac is put to sleep.

When you say the "unread messages counter" are you referring to the Dock icon? If so, you could add another observer to watch when the Dock AX items are created/destroyed, and then create your AXTextArea observers when that's triggered.

[iliyang]

The AX object will have been destroyed when the Mac is put to sleep.

Oh thanks, this is useful information! Do you know if there's a way to detect the destruction of the object (i.e. via some event), or perhaps the destruction of a parent AX object? Otherwise, I'll have to watch for resume-from-sleep events to re-hook the observer. (And maybe the AX object gets destroyed in other circumstances too?)

When you say the "unread messages counter" are you referring to the Dock icon? If so, you could add another observer to watch when the Dock AX items are created/destroyed, and then create your AXTextArea observers when that's triggered.

This was the first thing I tried actually, and this is a much more general method that works with every application (e.g. chat apps like Rambox, Franz, etc). While I can watch for Dock AX item creation/destruction, there the Dock doesn't expose that notify about changes to the badges of individual apps. The best thing I've found is to periodically check the badge for the app of interest using the function

function getDockBadge(appName)
    dockApp = hs.application.find('com.apple.dock')
    dockAppEl = ax.applicationElement(dockApp)

    for _, element in ipairs(dockAppEl[1]) do
        if element:AXTitle() == appName then
            return element:AXStatusLabel()
        end 
    end

    return nil
end

but polling is not an ideal approach – it comes with a performance/latency trade-off.

If someone knows a way to get notifications about dock badge changes, it would be fantastic.

[latenitefilms]

You should be able to use the same method you're currently using - the observer should trigger an AXUIElementDestroyed notification when an object is destroyed.

I find UI Browser is great for testing things out with the Accessibility API.

[iliyang]

You should be able to use the same method you're currently using - the observer should trigger an AXUIElementDestroyed notification when an object is destroyed.

Ah thanks! The AXUIElementDestroyed notification works, though only if I close the window. It's not sent if I quit the app via Cmd+Q. This could be specific to this app (Spark), and the app termination event can be handled through built-in Hammerspoon application event handling.

I find UI Browser is great for testing things out with the Accessibility API.

Nice, thanks. I've been using Xcode's the Accessibility Inspector, which doesn't look as nice but is free.

[asmagill]

I'm getting close to incorporating hs.axuielement into the next Hammerspoon release and want to close out (or move, if they're still an issue) as many of these as possible. Please let me know if you still consider this an active issue.

I'm also including some general notes due to syntactic changes that have occurred as the module readies for integration and that came to mind while reading through this issue:

• The naming conventions and dynamically generated functions/attributes has been formalized and the example function show above would now have to be written as follows (note I am not making any comments here as to the functions accuracy/sufficiency or its intent or purpose -- this is purely a syntactic update):

  function getDockBadge(appName)
  -- dockApp = hs.application.find('com.apple.dock') -- no longer required; applicationElement can take strings
  dockAppEl = ax.applicationElement('com.apple.dock')
  
  for _, element in ipairs(dockAppEl[1]) do
      if element.AXTitle == appName then -- attributes are now accessible as key'd members of the userdata
          return element.AXStatusLabel -- attributes are now accessible as key'd members of the userdata
      end 
  end
  
  return nil
  end

• In theory, hs.axuielement.observer can watch AXCreated which might help you avoid polling; however, when I attempted to use this on the Dock AXApplication element itself to detect when the AXGroup item for Expose and MissionControl was added to the Dock's AXChildren attribute, it never fired... I don't know yet if this is because I'm mis-understanding something or if this is because the developers didn't think to trigger it when these elements are being added... still, it might be worth attempting on the specific dock items you want to get the status labels for.

• As noted above, when an accessibility item that is being watched is destroyed, the observer isn't notified that the element is no longer valid. At present, I know of no way to detect this without occasional polling; hs.uielement (which uses an approach similar to hs.axuielement.observer but hides a lot of the details) addresses this (partially) by using hs.application.watcher (which does not use hs.uielement) and stops and purges any watchers for applications that have terminated. Of course this only addresses the case where the element is destroyed because the application itself terminated. Long term, I'd like to come up with a way to detect invalid elements within watchers, but don't have an eta.

• examples/axbrowse.lua can be used as a poor man's element browser (I have it tied to a hotkey myself)... it's not nearly as full featured as a dedicated app, but when you're just poking around or need a reminder of what something is named or where it's located, it may serve in a pinch.

[asmagill]

Hmm... looks like AXStatusLabel is a string attribute and not one referring to a new AXUIElement... so I'm going to go out on a limb here and say that I don't think "AXCreated" will detect it.

[iliyang]

Thanks for your continued work on this, @asmagill! Great news that hs.axuielement is planned for inclusion in the main release! I consider the original case to be closed due to my better understanding of the limitations. Seems like a lot of these come from the system (like AXStatusLabel being an attribute and not an element, for example).

Some dock replacement/enhancement apps like uBar and Pock do show app badges but I think they just poll the attribute at intervals.