Issue with Microsoft Teams: no reaction on button click

[tdalon]

Hi Descolada, I have tried followind scenario with your UIAutomation Library and Microsoft Teams, see https://tdalon.blogspot.com/2022/09/ahk-uia-teams-meeting-actions.html with the code available here: https://gist.github.com/tdalon/a3aa353a1738e3f6f76d861075471241

The elements are properly found but the problem I have is that no action happens when calling: ActionEl.Click() (this is clicking on the fullscreen button.

I would appreciate if you had any help, insights on this because I am pretty stuck right now ;-)

[Descolada]

Hello, Your code snippet seems to be missing some functions like Teams_GetMeetingWindow and the ClickAction routine isn't called anywhere, but for me this snippet worked:

#include <UIA_Interface>

winTitle := "New channel meeting" ; Teams_GetMeetingWindow()
WinActivate, % winTitle
WinWaitActive, % winTitle
UIA := UIA_Interface()
teamsEl := UIA.ElementFromHandle(winTitle)
; Make fullscreen
teamsEl.FindFirstBy("AutomationId=callingButtons-showMoreBtn").Click()
teamsEl.WaitElementExist("AutomationId=fullscreen-button").Click()
Sleep, 2000
; Restore normal window
teamsEl.FindFirstBy("AutomationId=callingButtons-showMoreBtn").Click()
teamsEl.WaitElementExist("AutomationId=fullscreen-button").Click()
ExitApp

Your codes equivalent should probably be Teams_MeetingAction("fullscreen"). If you haven't updated UIA_Interface.ahk for a while, I suggest doing that as well.

Though you can do this without UIA as well:

winTitle := "New channel meeting" ; Teams_GetMeetingWindow()
WinActivate, % winTitle
WinWaitActive, % winTitle
ControlSend,,{F11}, % winTitle
Sleep, 2000
ControlSend,,{F11}, % winTitle

[tdalon]

I thought this would work without requiring the window to be activated. Even if I activate the meeting window before, it is working sporadically for me. Sometimes it sets to fullscreen, sometimes not, most likely not. The toggle reset fullscreen seems to never work for me this way though.

Maybe some strange Windows 10/ Teams App behavior. (I have updated the UIA library to latest version.)

The F11 hotkey works also indeed to toggle the fullscreen mode. It seems it isn't documented by Microsoft.

The idea would be if it works for fullscreen this way with UIA to get other shortcuts for other meeting actions.

Many thanks for your help/ much appreciated :-)

[Descolada]

I think this is the same "strange behavior" that we have encountered previously, where the menu buttons aren't always visible to UIA. This is probably related to some kind of bug in Microsofts' implementation of UIAutomation or a bug in Chromium UIA implementation, since Acc can find those elements reliably (though the window needs to be active). Unfortunately this means I am unable to fix this, nor have I found any good workarounds...

[tdalon]

The particularity here is that the element is found properly in a reliable way _but the click function has no effect/ does not trigger the expected action. So not totally the same as the previously encountered issue.

[Descolada]

Hmm, okay, that is actually different. Unfortunately I couldn't reproduce it - I ran my example about 10 times and it was very reliable. Are there any specific steps to reproducing it? Also which part is not working exactly - pressing the "callingButtons-showMoreBtn" or "fullscreen-button"?

[tdalon]

I have tried also with your code. On the click on the Fullscreen button, it works sporadically only to set it and nothing happens if Fullscreen is already on.

[Descolada]

What happens when you create a new meeting with the title "New channel meeting" and run the following code:

#SingleInstance, Force
SendMode Input
#Persistent
SetWorkingDir, %A_ScriptDir%
SetTitleMatchMode, 2
SetBatchLines, -1

#include <UIA_Interface>

winTitle := "New channel meeting" ; Teams_GetMeetingWindow()
WinActivate, % winTitle
WinWaitActive, % winTitle
UIA := UIA_Interface()
teamsEl := UIA.ElementFromHandle(winTitle)
Loop, 5 {
    ToolTip, Try number %A_index%
    ; Make fullscreen
    teamsEl.FindFirstBy("AutomationId=callingButtons-showMoreBtn").Highlight().Click()
    teamsEl.WaitElementExist("AutomationId=fullscreen-button").Highlight().Click()
    Sleep, 2000
    ; Restore normal window
    teamsEl.FindFirstBy("AutomationId=callingButtons-showMoreBtn").Highlight().Click()
    teamsEl.WaitElementExist("AutomationId=fullscreen-button").Highlight().Click()
    Sleep, 2000
}
ExitApp

[tdalon]

When the meeting is already on fullscreen the click on the more button (now highlighted) does not open the menu with the other buttons like fullscreen.

When not in fullscreen, it works properly: buttons are highlighted and clicked with effect.

This is weird: if I highlight the More button, the fullscreen button then work. but if I skip Highlighting the More button the click on the fullscreen button does not work.

[Descolada]

@tdalon okay so whatever reason, the More button requires a small sleep after finding it and before clicking.

moreEl := teamsEl.FindFirstBy("AutomationId=callingButtons-showMoreBtn")
Sleep, 100
moreEl.Click()
teamsEl.WaitElementExist("AutomationId=fullscreen-button").Click()

This worked fairly reliably for me. A smaller sleep (about 50ms) tended to fail sometimes. I have NO explanation for why this is necessary, because it hasn't been in any other application I've tried. Another quirk of Teams?

[tdalon]

For me this isn't working. I have played around with some extended pauses. If the Window is not in fullscreen, the action Element (teamsEl.WaitElementExist("AutomationId=fullscreen-button")) is found but the click has no effect. If the Window is in fullscreen the More Element is found but the click has no effect i.e. does not open the menu.

---

.Highlight works but not if the window is set to my secondary screen. Is it a bug in the highlight method?