search

Descolada / UIA-v2

UIAutomation library for AHK v2, based on thqby's UIA library
MIT License
168 stars 25 forks source link

Using condition in tree walker gets the wrong element #2

Closed enaaab460 closed 1 year ago

enaaab460 commented 1 year ago

image I am trying to get the text element "CARDIOLOGY CHAPTER" from link element "Lec 2 Angina Pectoris" using treewalker. When using linkUiaElement.WalkTree("-1",{T:20}) i get the "1. Lec 1 Introduction" text element, which is not the expected behaviour. If I understand correctly, this is not a sibling, it is a nephew (xd). I would reach this element using the path "-1,3" When using linkUiaElement.WalkTree("-1") i get the "  1. Lec 1 Introduction" link element, which is the expected behaviour since it is the previous sibling, but I want to use the filter condition to choose the Last text sibling element previous to the pointed link element.

Is there a problem in my code or is this a bug? I understand that in this very example, i can use linkUiaElement.WalkTree("-2"), but this list is long complex with different patterns of link and text elements that I need to dynamically figure out the last text sibling for a group of link elements

Descolada commented 1 year ago

Hello, This is the expected behavior when using a TreeWalker with a filter-condition, because the tree gets "flattened" when using a condition. So even though • • 1. Lec 1 Introduction Text element is a child of a Link element, after flattening the tree it will be on the same level as your target Text element and will get selected instead.

I see two possible ways of tackling this problem: 1) Loop linkUiaElement.WalkTree("-1") until you find a Text-type element 2) Use FindElement from the parent of the "Angina Pectoris" element, with TreeScope Children and startingElement "Angina Pectoris": AnginaPectorisParent.FindElement({Type:"Text", i:-1, startingElement:AnginaPectorisLinkElement, scope:"Children"})

enaaab460 commented 1 year ago

The loop way works the findelemnt way matches the "Lec 2 Angina Pectoris" text element (the very last item in the screenshot, and the last child of the "Lec 2 Angina Pectoris" link element)

Descolada commented 1 year ago

The FindElement way should work as well, but make sure you are calling it on the parent of the element that is selected in the screenshot. If you are do that and using TreeScope Children, then it's not possible to match "Lec 2 Angina Pectoris" Text element.

enaaab460 commented 1 year ago

image Even when I tried to change the parent element several times it ways matched this element I tried to take a video/screenshot of the target program but it always appears black LinkElement is the selected Lec 2 Angina Pectoris

Descolada commented 1 year ago

Interesting, I will look into it. Is there any way for me to access that program for testing purposes?

enaaab460 commented 1 year ago

It is my college's online lectures' app, with tight lockdown on sign in access :(. to the point that i need to contact IT if i want to log in from another device. And the account has sensitive ID information.

Descolada commented 1 year ago

Okay. I'll try to find something similar on the web, but if you find an example first, please share it here.

enaaab460 commented 1 year ago

I tried replicating the script on your script's wiki and it works as expected:

WinActivate "Chrome" program := UIA.ElementFromWindow("Chrome") listElementParent := program.ElementFromPath({ T: 30 }, { T: 26, A: "js-repo-pjax-container" }, { T: 26, A: "wiki-pages-box" }) listElementParent.Highlight(1000) listElement := listElementParent.ElementFromPath({ T: 8 }) listElement.Highlight(1000) targetLast := listElementParent.FindElement({ Type: 20, i: -1, startingElement: listElement, scope: "Children" }) targetLast.Highlight

image the selected list is the list element I am not sure if this is equivalent to my first scenario

enaaab460 commented 1 year ago

New Update: I think the problem might be the tree itself I tried linkElement.Walktree("P1") and the parent I am getting is a group, even though the tree shows the parent as a document image The parent's first child is a group, could this be related? UIA shows the linkelements' path for verification Update 2: Group parent: image Document parent (ClassName:"Chrome_RenderWidgetHostHWND"): image

highlighting this new parent (nameless group) shows a more localised highilight over the Lecture list than the old parent (beIN document, which showed the whole screen). But still same outcome from the FindElement method, the chapter should have been Type: 50020 (Text) Name: "CARDIOLOGY CHAPTER 1 IHD" LocalizedType: "text"

image This is how the program looks, I want to match the chapter name above the lecture link.

enaaab460 commented 1 year ago

Looks like the looping method will be the right way to do this Thanks alot for your help!!!

Descolada commented 1 year ago

The issue appears to be instead with the internally used method FindFirstWithOptions, which is very sparsely documented. I had assumed that when using the root argument, the search scope would be applied in relation to the root and the first validated element would be the element the method was called from. Instead the search scope is scope for the element it's called from, but TreeScope Descendants for the root element. Whether this should be the expected behavior or a bug on Microsofts part, I am not yet sure and will be submitting some queries to Microsofts Q&A. I'll be keeping this issue open until I get a response from there.