Multiple frames - working solution found using visual automation

[Fry115]

Hi Ken Soh,

I am trying to generate an automation on a page of a bank but this page is created based on frames, I can click on the first frame but in the second I cannot, I have tried this:

frame tophead { click //[@id="hel3"] wait 3s echo 'rename Frame...' dom document.querySelector('#content').name = "content" frame content { click //[@id="el65"] wait 3s } }

Result: rename Frame... frame tophead click //[@id="hel3"] wait 3s rename Frame... frame content click //[@id="el65"] ERROR - cannot find //*[@id="el65"]

and tried this:

frame tophead { click //[@id="hel3"] wait 3s } echo 'rename Frame...' dom document.querySelector('#content').name = "content" frame content { click //[@id="el65"] wait 3s }

result: it ends well but does not click

Thanks.

[kensoh]

Hi @Fry115, there is a syntax for website frame that have a subframe -

frame mainframe_name_or_id | subframe_name_or_id

Can you share more about the frame structure on this website? Are they separate frames or one frame as a child of another frame? For using dom step to set the name, are you doing so because the frame that you want to reference has no name or id?

My initial guess base on above is the frame is a subframe and using dom with that JavaScript code to rename it doesn't work, because the node is not in the root document but under the parent frame.

There's an alternative solution another user experienced before. If there is a URL to directly access the frame you want, you can try loading the URL directly and then doing the interactions there, without using frame.

[Fry115]

Hi, The structure of the page is like this:

Going straight to the frame did not work since I don't know what put after = in frame the name "content".

With these instructions I get to "read" the name of the button, but I still can't click on it:

result:

Thanks for help me.

[kensoh]

Oh great, you are saying that your block of code to read el65 element works, but doing a click does not? If that's the case, that means the issue is isolated to be not related to frame anymore, as reading the element in the frame works.

I have not encounter this situation where clicking on something does not work. Can you check 2 things - first is the frame visible and the button visible to be clicked? second, check tagui/src/tagui_chrome.log file to see what clue is there.

Chrome will respond back to TagUI when TagUI sends a click event to it. From Chrome's response, and comparing the response for hel3 and el65, maybe there will be clues why this element can't be clicked.

[Fry115]

Hi,

At the time of sending the automation the menu is displayed but I cannot click on “Debit configuration” with xpath “//*[@id="el65"]”:

I have reviewed the file that you indicate to me and I do not observe any error or anything.

I send you the log file in case you can help me. tagui_chrome.log

Thanks.

[kensoh]

Looks like no error response from Chrome. From information so far, it could be the popup menu does not react well with the issuing of mousepress and mouserelease events to Chrome, on the position of the menu item.

I can think of 2 alternatives to try now, you can see if using JavaScript code directly on the Chrome console works to click this, and use dom step to repeat the same action. Alternatively, you can check out the visual automation method below. By specifying an image snapshot of the menu item, TagUI can move the mouse cursor to that item and trigger a low-level mouse click as if you are doing the clicking. This usually works for unique cases like this which cannot work using the normal click.

https://github.com/kelaberetiv/TagUI#visual-automation

[Fry115]

Hi, I tried with visual-automation, how is the syntax of command? only i have to create one image with the button? i tried with the syntax: and this:

but not working.

Java Version.

Thanks.

[kensoh]

Oh the image has to be .png or. bmp format. TagUI looks out for these 2 file extensions in order to detect that you are automating visually and not using Web identifiers.

[Fry115]

Hi, It works with the visual automation, thanks!

[kensoh]

Oh that's great! 😄