[🐛 Bug]: Unable to Drag And Drop on DesktopLibrary

[jsperafico]

What happened?

Hey everyone, First of all congratulations for your work so far! It's quite an accomplishment!

If anyone would be kind enough to investigate why I'm unable to drag and drop an ui component within the same window. Currently we have a WPF application that renders a node graph. Manually we can drag and drop the nodes around.

On my automation, I have tried the following:

    ${node_text_1}        Set Variable            Text[@ClassName="TextBlock"][@Name="Something"]
    ${node_text_2}        Set Variable            Text[@ClassName="TextBlock"][@Name="Something else"]
    Click Element          xpath=//${node_text_1}
    Drag And Drop       xpath=//${node_text_1}      xpath=//${node_text_2}

    ${node_text_1}        Set Variable            Text[@ClassName="TextBlock"][@Name="Something"]
    ${node_text_2}        Set Variable            Text[@ClassName="TextBlock"][@Name="Something else"]
    Drag And Drop       xpath=//${node_text_1}      xpath=//${node_text_2}

Unfortunately, while running the automation, seems the action never happened in the first place. It only place the mouse over the target.

Any advice that could be the problem?

Relevant log output

<kw name="Drag Node Under Another"">
  <arg>${node_1.Properties.Name}</arg>
  <arg>${node_2.Properties.Name}</arg>
  <kw name="Click Element" library="Zoomba.DesktopLibrary">
    <arg>xpath=//${node_text_1}</arg>
    <doc>Click element identified by `locator`.</doc>
    <msg timestamp="20220329 14:12:41.214" level="INFO">Clicking element 'xpath=//Text[@ClassName="TextBlock"][@Name="Something"]'.</msg>
    <status status="PASS" starttime="20220329 14:12:41.212" endtime="20220329 14:12:44.007" />
  </kw>
  <kw name="Drag And Drop" library="Zoomba.DesktopLibrary">
    <arg>xpath=//${node_text_1}</arg>
    <arg>xpath=//${node_text_2}</arg>
    <doc>Drags the element found with the locator ``source`` to the element found with the
locator ``target``.</doc>
    <msg timestamp="20220329 14:12:49.079" level="INFO">Dragging source element "xpath=//Text[@ClassName="TextBlock"][@Name="Something"]" to target element "xpath=//Text[@ClassName="TextBlock"][@Name="Something else"]".</msg>
    <status status="PASS" starttime="20220329 14:12:44.007" endtime="20220329 14:12:49.311" />
  </kw>
</kw>

Operating System

Windows 10 - WPF-based app

RobotFramework-Zoomba version (tag)

2.14.1

[Wolfe1]

Hey @jsperafico,

I am wondering if we may just have an issue with the locators. Could you use https://accessibilityinsights.io/ and post what the properties and the pathing for one of those squares looks like?

I am just wondering if there is an easier path here than xpath. Otherwise what are you using to get that Xpath? If it is the WinAppDriver UI Recorder could you give me the full xpath it gives you when you select that item?

https://github.com/microsoft/WinAppDriver/releases/tag/UIR-v1.1

[jsperafico]

Hey @Wolfe1, Hopefully this can help...

WinAppDriver UI Recorder:

"/Pane[@ClassName=\"#32769\"][@Name=\"Desktop 1\"]/Window[@Name=\"-------------------\"][@AutomationId=\"mainPageOrganiser\"]/Window[@Name=\"-------------------\"][@AutomationId=\"-------------------\"]/Text[@ClassName=\"TextBlock\"][@Name=\"East\"]"

Windows Inspect Tool:

Accessibility Tool:

I have tried to click on both elements and works fine. Just the Drag & Drop isn't. Anything else that I can provide you with? Thanks a lot for the help!

[Wolfe1]

"I have tried to click on both elements and works fine. Just the Drag & Drop isn't. Anything else that I can provide you with?"

Ah understood, in that case, I guess now I just need some kind of a demo application where I can replicate this issue. I am assuming that since you redacted the window name that this is an internal application?

Also provided your blocks all have unique names I assume you could just use something like this rather than xpath (should work faster as well):

Click Element name=East

[Wolfe1]

@jsperafico Also I know in another issue you mentioned not using touch actions but could you try giving Drag and Drop by Touch a try?

[Wolfe1]

Any update @jsperafico ?

[jsperafico]

Hey @Wolfe1, Unfortunately:

• our dev team won't be able to create a simple app that represents this test case.
• also the team decided to stay with my work-around when it comes to scrollable content (by click on the down arrow every now and then). Sorry not being able to help you more to keep improving this library. I will make sure whenever I have time, I will circle back to those topics and if a solution is implemented to share the code with you all. Thanks you for your support and once again sorry not being helpful.

[Wolfe1]

Code moved to https://github.com/Accruent/robotframework-applicationlibrary so losing this issue.