Can we perform action on window individual components

[duvemula]

Hi @samick17 ,

I would like to get the individual components of window and perform some action on those (like enter text, clicking buttons). Can we do with windows-automator-lib ?

Regards, Durga Prasad

[samick17]

Hi @duvemula,

Do you mean get information like Visual Studio Automation Tool: Coded UI? If so, it doesn't support this feature in current version.

Actually, at the beginning time of this wrapper was implemented, I plan to add this feature, but cannot found related API to do. I'll add this functions if spy++ library can do these. Thank you for your contibution.

reference: https://msdn.microsoft.com/en-us/library/dd460756.aspx

[duvemula]

Thank you very much @samick17 for information. I am looking forward for those function.

Regards, Durga Prasad

[samick17]

Hi @duvemula,

Maybe we can integrate with web automation framework, like nightwatch or selenium. And access the web context, to test with native mouse/keyboard API. To visualize element on browser, the only I concerned is that, "the invisible elements". Do you have any idea?

Regards, Samick

[duvemula]

Hi @samick17, we usually scroll the Invisible elements to make it visible in a web. here is the code for scrolling: var scrollIntoView = function () { arguments[0].scrollIntoView(); };

browser.executeScript(scrollIntoView, elementReference); here we need to give element reference of an invisible element above code is in protractor which is using webdriverjs to interact with the browser. please let me if I am thinking in an unrelated way. If yes would you mind could you please explain more about "the invisible elements".

Regards, Durga Prasad

[samick17]

Hi @duvemula,

This picture shows what I'm concerned. If the webUI contains some invisible elements(maybe for ad. usage). Usually, those elements will be put on the top of z-axis(z-index). So, If we make a tool for visualize all web elements, and retrieve it's info(like x-path, class, id..etc.) I think it is impossible to retrieve all elements in one step. However, if user do something to UI(make the top, full-sized element dispear), it's possible to retrieve element's info.

I just try to simplify the flow of this feature. If it's good solution, maybe we can create a new module, and integrate with this module.

Regards, Samick

[duvemula]

Hi @samick17,

We can handle web-related actions with selenium, I would like to deal with desktop windows actions

is the above concern about web window? if yes then we can handle with selenium

Please let me know if you need more information

Regards, Durga Prasad

[samick17]

Hi @duvemula,

If your test-bot can retrieve information of specified element information(relative position) in browser, You can use following code to move mouse position relative to browser left-top corner:

let hwndBrowser = 5308738
Core.setActiveWindow(hwndBrowser)
Core.mouseMoveRelative(200, 200)

But if you want to visualize element relative position on browser, this library cannot retrieve DOM element in browser, you should implement this function with selenium. And integrate this function with windows-automator-lib.

If you want to simplify grab-mouse-point flow, you can retrieve all element's relative position information, and render-rectangles, grab-mouse-point by lib.

Those two functions is not implement as a function, I can add those two functions at next version if you need.

Or you can implement it by it's functions.

Reference:

1. How to grab mouse point with global hot keys: https://github.com/samick17/windows-automator-lib/blob/master/examples/register-hotkey.js then getMousePosition

Regards, Samick

[duvemula]

Hi @samick17, the above information is about elements in a browser? if yes we can handle those with selenium/protractor.

I need elements in a desktop window and I am using core.getChildWindowsById(hwnd) it returns some child elements and but some of them have no name.

Can we get the reference of expand/collpase buttons in below screenshot

please let me know if you need more information Regards, Durga Prasad

[samick17]

Hi @duvemula,

I tried to use API: EnumProps to get window properties, the output as following picture. The details seems to be invisible.

[duvemula]

Thank you @samick17 for the information, Could you please add if we have extra functions about window properties apart from existing functions.

Regards, Durga Prasad

[samick17]

Hi,

I've updated to 1.4.8 for these function see also examples/get-child-windows.js

Samick