robocorp / rpaframework

Collection of open-source libraries and tools for Robotic Process Automation (RPA), designed to be used with both Robot Framework and Python
https://www.rpaframework.org/
Apache License 2.0
1.09k stars 210 forks source link

Tools tips in windows? #1141

Open bruce-optibrium opened 6 months ago

bruce-optibrium commented 6 months ago

I am experimenting with RPA.windows. I don't grok the robot syntax at all I'd rather use pure python but I don't grok the python API either yet and it isn't as well documented.

Looking with accessiblity insights I see some text called "FullDescription" which corresponds to a tooltip but there is no matching attribute if I list the attributes of that element.

Are there some limitations to robot here or is there a way to get this? An example of an element with this is the find button in a word document.

image

Here is a hacky script to list the attributes via RPA.Windows

from RPA.Windows import Windows 

lib=Windows()
tree = lib.print_tree("class:NetUIRibbonButton", return_structure=True)
p.pprint(tree)

items = lib.get_elements("class:NetUIRibbonButton")
p.pprint(items);

for item in items:
   attributes = lib.list_attributes(item)
   p.pprint(attributes);
   for attr in attributes:
      p.pprint(attr);
      result = lib.get_attribute("class:NetUIRibbonButton", attr)
      if result is not None:
         print("attr={}".format(result))

This isn't a great example as "NetUIRibbonButton" might correspond to other elements. Though it reproduces the error on the custom application I'm looking at where the element is sufficiently unique.

I also tried using the automation id as a locator for this example like "id:NavigationPaneFind" and it doesn't work. What are some more correct ways to do this?

Anyway this lists the attributes as:


[   'AcceleratorKey',
    'AccessKey',
    'AriaProperties',
    'AriaRole',
    'AutomationId',
    'BoundingRectangle',
    'ClassName',
    'ControlType',
    'ControlTypeName',
    'CreateControlFromControl',
    'CreateControlFromElement',
    'Culture',
    'Element',
    'FrameworkId',
    'HasKeyboardFocus',
    'HelpText',
    'IsContentElement',
    'IsControlElement',
    'IsDataValidForForm',
    'IsEnabled',
    'IsKeyboardFocusable',
    'IsOffscreen',
    'IsPassword',
    'IsRequiredForForm',
    'ItemStatus',
    'ItemType',
    'LocalizedControlType',
    'Name',
    'NativeWindowHandle',
    'Orientation',
    'ProcessId',
    'ProviderDescription',
    'ValidKeys',
    'foundIndex',
    'regexName',
    'robocorp_click_offset',
    'searchDepth',
    'searchFromControl',
    'searchInterval',
    'searchProperties']

None seem to correspond to "FullDescription".

kkotenko commented 6 months ago

At a first glance, it appears to me that the underlying Python API of uiautomation does not expose this property. However, you may be able to add it to (your local copy of) the uiautomation.py file:

https://github.com/yinkaisheng/Python-UIAutomation-for-Windows/blob/abb247330bcba4fdb9244f669f451edcab5f4125/uiautomation/uiautomation.py#L6003

    @property
    def FullDescription(self) -> str:
        """
        Property FullDescription.
        Call IUIAutomationElement::get_CurrentFullDescription.
        Return str, such as Win32, WPF...
        Refer https://learn.microsoft.com/en-us/windows/win32/api/uiautomationclient/nf-uiautomationclient-iuiautomationelement6-get_currentfulldescription
        """
        return self.Element.CurrentFullDescription

That should also add it to the above attribute list, I think. (Note I am not a maintainer or anything, I just have happened to look under the hood.)