ms_uia Automation ID locators

[mikenac]

One of the best about the Windows automation library is being able to use the AutomationID of a given control/window. This would be really great to have in the library.

[jarmo]

@leviwilson isn't this already supported in MsUia adapter?

[leviwilson]

Yes, this is already supported in the MsUia adapter but only for Control types (children), not as a root Window.

window = RAutomation::Window.new id: 'someAutomationId', adapter: :ms_uia # => does not work

window = RAutomation::Window.new title: /My Main Window Title/, adapter: :ms_uia
control = window.control id: 'someAutomationId' # => works

@mikenac can you be more specific about what you would like to see?

[mikenac]

Thanks. That explains why the Window locator was not working. I found what was happening with the controls on my form that could not be found by the ID. Infragistics UltraTextBox controls are of "ControlType.Pane" and not "ControlType.Text", which is why they were not being found apparently. Any suggestions on how to make this work for these controls? Thanks.

[leviwilson]

window.text(:id => 'yourUltraTextBox') # => does not work?

window.control(:id => 'yourUltraTextBox') # => should work

If you use the 2nd form in that example, it should find it regardless of the ControlType. However, that all depends on what you want to do with it :-) If your intent is to interact with it like it is a TextBox, then what you're able to do with it will depend on what Patterns it supports. Even though the UltraTextBox says that it is a pane, what Patterns does it say it implements? If it says that it implements the ValuePattern, you can get/set its value like so:

ultra_text = window.value_control(:id => 'yourUltraTextBox')
ultra_text.value # => get the value
ultra_text.set 'Some New Value'

[mikenac]

Using VisualUIVerify, it does not show that it has any supported patterns. I did get this to work:

window = RAutomation::Window.new(:title => /TransferCenter.*Login/i, :adapter => :ms_uia)
user_field = window.control(:id => 'tUser')
user_field.focus
window.send_keys("someuser")

pass_field = window.control(:id => 'tPass')
pass_field.focus
window.send_keys("somepass")

ok_button = window.button(:id => 'bOK')
ok_button.click

but after it successfully sends keys and clicks, there is a ruby segmentation fault with a bunch of jazz after:

RA_ElementFromHandle: Cannot find element from handle 0x180df2. HRESULT was 0x80040201 

[mikenac]

@leviwilson your code also worked so it must support the ValuePattern. It does, still however, cause the same segmentation fault.

[leviwilson]

My code worked to actually set the value? Which line caused it to crash?

[leviwilson]

Could you provide an example app that has one of these controls that shows it crashing?

[mikenac]

@leviwilson I have one, but how can I get it to you?

[leviwilson]

Dropbox? GitHub Repository? Whatever is fine.

[mikenac]

Try this: https://www.dropbox.com/sh/rmhonimnu4ijzzz/j3NIm680Wi

The project is compiled as it may not let you do that without an Infragistics license. The source code is also included, as is the ruby file used for the test.

[leviwilson]

Cool, I'll take a look at it when I get some free time.

[leviwilson]

The problem is in the click method. click takes a block to indicate when it's done. Here is the listing that will fix your issue:

require "rautomation"

window = RAutomation::Window.new(:title => /LoginForm/i, :adapter => :ms_uia)

ultra_text = window.value_control(:id => 'tUser')
puts ultra_text.exist?
puts ultra_text.value # => get the value
ultra_text.set 'someuser'

ultra_text = window.value_control(:id => 'tPass')
puts ultra_text.exist?
puts ultra_text.value # => get the value
ultra_text.set 'somepass'

ok_button = window.button(:id => 'bOK')
ok_button.click { true } # this fixes the crash

You can see here that it will quit if the window doesn't exist? anymore, but it's still trying to use the HWND value after the form has went away (because you clicked OK).

[jarmo]

I'm not sure i understand why the crash occurred when #click without block was used. You're saying that clicking on the ok button will close the form, hence #exist? should return false and everything should be working as expected. Or are you saying that #exist? itself will try to use non-existing HWND and raise that error?

[leviwilson]

I think this is a result of the #cached_hwnd in the :ms_uia adapter. I'm guessing when AutomationElement.FromHandle is called is when it is happening, and the UiaDll doesn't trap the exception (or send it back to ruby) and so ruby takes a :poop:.

[jarmo]

Ok, now i understand it after you've created an issue for that :) I felt that this behavior needs an issue to be created, that's why i didn't understand the solution to this issue here.

[rabos5]

I realize this issue is closed and but the initial question posted by mikenac is what drew my attention. I'm looking to automate a windows WPF application where incoming chat requests to the app are received/accepted through popup windows where the title of that popup window change based on the name of the customer initiating the chat request. The AutomationID obviously does not. Being able to control that window by the AutomationID would be great since it doesn't change per incoming chat. Is this currently not possible or are there any good work arounds you'd suggest?

Ideal: window = RAutomation::Window.new(:id => 'incoming_chat_window_id', :adapter => :ms_uia)

[jarmo]

Can't you use a regexp to match a title? For example if the window title is something like "customer 1 chat window" then find that window like this:

RAutomation::Window.new(:title => /chat window$/, :adapter => :ms_uia)

[rabos5]

In this particular case, the window title is only the customer name. Not a huge issue because

[rabos5]

Sorry, it looks like my phone posted the comment before I was finished my thoughts :).. In this particular case, the window title is only the customer name. It's not a huge issue because when we initiate an incoming test chat we pass in that customer name. I feel like it would be a bit more ideal though in some cases to control that window using the automationid. This stuff works great though so I definitely appreciate all the work that went into it. Thanks for the quick response also!

Ryan