Not able to inspect the UI elements from the Microsoft login webview

[RRaulAndrei]

We are trying to automate some tests on an Android app using Appium and we are not able at all to interact with the UI elements from the Microsoft login webview.

The issue is reproducing only on Android real devices, on emulator it is working fine.

• Devices: Samsung Galaxy Tab Active 3, Lenovo Tab M10 Plus
• Android Version: 10
• Appium version 1.21

I found a related issue opened on stack overflow - https://stackoverflow.com/questions/65355614/appium-android-not-able-to-find-the-login-elements/68622334#68622334

[rpdome]

@RRaulAndrei MSAL supports both Chrome custom tabs and Webview. (The difference is that You'll see a 'top bar' in Chrome Custom tabs - as seen in the image in the attached thread).

MSAL will pick what to use based on the following criteria

1. If Broker app is installed (Authenticator/ Company Portal), and MSAL is NOT opted out from brokered auth, then WebView will always be used.
2. By default, MSAL will use Chrome Custom Tabs (this is configurable in the MSAL configuration file). However, if no browser in MSAL's browser safe list is installed, it will fall back to using WebView.

May I ask if Android webview are being used for both emulator and real devices?

Having MSAL logs and screenshots on both emulator and device might help.

[RRaulAndrei]

Hi @rpdome! Thank you for your input and sorry for the late response.


I did more investigations based on your comment and I have the following information.

We are using webview on both device and emulator.

useEmbeddedWebview flag is set as true.

I will also provide the logs from both device and emulator. What can be seen there is that within device logs we have anything regarding chromium while on emulator we have. However, we can see that on the device the webview is initialized with success: _[WebViewFactory] Loading com.google.android.webview version 92.0.4515.159 (code 451515933) [crWVCFactoryProvider] Loaded version=92.0.4515.159 minSdkVersion=29 isBundle=true multiprocess=true packageId=2

Also this is the way we are consctructing the PublicClientApplication: var builder = PublicClientApplicationBuilder .Create(AppConfiguration.Current.ClientId) .WithLogging(Log, LogLevel.Verbose, true) .WithTenantId(AppConfiguration.Current.TenantId);

DeviceLogs-Verbose.txt EmulatorLogs-Verbose.txt

[RRaulAndrei]

Hi @rpdome , Any updates on this one? Do you have in pipeline to address this bug/enhancement? Or can we contribute to it somehow?

It is impacting us a lot as we cannot automate tests on a real android device for now and we have to use an emulator which is not the fastest way to run them.

[rpdome]

@RRaulAndrei I just filed a bug to our server team. Will keep you updated once we hear back. Thanks!

[RRaulAndrei]

Hi @rpdome, any updates on this one?

[dragojs]

We've sort of hit this wall again; can't find the password field and the Sign In button anymore on this screen:

Last Thursday, we would still be able to find them with these Xpaths:

    [FindsByAndroidUIAutomator(XPath = "//*[@resource-id=\"i0118\"]")]
    private IMobileElement<AppiumWebElement> _passwordField;

    [FindsByAndroidUIAutomator(XPath = "//*[@resource-id=\"idSIButton9\"]")]
    private IMobileElement<AppiumWebElement> _signInButton;

We have found a workaround for our particular use cases on this screen, although it's not quite ideal.

[dragojs]

Today, everything seems to be completely hidden behind a generic webview:

We could initially refer to individual elements from further down into the hierarchy but now they are gone. @rpdome any thoughts ?

[aabirdaneshyar]

Hello @dragojs ,

I am also getting the same issue. Do you know a workaround for this ?

Thanks.

[dragojs]

Hi @aabirdaneshyar,

Yes; We are banking on the fact that the password textbox is focused when we reached this screen and the keyboard is already displayed. If it's not in your case, you may want to try and tap the lightbox element in the center. Afterwards, we do something like:

// Sit in a loop until the keyboard is displayed so we don't start typing the password too early
WaitFor(() => Driver.IsKeyboardShown()); 
// Using Actions, we call SendKeys to populate the textbox
 new Actions(Driver).SendKeys(password).Perform(); 
 // Tap "Enter" to finish the input and move on
 Driver.PressKeyCode(AndroidKeyCode.Enter); 

 // Alternatively, if tapping enter doesn't work, you may want to try ExecuteScript like this:
 // Driver.ExecuteScript("mobile: performEditorAction", new Dictionary<string, string> { { "action", "Go" } });

[aabirdaneshyar]

Thanks @dragojs for the response.

Following worked for me -

driver.executeScript("mobile:performEditorAction", ImmutableMap.of("action", "T"));
driver.pressKey(new KeyEvent(AndroidKey.E));
driver.pressKey(new KeyEvent(AndroidKey.S));
driver.pressKey(new KeyEvent(AndroidKey.T));
driver.pressKey(new KeyEvent(AndroidKey.DIGIT_1));
driver.pressKey(new KeyEvent(AndroidKey.DIGIT_2));
driver.pressKey(new KeyEvent(AndroidKey.DIGIT_3));
driver.pressKey(new KeyEvent(AndroidKey.TAB));
driver.pressKey(new KeyEvent(AndroidKey.TAB));
driver.pressKey(new KeyEvent(AndroidKey.ENTER));

[aabirdaneshyar]

@rpdome , @RRaulAndrei ,

We are getting this issue again. Should we raise a fresh ticket for this issue ?