karatelabs / karate

Test Automation Made Simple
https://karatelabs.github.io/karate
MIT License
8.13k stars 1.94k forks source link

[Appium] Appium Driver does not wait properly and missing features #2025

Closed mathissprlch closed 2 years ago

mathissprlch commented 2 years ago

Hi there, I am writing UI Tests for an app (cross platform) and want to use Karate for this task. After writing a simple test case in java with the native java client for Appium and the same test case in Karate, I notice that a waitFor in Karate (waiting for an element to be enabled, i.e. in my case clickable) does not work as expected. Karate clicks prematurely and naturally all my subsequent actions fail.

My workaround is currently to wait 4 seconds in the background part of the .feature before any action is taken in the scenarios, but this is not a permanent solution for me.

Testcase in java ```java package appium; import java.net.MalformedURLException; import java.time.Duration; import java.util.ArrayList; import org.junit.After; import org.junit.Before; import org.junit.Test; import org.openqa.selenium.By; import org.openqa.selenium.WebElement; import org.openqa.selenium.remote.DesiredCapabilities; import org.openqa.selenium.support.ui.ExpectedConditions; import org.openqa.selenium.support.ui.WebDriverWait; import io.appium.java_client.android.AndroidDriver; import io.appium.java_client.service.local.AppiumDriverLocalService; import io.appium.java_client.service.local.AppiumServiceBuilder; import io.appium.java_client.service.local.flags.GeneralServerFlag; public class SampleTest { private AndroidDriver driver; private AppiumDriverLocalService service; private final int WAIT_TIME_SECONDS = 5; protected void clickAndWait(By by) { WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(WAIT_TIME_SECONDS)); WebElement el = wait.until(ExpectedConditions.elementToBeClickable(by)); el.click(); } @Before public void setUp() throws MalformedURLException { System.out.println("Start Setup"); java.io.File nodefile = new java.io.File("C:\\Program Files\\nodejs\\node.exe"); service = new AppiumServiceBuilder() .withIPAddress("127.0.0.1") .usingPort(4723) .usingDriverExecutable(nodefile) .withArgument(GeneralServerFlag.ALLOW_INSECURE, "chromedriver_autodownload") .build(); System.out.println("service URL:" + service.getUrl()); service.start(); DesiredCapabilities desiredCapabilities = new DesiredCapabilities(); desiredCapabilities.setCapability("appium:deviceName", "emulator-5554"); desiredCapabilities.setCapability("platformName", "android"); desiredCapabilities.setCapability("appium:appPackage", "com.senecapp.android.abnahme.debug"); desiredCapabilities.setCapability("appium:appActivity", "de.enbw.pvplus.jamitlabs.ui.main.SenecBootstrapActivity"); desiredCapabilities.setCapability("appium:noReset", false); desiredCapabilities.setCapability("appium:ensureWebviewsHavePages", true); desiredCapabilities.setCapability("appium:nativeWebScreenshot", true); desiredCapabilities.setCapability("appium:newCommandTimeout", 3600); desiredCapabilities.setCapability("appium:connectHardwareKeyboard", true); desiredCapabilities.setCapability("appium:automationName", "UiAutomator2"); // desiredCapabilities.setCapability("appium:autoWebview", true); // desiredCapabilities.setCapability("app", "__application_path_or_name__"); driver = new AndroidDriver(service.getUrl(), desiredCapabilities); } @Test public void sampleTest() throws InterruptedException { clickAndWait(By.id("com.senecapp.android.abnahme.debug:id/accept_button")); clickAndWait(By.id("com.senecapp.android.abnahme.debug:id/login_button")); clickAndWait(By.id("com.senecapp.android.abnahme.debug:id/accept_all")); } @After public void tearDown() { driver.quit(); } } ```
Same Testcase in Karate ```gherkin Feature: android test Background: App Preset * def appium2ServerConf = { type: 'android', webDriverPath : "/", start: false, httpConfig : { readTimeout: 120000 }} * configure driver = appium2ServerConf * def appPackage = androidCaps.desiredCapabilities['appium:appPackage'] * def ById = function(id) { return '#' + appPackage + ':id/' + id } * driver { webDriverSession: "#(androidCaps)" } # this is where I wait currently to avoid the problem. without this sleep, the test fails * driver.getOptions().sleep(4000) Scenario: android login waitFor # this is where the driver does not wait and just proceeds to the click, which it thinks is successfully performed and moves on When driver.waitFor(ById('accept_button')) Then driver.click(ById('accept_button')) # this is where the test fails then, as it can't find the login button, the app is not on that page yet because accept was not clicked When driver.waitFor(ById('login_button')) Then driver.click(ById('login_button')) When driver.waitFor(ById('accept_all')) Then driver.click(ById('accept_all')) ```

I had an extended look at the source code in karate for AppiumDriver, AndroidDriver etc. and I noticed that the implementation just connects to the Appium server and only features are handful of possible API calls to Appium. Features like for example send-key are missing.

From my understanding, these cannot be accessed through driver.script("mobile: ..) because only mobile or webscript is supported,

see exerpt from $HOME\.appium\node_modules\appium-uiautomator2-driver\node_modules\appium-android-driver\lib\commands\execute.js ```js extensions.execute = async function execute (script, args) { if (script.match(/^mobile:/)) { this.log.info(`Executing native command '${script}'`); script = script.replace(/^mobile:/, '').trim(); return await this.executeMobile(script, _.isArray(args) ? args[0] : args); } if (!this.isWebContext()) { throw new errors.NotImplementedError(); } const endpoint = this.chromedriver.jwproxy.downstreamProtocol === PROTOCOLS.MJSONWP ? '/execute' : '/execute/sync'; return await this.chromedriver.jwproxy.command(endpoint, 'POST', { script, args, }); }; ```

I wonder if it would help to include the appium-java-client and provide a wrapper to make calls possible from Karate to the java client, instead of communicating with the Appium server directly as per current implementation? The java client is kept up to date and would also include seamlessly into Karate since all in java.

I think it would not only solve my arbitrary wait-on-launch problem, but also provide better appium support, more troubleshooting info and easy future feature capability.

Are there reasons why the java-client was not considered to be integrated in Karate? Or does someone know (@babusekaran maybe? I think you wrote the most code of mobile support) how I could solve the waiting problem, as well as access to extended features like sending a custom keystroke? Any help is greatly appreciated :) I am also more than happy to provide my support as well to contribute to Karate.

ptrthomas commented 2 years ago

@mathis-1337 yes @babusekaran wrote most of the mobile support so I'll request his comments as well.

I certainly would like to see mobile support improved in karate and if you are willing to contribute that will be awesome. I don't know much about the appium-java-client, but on the surface it certainly sounds like a technical approach that I would prefer instead of appium. one question I have is that would it support ios, android and things like flutter also eventually

my recommendation is to create a new karate-appium project and structure it similar to how karate-robot is. because I am not in favor of adding the appium java client dependency to karate-core. another decision we need to make is if we implement the Driver interface

babusekaran commented 2 years ago

Hi @mathis-1337, First of all, thank you for explaining descriptively. Karate has evolved a lot since I added this as a feature and one of the key reasons I added is to reap the rich features on match and so on.

Why not appium java-client as a dependency in karate ❓

The way I see appium server is an implementation of protocols like W3C spec, and Mobile JSON Wire Protocol to drive mobile devices. so yes there are official clients but If I want to use that I would use it directly instead of adding it as a dependency into karate core.

I didn't want a tight coupling of dependency with appium client implementation which would implicitly bring all of its dependency into karate (eg: selenium) I feel @ptrthomas was also sharing the same thoughts at that time. If you see a much more niche way I am happy to discuss it over a PR.

Missing features 🤔

Yes, there is no direct implementation for all the endpoints supported in appium. IMO, I think it has most of the basics covered but, of course, we are open to adding more upon PRs or compelling reasons. Also, there is no such thing as missing features in karate when comes to invoking webdriver/appium. you can get the client itself with a session and call all appium or w3c supported endpoints as you wish. for more read here https://github.com/karatelabs/karate/blob/master/examples/ui-test/README.md#webdriver-tips. there is a lot of such hidden 💎s you will find when you deep dive into karate documentation.

Appium Driver does not wait properly❗

It could be due to many reasons. I couldn't infer much from your examples, can you share the logs here to see what is happening during the execution. you mentioned it is a cross-platform app could you elaborate on that as well. could It be that the app became unresponsive during loading?

A reproducible code would help triage this faster. Lastly, as @ptrthomas mentioned you are free and most welcome to contribute if you see it can be improved in any way.

ptrthomas commented 2 years ago

moving to project board: https://github.com/karatelabs/karate/projects/3#card-84721003

ptrthomas commented 2 years ago

there's this new mobile automation option "Maestro" that I hope someone can volunteer to look into: https://blog.mobile.dev/introducing-maestro-painless-mobile-ui-automation-bee4992d13c1