search

microsoft / azure-pipelines-tasks

Tasks for Azure Pipelines
https://aka.ms/tfbuild
MIT License
3.47k stars 2.61k forks source link

Unable to run Android UI automation test #644

Closed kfstorm closed 8 years ago

kfstorm commented 8 years ago

I'm trying to setup UI automation test in Android with AndroidJUnit4 framework. But we meet this error if uncheck "Headless Display":

SDL init failure, reason is: No displays available

If we check "Headless Display", the tests will with below error:

android.view.InflateException: Binary XML file line #45: Error inflating class fragment

We are using Windows Service to host the agent and below this the config of build definition.

{
  "build": [
    {
      "enabled": true,
      "continueOnError": false,
      "alwaysRun": false,
      "displayName": "Run cmd",
      "task": {
        "id": "d9bafed4-0b18-4f58-968d-86655b4d2ce9",
        "versionSpec": "*"
      },
      "inputs": {
        "filename": "cmd",
        "arguments": "/c \"echo y | %ANDROID_HOME%\\tools\\android update sdk --all --no-ui --filter sys-img-armeabi-v7a-android-21\"",
        "workingFolder": "",
        "failOnStandardError": "false"
      }
    },
    {
      "enabled": true,
      "continueOnError": false,
      "alwaysRun": false,
      "displayName": "Android Build ",
      "task": {
        "id": "df857559-8715-46eb-a74e-ac98b9178aa0",
        "versionSpec": "*"
      },
      "inputs": {
        "gradleWrapper": "gradlew.bat",
        "gradleProj": "",
        "gradleArguments": "build uninstallAll connectedAndroidTest --gradle-user-home=$(Agent.BuildDirectory)",
        "avdName": "AndroidBuildEmulator",
        "createAvd": "true",
        "emulatorTarget": "android-21",
        "emulatorDevice": "Nexus 5",
        "avdAbi": "default/armeabi-v7a",
        "avdForce": "true",
        "avdOptionalArgs": "",
        "startEmulator": "true",
        "emulatorTimeout": "3600",
        "emulatorHeadless": "true",
        "emulatorOptionalArgs": "-no-snapshot-load -no-snapshot-save",
        "deleteAvd": "true"
      }
    },
    {
      "enabled": true,
      "continueOnError": false,
      "alwaysRun": false,
      "displayName": "Publish Artifact: outputs",
      "task": {
        "id": "1d341bb0-2106-458c-8422-d00bcea6512a",
        "versionSpec": "*"
      },
      "inputs": {
        "CopyRoot": "app/build/outputs",
        "Contents": "*",
        "ArtifactName": "outputs",
        "ArtifactType": "Container",
        "TargetPath": ""
      }
    }
  ],
  ...
}
ericsciple commented 8 years ago

You may need to run as interactive agent for UI tests to work. I'll ping some folks who would know.

yacaovsnc commented 8 years ago

@kfstorm is this error reported by the compiler or the emulator? Would anything from these links help? http://stackoverflow.com/questions/6424853/error-inflating-class-fragment http://stackoverflow.com/questions/13787517/inflateexception-error-inflating-class-fragment

Can you also please post the full stacktrace if that's available? Thanks.

kfstorm commented 8 years ago

@ericsciple But I can't since we use Windows Service to host the agent. Of cause I can start the agent every time I log in to OS, but we need full automation.

@yacaovsnc the SDL init failure, reason is: No displays available error is reported by the emulator. The emulator output this error message and then close itself instantly. As for the android.view.InflateException: Binary XML file line #45: Error inflating class fragment error, it's test failure when running UI tests, so it's a runtime error. Sorry I can't repro it right now. But I see other errors. Seems we can't interact with UI via Espresso. Can you look at this? (Please ignore the Failed to enable "Don't keep activities"! failures.)

All test cases can pass in local environment with emulator or real device.

Build Log:

:app:connectedProductionDebugAndroidTest
com.microsoft.bingads.app.views.AccountSummaryStateRecoveryTest > basicValidation[AndroidBuildEmulator(AVD) - 5.0.2] FAILED 
java.lang.AssertionError: Failed to enable "Don't keep activities"!
at org.junit.Assert.fail(Assert.java:88)
com.microsoft.bingads.app.views.AccountSummaryTest > cancelMajorKpi[AndroidBuildEmulator(AVD) - 5.0.2] FAILED 
android.support.test.espresso.PerformException: Error performing 'single click' on view 'has child: has child: (with id: com.microsoft.bingads:id/view_performance_item_selected and with background: )'.
at android.support.test.espresso.PerformException$Builder.build(PerformException.java:83)
com.microsoft.bingads.app.views.AccountSummaryTest > cancelMinorKpi[AndroidBuildEmulator(AVD) - 5.0.2] FAILED 
android.support.test.espresso.PerformException: Error performing 'single click' on view 'has child: has child: (with id: com.microsoft.bingads:id/view_performance_item_selected and with background: )'.
at android.support.test.espresso.PerformException$Builder.build(PerformException.java:83)
com.microsoft.bingads.app.views.BasicWorkFlowTest > basicWorkFlow[AndroidBuildEmulator(AVD) - 5.0.2] FAILED 
android.support.test.espresso.PerformException: Error performing 'type text(SYS-AC-AdInsightTeam)' on view 'with id: com.microsoft.bingads:id/fragment_login_username'.
at android.support.test.espresso.PerformException$Builder.build(PerformException.java:83)
com.microsoft.bingads.app.views.DateRangePickerStateRecoveryTest > validate[AndroidBuildEmulator(AVD) - 5.0.2] FAILED 
java.lang.AssertionError: Failed to enable "Don't keep activities"!
at org.junit.Assert.fail(Assert.java:88)
com.microsoft.bingads.app.views.FeatureGuideActivityStateRecoveryTest > testFeatureGuideStateRecovery[AndroidBuildEmulator(AVD) - 5.0.2] FAILED 
android.support.test.espresso.base.DefaultFailureHandler$AssertionFailedWithCauseError: 'is displayed on the screen to the user' doesn't match the selected view.
Expected: is displayed on the screen to the user
com.microsoft.bingads.app.views.LoginTest > testLoginWithMicrosoftAccount[AndroidBuildEmulator(AVD) - 5.0.2] FAILED 
android.support.test.espresso.PerformException: Error performing 'single click' on view 'with id: com.microsoft.bingads:id/fragment_login_loginByMsAccount'.
at android.support.test.espresso.PerformException$Builder.build(PerformException.java:83)
com.microsoft.bingads.app.views.LoginTest > testLoginWithUsernamePassword[AndroidBuildEmulator(AVD) - 5.0.2] FAILED 
android.support.test.espresso.PerformException: Error performing 'type text(SYS-AC-AdInsightTeam)' on view 'with id: com.microsoft.bingads:id/fragment_login_username'.
at android.support.test.espresso.PerformException$Builder.build(PerformException.java:83)
:app:connectedProductionDebugAndroidTest FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':app:connectedProductionDebugAndroidTest'.
> There were failing tests. See the report at: file:///C:/VSOAgent/_work/4/s/app/build/reports/androidTests/connected/flavors/PRODUCTION/index.html
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
BUILD FAILED
Total time: 12 mins 9.662 secs

Stacktrace of AccountSummaryTest.cancelMajorKpi:

android.support.test.espresso.PerformException: Error performing 'single click' on view 'has child: has child: (with id: com.microsoft.bingads:id/view_performance_item_selected and with background: )'.
at android.support.test.espresso.PerformException$Builder.build(PerformException.java:83)
at android.support.test.espresso.base.DefaultFailureHandler.getUserFriendlyError(DefaultFailureHandler.java:70)
at android.support.test.espresso.base.DefaultFailureHandler.handle(DefaultFailureHandler.java:53)
at android.support.test.espresso.ViewInteraction.runSynchronouslyOnUiThread(ViewInteraction.java:184)
at android.support.test.espresso.ViewInteraction.doPerform(ViewInteraction.java:115)
at android.support.test.espresso.ViewInteraction.perform(ViewInteraction.java:87)
at com.microsoft.bingads.app.views.AccountSummaryTest.cancelMajorKpi(AccountSummaryTest.java:58)
at java.lang.reflect.Method.invoke(Native Method)
at java.lang.reflect.Method.invoke(Method.java:372)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at android.support.test.internal.statement.UiThreadStatement.evaluate(UiThreadStatement.java:55)
at android.support.test.rule.ActivityTestRule$ActivityStatement.evaluate(ActivityTestRule.java:257)
at com.microsoft.bingads.app.helpers.CustomActivityTestRule$1.evaluate(CustomActivityTestRule.java:44)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:27)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
at android.support.test.internal.runner.TestExecutor.execute(TestExecutor.java:54)
at android.support.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:228)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1837)
Caused by: java.lang.RuntimeException: Action will not be performed because the target view does not match one or more of the following constraints:
at least 90 percent of the view's area is displayed to the user.
Target view: "PerformanceItemView{id=-1, visibility=VISIBLE, width=64, height=257, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=true, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0, child-count=1}"
at android.support.test.espresso.ViewInteraction$1.run(ViewInteraction.java:138)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:422)
at java.util.concurrent.FutureTask.run(FutureTask.java:237)
at android.os.Handler.handleCallback(Handler.java:739)
at android.os.Handler.dispatchMessage(Handler.java:95)
at android.os.Looper.loop(Looper.java:135)
at android.app.ActivityThread.main(ActivityThread.java:5221)
at java.lang.reflect.Method.invoke(Native Method)
at java.lang.reflect.Method.invoke(Method.java:372)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:899)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:694)
kfstorm commented 8 years ago

Hi guys,

I found the root cause. It's because my config created an AVD with 480 density but only 480x800 resolution. So some views are out of the screen. That's why all the click/type actions fail. To fix this, I changed the device from Nexus 5 (with 480 dpi) to Nexus S (with 240 dpi), and removed -no-skin argument when starting the AVD by uncheck Headless Display and add -no-audio -no-window to optional arguments.

Thanks for your help.