Instrumentation failing for android

[othiagosouto]

Hi,

I was trying to setup my instrumentation and following the steps provided here, however, seems that it's not working as expected.

This is the log:

+------------------------------------------------------------------------------+
| (8) virtual-device-testing-for-android@1                                     |
+------------------------------------------------------------------------------+
| id: virtual-device-testing-for-android                                       |
| version: 1.1.2                                                               |
| collection: https://github.com/bitrise-io/bitrise-steplib.git                |
| toolkit: go                                                                  |
| time: 2020-05-18T01:37:31Z                                                   |
+------------------------------------------------------------------------------+
|                                                                              |

Configs:
- AppPath: /bitrise/deploy/app-debug.apk
- TestTimeout: 900.000000
- FlakyTestAttempts: 0
- DownloadTestResults: false
- DirectoriesToPull: 
- AutoGoogleLogin: false
- EnvironmentVariables: 
- ObbFilesList: 
- TestDevices:
---
Model    API Level   Locale   Orientation   
Pixel2   28          en       portrait      
---
- TestType: instrumentation
- TestApkPath: /bitrise/deploy/app-debug-androidTest.apk
- InstTestPackageID: 
- InstTestRunnerClass: androidx.test.runner.AndroidJUnitRunner
- InstTestTargets: 
- UseOrchestrator: false

Uploading app and test files
=> Files uploaded

Starting test
=> Test started

Waiting for test results
- Validating
- (1/1) running
- (0/1) running
=> Test finished

Test results:
Model    API Level   Locale   Orientation   Outcome              
Pixel2   28          en       portrait      failure   
|                                                                              |
+---+---------------------------------------------------------------+----------+
| x | virtual-device-testing-for-android@1 (exit code: 1)           | 3.4 min  |
+---+---------------------------------------------------------------+----------+
| Issue tracker: ...se-steplib/steps-virtual-device-testing-for-android/issues |
| Source: ...thub.com/bitrise-steplib/steps-virtual-device-testing-for-android |
+---+---------------------------------------------------------------+----------+

Workflow:

Someone have any idea how to solve this or get more info about what it's causing this issue?

[uberbinge]

I am also running into a similar issue, any update for this one?

[Roland-Bak]

Hi there!

The cause of why Instrumentation Test issues fail can vary greatly, and it is possible that you have very different reasons for failing. To keep it simple and to be able to focus in each of your issues individually, could you please submit a Support Ticket at our site? Please include the URL of a related build and enable Support Access (https://devcenter.bitrise.io/troubleshooting/enabling-bitrise-support-user/) so that we may investigate in detail? 🙂

Thanks!

[cchierici]

I also regularly run into the same issue, and yes, i know that instrumentation tests tend to be flaky, but i wonder if it's possible to get more log information to see where the test exactly has failed

[cathyharmon]

Hello,

The step has a couple of inputs you might want to check into. If you click on "DEBUG", you will see various options. You can set test timeouts and retries to help determine if it is a timing issue or if it was a temporary issue that resolves with a retry.

If there is specific output from a test, you can add the directory where the output is stored to have that information downloaded.

Finally, there is a Verbose log that can be set to true to enable verbose level logging.

If you include the Deploy to bitrise.io step, you can also check the Test Reports using the Test reports Add-on. This can often provide additional information, logs, etc from the test and can help in determining the reason for the failure.

Thanks, cathy

[Madonahs]

@othiagosouto Not sure if you solved this, I wanted to share what worked for me. I checked the Test reports as @cathyharmon mentioned and saw the error (tests that actually failed), and had to handle the idlingResources and that fixed the error. Hope this helps. It now works well.

[bitrise-coresteps-bot]

Hello there, I'm a bot. On behalf of the community I thank you for opening this issue.

To help our human contributors focus on the most relevant reports, I check up on old issues to see if they're still relevant. This issue has had no activity for 90 days, so I marked it as stale.

The community would appreciate if you could check if the issue still persists. If it isn't, please close it. If the issue persists, and you'd like to remove the stale label, you simply need to leave a comment. Your comment can be as simple as "still important to me".

If no comment left within 21 days, this issue will be closed.

[bitrise-coresteps-bot]

I'll close this issue as it doesn't seem to be relevant anymore. We believe an old issue probably has a bunch of context that's no longer relevant, therefore, if the problem still persists, please open a new issue.