bitrise-steplib / steps-virtual-device-testing-for-android

MIT License
22 stars 23 forks source link

Need better error logging #24

Closed tir38 closed 3 years ago

tir38 commented 6 years ago

I'm trying to do something specific but I'm not getting very good error logs to help debug the problem. So two parts: What is causing the initial error? How can I get better error logs in the future to self-debug these?

In my hack to get multiple Gradle modules tested on Firebase I was testing two "library" modules. I switched and now I'm testing a library module first, then an "application" module. The tests run fine for the first module, but the second module produces 403 error

| id: virtual-device-testing-for-android                                       |
| version: 1.0.2                                                               |
| collection: https://github.com/bitrise-io/bitrise-steplib.git                |
| toolkit: go                                                                  |
| time: 2018-07-27T21:39:43Z                                                   |
+------------------------------------------------------------------------------+
|                                                                              |
INFO[21:39:43] Start installing (golang) with apt-get       
INFO[21:39:43]  * [OK] Step dependency (go) installed, available. 
Configs:
- ApkPath: /bitrise/deploy/app-debug.apk
- TestTimeout: 900
- DirectoriesToPull: 
- EnvironmentVariables: 
- TestDevices:
---
Model     API Level   Locale     Orientation   
Nexus5X   25          portrait   en            
---
- AppPackageID: 
- TestType: instrumentation
- TestApkPath: /bitrise/deploy/app-debug-androidTest.apk
- InstTestPackageID: 
- InstTestRunnerClass: 
- InstTestTargets: 
- UseOrchestrator: true
Upload APKs
Failed to start test: 403, error: {"error":"Build already exists"}

I'm not sure if this error is coming from Bitrise or if it's just bubbled up from Firebase. Here is the pertinent part of my bitrise.yml file:

temp:
    steps:
    - activate-ssh-key:
        run_if: '{{getenv "SSH_RSA_PRIVATE_KEY" | ne ""}}'
    - git-clone: {}
    - cache-pull: {}
    - install-missing-android-tools: {}
    - gradle-runner:
        inputs:
        - gradle_file: "$GRADLE_BUILD_FILE_PATH"
        - gradle_task: assembleDebug
        - gradlew_path: "$GRADLEW_PATH"
        title: Build debug apk
    - gradle-runner:
        inputs:
        - mapping_file_include_filter: ''
        - apk_file_include_filter: ''
        - gradle_task: api:assembleDebugAndroidTest
        title: Build api module instrumentation test apk
    - virtual-device-testing-for-android:
        inputs:
        - test_devices: "$INSTRUMENTATION_DEVICE_LIST"
        - inst_use_orchestrator: 'true'
        - test_type: instrumentation
        title: Run api module instrumentation test on virtual devices
    - script:
        inputs:
        - content: rm $BITRISE_TEST_APK_PATH
        title: Delete api test apk so it doesn't get uploaded to bitrise.io
    - gradle-runner:
        inputs:
        - mapping_file_include_filter: ''
        - apk_file_include_filter: ''
        - apk_file_exclude_filter: "*.apk"
        - gradle_task: app:assembleDebugAndroidTest
        title: Build app module instrumentation test apk
        description: Exclude app apk since it was built in a previous step
    - virtual-device-testing-for-android:
        inputs:
        - test_devices: "$INSTRUMENTATION_DEVICE_LIST"
        - inst_use_orchestrator: 'true'
        - test_type: instrumentation
        title: Run app module instrumentation test on virtual devices THIS IS FAILING
    - script:
        inputs:
        - content: rm $BITRISE_TEST_APK_PATH
        title: Delete app test apk so it doesn't get uploaded to bitrise.io
    - deploy-to-bitrise-io:
        inputs:
        - is_enable_public_page: 'false'
        - notify_user_groups: none
    - cache-push: {}
BirmacherAkos commented 6 years ago

Hi @tir38 !

I think the problem is that you want to run two "virtual-device-testing-for-android" step with the same Build Slug.

I would split this workflow into 3 different workflows (You will need to have multiple concurrencies to do this.).


1) primary: The parent workflow. This will start the second and the third workflow.

2) virtual_test_first: In the second workflow you can run the first "Virtual Device Testing For Android" step.

3) virtual_test_second: In the third workflow run the second "Virtual Device Testing For Android" step.

workflows:
  primary:
    steps:
    - build-router-start@0.9.0:
        inputs:
        - access_token: "$API_TOKEN"
        - wait_for_builds: 'true'
        - workflows: |-
            virtual_test_first
            virtual_test_second

  virtual_test_first:
    steps:
    - activate-ssh-key:
        run_if: '{{getenv "SSH_RSA_PRIVATE_KEY" | ne ""}}'
    - git-clone: {}
    - cache-pull: {}
    - install-missing-android-tools: {}
    - gradle-runner:
        inputs:
        - gradle_file: "$GRADLE_BUILD_FILE_PATH"
        - gradle_task: assembleDebug
        - gradlew_path: "$GRADLEW_PATH"
        title: Build debug apk
    - gradle-runner:
        inputs:
        - mapping_file_include_filter: ''
        - apk_file_include_filter: ''
        - gradle_task: api:assembleDebugAndroidTest
        title: Build api module instrumentation test apk
    - virtual-device-testing-for-android:
        inputs:
        - test_devices: "$INSTRUMENTATION_DEVICE_LIST"
        - inst_use_orchestrator: 'true'
        - test_type: instrumentation
        title: Run api module instrumentation test on virtual devicesFAILING
    - script:
        inputs:
        - content: rm $BITRISE_TEST_APK_PATH
        title: Delete app test apk so it doesn't get uploaded to bitrise.io
    - deploy-to-bitrise-io:
        inputs:
        - is_enable_public_page: 'false'
        - notify_user_groups: none
    - cache-push: {}

  virtual_test_second:
    steps:
    - activate-ssh-key:
        run_if: '{{getenv "SSH_RSA_PRIVATE_KEY" | ne ""}}'
    - git-clone: {}
    - cache-pull: {}
    - install-missing-android-tools: {}
    - gradle-runner:
        inputs:
        - gradle_file: "$GRADLE_BUILD_FILE_PATH"
        - gradle_task: assembleDebug
        - gradlew_path: "$GRADLEW_PATH"
        title: Build debug apk
    - gradle-runner:
        inputs:
        - mapping_file_include_filter: ''
        - apk_file_include_filter: ''
        - apk_file_exclude_filter: "*.apk"
        - gradle_task: app:assembleDebugAndroidTest
        title: Build app module instrumentation test apk
        description: Exclude app apk since it was built in a previous step
    - virtual-device-testing-for-android:
        inputs:
        - test_devices: "$INSTRUMENTATION_DEVICE_LIST"
        - inst_use_orchestrator: 'true'
        - test_type: instrumentation
        title: Run app module instrumentation test on virtual devices THIS IS FAILING
    - script:
        inputs:
        - content: rm $BITRISE_TEST_APK_PATH
        title: Delete app test apk so it doesn't get uploaded to bitrise.io
    - deploy-to-bitrise-io:
        inputs:
        - is_enable_public_page: 'false'
        - notify_user_groups: none
    - cache-push: {}

Other solution: Create two apps on Bitrise. https://discuss.bitrise.io/t/android-virtual-device-testing-on-multiple-modules/4241/2

tir38 commented 6 years ago

yep experimenting with this now. Thanks!

tir38 commented 6 years ago

Update: After some time, the parallel workflows has been working great (aside from build deadlocks). I'll close this issue out.

tir38 commented 5 years ago

Actually we started seeing this, even on parallelized workflows like described above. I'm happy to shared build logs or URL

bitce commented 5 years ago

Hi @tir38! Sorry to hear this, please do share the logs so we can have the team look into it!