search

wix / Detox

Gray box end-to-end testing and automation framework for mobile apps
https://wix.github.io/Detox/
MIT License
11.04k stars 1.91k forks source link

detox test fails to install apk #3474

Closed RohovDmytro closed 1 year ago

RohovDmytro commented 2 years ago

What happened?

  1. I've setup Github Action to run detox tests.

    • sdk manager
    • platform tools
    • emulator
    • butler
    • adb
    • ...

  2. Running detox test fails to install an apk. The pm install command fails every time after 45 seconds. Local tests are working.

13:14:19.683 detox[4431] DEBUG: [SPAWN_CMD, #25, cpid=4643] /Users/runner/Library/Android/sdk/platform-tools/adb -s emulator-5554 shell pm install -r -g -t /data/local/tmp/detox/Application.apk
13:15:04.686 detox[4431] DEBUG: [SPAWN_END, #25, cpid=4643] /Users/runner/Library/Android/sdk/platform-tools/adb -s emulator-5554 shell pm install -r -g -t /data/local/tmp/detox/Application.apk terminated with SIGTERM
13:15:04.818 detox[4431] TRACE: [SPAWN_TRY_FAIL, #25] 
13:15:04.828 detox[4431] DEBUG: [SPAWN_CMD, #25, cpid=4677] (Retry #1) /Users/runner/Library/Android/sdk/platform-tools/adb -s emulator-5554 shell pm install -r -g -t /data/local/tmp/detox/Application.apk
13:15:49.975 detox[4431] DEBUG: [SPAWN_END, #25, cpid=4677] /Users/runner/Library/Android/sdk/platform-tools/adb -s emulator-5554 shell pm install -r -g -t /data/local/tmp/detox/Application.apk terminated with SIGTERM
13:15:50.213 detox[4431] TRACE: [SPAWN_TRY_FAIL, #25] 
13:15:50.225 detox[4431] DEBUG: [SPAWN_CMD, #25, cpid=4[774](https://github.com/houserion/universage/runs/7212318253?check_suite_focus=true#step:21:778)] (Retry #2) /Users/runner/Library/Android/sdk/platform-tools/adb -s emulator-5554 shell pm install -r -g -t /data/local/tmp/detox/Application.apk
13:16:35.268 detox[4431] DEBUG: [SPAWN_END, #25, cpid=4774] /Users/runner/Library/Android/sdk/platform-tools/adb -s emulator-5554 shell pm install -r -g -t /data/local/tmp/detox/Application.apk terminated with SIGTERM
13:16:35.570 detox[4431] TRACE: [SPAWN_TRY_FAIL, #25] 
13:16:35.578 detox[4431] DEBUG: [SPAWN_CMD, #25, cpid=4904] (Retry #3) /Users/runner/Library/Android/sdk/platform-tools/adb -s emulator-5554 shell pm install -r -g -t /data/local/tmp/detox/Application.apk
13:17:20.699 detox[4431] DEBUG: [SPAWN_END, #25, cpid=4904] /Users/runner/Library/Android/sdk/platform-tools/adb -s emulator-5554 shell pm install -r -g -t /data/local/tmp/detox/Application.apk terminated with SIGTERM
13:17:20.706 detox[4431] ERROR: ChildProcessError: `/Users/runner/Library/Android/sdk/platform-tools/adb -s emulator-5554 shell pm install -r -g -t /data/local/tmp/detox/Application.apk` failed with code null
    at ChildProcess.<anonymous> (/Users/runner/work/universage/universage/node_modules/child-process-promise/lib/index.js:132:23)
    at ChildProcess.emit (node:events:390:28)
    at maybeClose (node:internal/child_process:1064:16)
    at Socket.<anonymous> (node:internal/child_process:450:11)
    at Socket.emit (node:events:390:28)
    at Pipe.<anonymous> (node:net:687:12)

What was the expected behaviour?

I expect detox test to install Application.apk and launch tests.

Was it tested on latest Detox?

Did your test throw out a timeout?

Help us reproduce this issue!

No response

In what environment did this happen?

Detox version: 19.7.1 React Native version: 0.64.3 Node version: 16.13.1 Device model: Pixel XL Android version: 12 Test-runner (select one): jest-circus

Detox logs

Detox logs ``` /bin/sh -c npx detox test --configuration houserion.android.e2e --loglevel trace --record-logs failing --record-videos failing INFO | boot completed INFO | boot time 25248 ms INFO | Increasing screen off timeout, logcat buffer size to 2M. INFO | Revoking microphone permissions for Google App. 13:13:01.583 detox[4428] INFO: [test.js] DETOX_CONFIGURATION="houserion.android.e2e" DETOX_LOGLEVEL="trace" DETOX_RECORD_LOGS="failing" DETOX_RECORD_VIDEOS="failing" DETOX_REPORT_SPECS=true DETOX_START_TIMESTAMP=1657102381569 DETOX_USE_CUSTOM_LOGGER=true jest --config e2e/config.json --testNamePattern '^((?!:ios:).)*$' e2e 13:13:05.524 detox[4431] TRACE: [DETOX_CREATE] created a Detox instance with config: { appsConfig: { default: { type: 'android.apk', binaryPath: 'android/app/build/outputs/apk/houserion/e2e/app-houserion-e2e.apk', build: 'cd ./android && RN_SRC_EXT=e2e.js,e2e.ts,e2e.tsx APP=houserion BUILD_TYPE=e2e GENERATE_SOURCEMAP=false ./gradlew assembleHouserionE2e assembleHouserionE2eAndroidTest -DtestBuildType=e2e && cd ..' } }, artifactsConfig: { rootDir: '.artifacts/houserion.android.e2e.2022-07-06 10-13-01Z', plugins: { log: { enabled: true, keepOnlyFailedTestsArtifacts: true }, screenshot: { enabled: true, ● Deep Link, Guest › should work Environment setup failed. See the detailed error below. ● Deep Link, Guest › should work Environment setup failed. See the detailed error below. ● Deep Link, Authorized › should onboard Environment setup failed. See the detailed error below. ● Deep Link, Authorized › should work Environment setup failed. See the detailed error below. ● Deep Link, Broken › should work Environment setup failed. See the detailed error below. ● Test suite failed to run ChildProcessError: `/Users/runner/Library/Android/sdk/platform-tools/adb -s emulator-5554 shell pm install -r -g -t /data/local/tmp/detox/Application.apk` failed with code null at ChildProcess. (../node_modules/child-process-promise/lib/index.js:[132](https://github.com/houserion/universage/runs/7212318253?check_suite_focus=true#step:21:136):23) 13:17:49.456 detox[4431] TRACE: [SESSION_TORN] tester exited session 695db362-0f12-2db8-f69f-fe39da698771 13:17:49.961 detox[4428] ERROR: [cli.js] Command failed: jest --config e2e/config.json --testNamePattern '^((?!:ios:).)*$' e2e ```

Device logs

Device logs ``` paste logs here! ```

More data, please!

No response

jonathanmos commented 1 year ago

@RohovDmytro do you see anything relevant in adb logcat that could help debug this? Does this still occur after restarting your emulator and restarting adb (adb kill-server, adb start-server)?

viljamik commented 1 year ago

Try to install the .apk manually with the same command to see if it works. I had the issue of not having enough disk space on my android emulator.

ErickMaeda commented 1 year ago

I have the exactly same issue, but using Bitrise CI.

I can find the device via adb -s emulator-5554 emu avd name

But when running the 

detox test -c android.emu.release

I have the issue:

detox[8453] INFO:  [test.js] DETOX_CLEANUP=true DETOX_CONFIGURATION="android.emu.release" DETOX_REPORT_SPECS=true DETOX_START_TIMESTAMP=1657707517233 DETOX_USE_CUSTOM_LOGGER=true jest --config e2e/config.json --testNamePattern '^((?!:ios:).)*$' e2e

detox[8465] ERROR: ChildProcessError: `/opt/android-sdk-linux/platform-tools/adb -s emulator-5554 shell pm install -r -g -t /data/local/tmp/detox/Application.apk` failed with code null
    at ChildProcess.<anonymous> (/bitrise/src/app/node_modules/child-process-promise/lib/index.js:132:23)
    at ChildProcess.emit (node:events:527:28)
    at maybeClose (node:internal/child_process:1092:16)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:302:5)
Arron-Gill commented 1 year ago

I have the exact same issue as @ErickMaeda - spent days trying to find a workaround but just can't get Bitrise and Detox to play ball.

jonathanmos commented 1 year ago

@ErickMaeda @Arron-Gill Can you try to install the apk manually from the commandline and check whether that works for you? Also, if you run the tests locally, and not through bitrise, do they work as expected? Do you see any relevant messages in the adb logcat when the problem occurs?

ggilchrist-ledger commented 1 year ago

Exact same issue for Github Actions here. Is everyone trying to install on an x86_64 emulator?

jonathanmos commented 1 year ago

Try to use the forceAdbInstall flag in the device config and see if it solves the issue for you -> https://github.com/wix/Detox/blob/246317d3ee5b7587bb966c78204b5def5b72dfaf/docs/APIRef.Configuration.md

evseevnn commented 1 year ago

@jonathanmos I have same issue and flag didn't help me

d4vidi commented 1 year ago

Everyone facing this issue - please follow the suggestion of bundling the installation of the apk (or any other APK) in your buildscripts, prior to Detox, and see whether that works.

@RohovDmytro since in your case the command terminates with a SIGTERM, I postulate that github-actions doesn't approve of something happening in your environment, and therefore terminates the command prematurely. Since you mentioned 45 seconds, I would guess an enforcement of a timeout of some sort.

ErickMaeda commented 1 year ago

I tried with the flag forceAdbInstall but it didn't result,

I also I tried installing manually both apk's generated from build

The installation was successfull as you'll see on logs, however it fails when trying to run the detox test -c android.emu.release

Performing Streamed Install
Success
Performing Streamed Install
Success
00:10:29.053 detox[8892] INFO:  [test.js] DETOX_CONFIGURATION="android.emu.release" DETOX_REPORT_SPECS=true DETOX_START_TIMESTAMP=1660176629017 DETOX_USE_CUSTOM_LOGGER=true jest --config e2e/config.json --testNamePattern '^((?!:ios:).)*$' e2e
00:13:43.968 detox[8904] ERROR: ChildProcessError: `/opt/android-sdk-linux/platform-tools/adb -s emulator-5554 shell pm install -r -g -t /data/local/tmp/detox/Application.apk` failed with code null
    at ChildProcess.<anonymous> (/bitrise/src/app/node_modules/child-process-promise/lib/index.js:132:23)
    at ChildProcess.emit (node:events:527:28)
    at maybeClose (node:internal/child_process:1092:16)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:302:5)
d4vidi commented 1 year ago

@ErickMaeda as a workaround, try keeping the pre-install sequence and run Detox with the --reuse argument.

professorkolik commented 1 year ago

@d4vidi Could you please give more info regarding this

try keeping the pre-install sequence

I have exact same issue on Bitrise

d4vidi commented 1 year ago

@professorkolik largely suggesting to launch the emulators in advance, build your apps (e.g. using detox build) and run adb install manually before running detox test (which, in turn, should be run with the --reuse argument).

professorkolik commented 1 year ago

@d4vidi thank you for suggestion, we do that everything except running adb install manually We'll try it out and update you Screenshot 2022-08-30 at 15 36 49

professorkolik commented 1 year ago

@d4vidi So after manual install of apk, and adding --reuse to detox runner, test is able to connect to emulator, but following error happens

No instrumentation runner found on device emulator-5554 for package

I assume it's another issue, I would be thankful if you can point out the direction

d4vidi commented 1 year ago

This is because you need to build and adb-install the test APK, in the same way as you do with the main (app) APK.

david-alza commented 1 year ago

Everyone facing this issue - please follow the suggestion of bundling the installation of the apk (or any other APK) in your buildscripts, prior to Detox, and see whether that works.

@RohovDmytro since in your case the command terminates with a SIGTERM, I postulate that github-actions doesn't approve of something happening in your environment, and therefore terminates the command prematurely. Since you mentioned 45 seconds, I would guess an enforcement of a timeout of some sort.

Hey @d4vidi, just wanted to comment specifically on this, since I think this is intended? The install timeout is specified as 45 seconds here: https://github.com/wix/Detox/blob/1144d13d7eafcb8527f4b9a08fcf9eb79f9344ef/detox/src/devices/common/drivers/android/exec/ADB.js#L11.

I wonder if that should be bumped up a little bit? At the very least it'd be nice to have a specific error message logged when an install is getting SIGTERM'd due to the timeout.

We're hitting this on GH actions as well where these operations seem to really drag, though seemingly not as much with just a simple adb install run manually before detox actually runs.

I'll try experimenting with a higher install timeout and report back if it helps.

david-alza commented 1 year ago

With a few tests it seems like bumping this timeout up to 90s did resolve our issues here, though it's hard to say with certainty since these were always a bit flakey overall... not sure what's the root cause of the slow install times.

Logging the timeouts doesn't seem possible actually since the timeout argument is just being passed down into node's child_process.exec command which will just SIGTERM a command on timeout without a specific message [source]

ggilchrist-ledger commented 1 year ago

I'm going to try upping the install timeout on my side too. Is it worth having this as an env variable for the detox test command so we can pass in if we want to override the value for those of us that experience the issue? (slow build servers, etc)

ggilchrist-ledger commented 1 year ago

This is because you need to build and adb-install the test APK, in the same way as you do with the main (app) APK.

@d4vidi Is this in addition to the main app APK or we just install the test APK onto the emulator?

Edit: Did some research into the test APK and saw it's just for instrumentation, and not the actual app

david-alza commented 1 year ago

FWIW on our end the other biggest thing that helped with Detox + Bitrise was to swap to their Linux machines. Runs got way more stable and we haven't seen install timeouts since then. For other performance issues, we also had some luck giving more memory to the emulator and tweaking the heap size of the JVM.

ggilchrist-ledger commented 1 year ago

Diving into our Github Actions logs a bit more, it looks very much like the install is failing due to the 45 second timeout:

14:00:46.615 detox[14630] DEBUG: [SPAWN_CMD, #19, cpid=17258] /Users/runner/Library/Android/sdk/platform-tools/adb -s emulator-5554 shell pm install -r -g -t /data/local/tmp/detox/Application.apk
14:01:31.638 detox[14630] DEBUG: [SPAWN_END, #19, cpid=17258] /Users/runner/Library/Android/sdk/platform-tools/adb -s emulator-5554 shell pm install -r -g -t /data/local/tmp/detox/Application.apk terminated with SIGTERM
karlhorky commented 1 year ago

@d4vidi would Detox consider changing the INSTALL_TIMEOUT value that @david-alza found, given that it appears to be affecting others?

largely suggesting to launch the emulators in advance, build your apps (e.g. using detox build) and run adb install manually before running detox test (which, in turn, should be run with the --reuse argument)

If the timeout will not be changed and these installation steps should be indeed reused maybe it would be good to get a guide on which commands with which flags are implied here for CI users...

karlhorky commented 1 year ago

largely suggesting to launch the emulators in advance, build your apps (e.g. using detox build) and run adb install manually before running detox test (which, in turn, should be run with the --reuse argument)

If the timeout will not be changed and these installation steps should be indeed reused maybe it would be good to get a guide on which commands with which flags are implied here for CI users...

@d4vidi just looking for an example to document this now, would this example workflow by @mikehardy contain any of what you're talking about?

https://github.com/ankidroid/Anki-Android/blob/2c29fa9b13a442a7f94f16b6bb48d8957d8c508a/.github/workflows/tests_emulator.yml

rodperottoni commented 1 year ago

Why is this closed? :(

bataevvlad commented 11 months ago

@rodperottoni

For anyone who is facing this problem, here is the solution that worked for me.

A little description of my problem:

I run tests every time a new PR is created. It took me a long time to figure out what the cause was, but eventually I realized that the problem is in the signature of my APK files.

I'm building release versions that require a signature.

I deployed via Github Actions, and this is what I did:

Be notice that i have not basic flavors! It is release and staging flavors, change it accordingly to yours.

      - name: Encode the keystore to base64
        id: encode_keystore
        run: |
          echo "SIGNINGKEYBASE64=$(openssl base64 < $RUNNER_TEMP/my_production.keystore | tr -d '\n')" >> $GITHUB_ENV

      - name: Sign APK
        id: sign_apk
        uses: r0adkll/sign-android-release@v1
        with:
          releaseDirectory: android/app/build/outputs/apk/production/release
          signingKeyBase64: ${{ env.SIGNINGKEYBASE64 }}
          alias: ${{ secrets.KEY_ALIAS }}
          keyStorePassword: ${{ secrets.KEY_STORE_PASSWORD }}
          keyPassword: ${{ secrets.KEY_PASSWORD }}

      - name: Sign AndroidTest APK for Detox
        id: sign_androidTest_apk
        uses: r0adkll/sign-android-release@v1
        with:
          releaseDirectory: android/app/build/outputs/apk/androidTest/production/release/
          signingKeyBase64: ${{ env.SIGNINGKEYBASE64 }}
          alias: ${{ secrets.KEY_ALIAS }}
          keyStorePassword: ${{ secrets.KEY_STORE_PASSWORD }}
          keyPassword: ${{ secrets.KEY_PASSWORD }}

Main issue was that we also need to sign-up androidTest APK.

And im also provide more careful way to my apks, because detox do wrong thing with my folders. 

    'production.android.release': {
      type: 'android.apk',
      binaryPath:
          'android/app/build/outputs/apk/production/release/app-production-release-signed.apk',
      testBinaryPath:
          'android/app/build/outputs/apk/androidTest/production/release/app-production-release-androidTest-signed.apk',
      build: 'cd android && ./gradlew assembleProductionRelease assembleProductionReleaseAndroidTest -DtestBuildType=release && cd ..'
    }

For careful deployment im suggest to install brew install tree and check the paths on your virtual machine.

If you have any questions, I'm happy to help :)

estiventhneira commented 2 weeks ago

yeah, please stop wasting time on this

you just need to sign the app

https://reactnative.dev/docs/signed-apk-android