Runner action hangs after killing emulator with stop: not implemented

[ericswpark]

When running my GitHub Actions workflow, the emulator runner action hangs after the emulator is killed, with the following log output:

Terminate Emulator
  /usr/local/lib/android/sdk/platform-tools/adb -s emulator-5554 emu kill
  OK: killing emulator, bye bye
  OK
  INFO    | Wait for emulator (pid 4126) 20 seconds to shutdown gracefully before kill;you can set environment variable ANDROID_EMULATOR_WAIT_TIME_BEFORE_KILL(in seconds) to change the default value (20 seconds)
INFO    | Discarding the changed state: command-line flag
WARNING | Discarding the changed state (command-line flag).
ERROR   | stop: Not implemented

[grodin]

This seems to be related to #381. I debugged my workflow run with tmate which similarly had two crashpad_handler processes running and the same logs as in the issue.

Terminating the two crashpad_handler processes with SIGTERM allowed the android-emulator-runner step to complete.

[ericswpark]

Seems like there should be a step at the end that kill -9s all the crashpad_handler processes.

Either that or disable crashpad_handler from running in the first place (I'm guessing it's some sort of error reporting mechanism from Google to report errors with the Android emulator?)

[grodin]

So far I've managed to find out that crashpad-handler is the daemon part of crashpad, a crash reporter.

I've done some digging in the emulator source repo. It seems the emulator uses crashpad to report crashes back to Google, so we're not going to be able to prevent the crashpad-handler processes getting started.

Killing them after the emulator has shutdown seems like a reasonable workaround, but we should do that with SIGTERM first! Going straight to kill -KILL is a bit, er, overkill. I can't immediately think of any harm, given that the VM the action is running in will be thrown away soon, but it's generally not recommended to use SIGKILL until necessary.

I'm fairly certain that this will need to be done as part of the action though.

I suspect, but haven't confirmed, that the root cause of this is that when the emulator is told to shutdown, some node.js code sends a signal to the emulator, but then waits for the whole process tree to end, not just the emulator process. However, if the emulator doesn't pass on signals to it's child processes, they won't receive any signal telling them to quit, so the waiting will go on forever.

If I'm correct (will try to find out this week), trying to kill the extra processes outside of the action won't work, since that code will be waiting for the action to finish to have a chance to run. It's a classic deadlock effectively.

[benszedlmayer]

Are there any updates here? My log output is exactly same as in the original issue, it completes all processes then just hangs forever. Trying to kill the crashpad process with kill -9 $(pgrep -f crashpad_handler) fails.

[troZee]

I have a similar issue here: https://github.com/callstack/react-native-pager-view/actions/runs/9576519647/job/26403174451?pr=829 . Does anyone know how to fix it?

[limpbrains]

I have the same issue. Emulator just hangs forever https://github.com/synonymdev/react-native-ldk/actions/runs/9724215820/job/26840196419?pr=251

[mustalk]

For anyone facing this issue, I've discovered that setting the environment variable ANDROID_EMULATOR_WAIT_TIME_BEFORE_KILL: 60 helps sometimes, but it's inconsistent.

In my case, the issue seemed to be affected by the emulator saved snapshot, so not saving the AVD snapshot should help as well.

If none of the previous solutions work, as @grodin mentioned regarding the crashpad_handler, you can use the following steps to terminate the processes:

- name: Kill crashpad_handler processes
  if: always()
  run: |
    pkill -SIGTERM crashpad_handler || true
    sleep 5
    pkill -SIGKILL crashpad_handler || true

This should definitely stop the hang issue.

[ericswpark]

@mustalk are you sure that step will work? My understanding is that the previous step will hang and stop execution of that step that will kill crashpad_handler.

[mustalk]

@ericswpark that's what i thought at first too, but to my surprise it did execute, even without the if: always(), at least in my setup, give it a try.

[fernando-jascovich]

I was having this issue while running manually android emulator (I'm not using android-emulator-runner). And looking for answers I came here. After that I discovered the solution. You'll need to kill android emulator's qemu process with SIGSTOP. For example:

# Being XXXXXX pid for android sdk qemu-system process
kill STOP XXXXXXX

That will handle snapshot generation and crashpad_handler as expected and emulator will end successfully

[ashishb]

@mustalk your suggestion didn't work for me https://github.com/ashishb/adb-enhanced/actions/runs/10024919828/job/27707518728?pr=246, it is stuck at the emulator execution step for me

Strangely, it only impacts API 26 and 29 though for me.

[Bhuvanaarkala07]

Hi Team,

We are also facing similar kind of issue. It was working 2 days back, but suddenly stops failing with below error,

Script what we are using is, runs-on: macos-13 timeout-minutes: 25

• name: Checkout the code uses: actions/checkout@v4

  • name: set up JDK 17 uses: actions/setup-java@v4 with: distribution: 'temurin' java-version: 17

    • name: Gradle cache uses: gradle/gradle-build-action@v3

  • name: AVD cache uses: actions/cache@v4 id: avd-cache with: path: | ~/.android/avd/ ~/.android/adb key: avd-29

  • name: create AVD and generate snapshot for caching if: steps.avd-cache.outputs.cache-hit != 'true' uses: reactivecircus/android-emulator-runner@v2 env: ANDROID_EMULATOR_WAIT_TIME_BEFORE_KILL: 60 with: api-level: 29 force-avd-creation: false emulator-options: -no-window -gpu swiftshader_indirect -noaudio -no-boot-anim -no-metrics -camera-back none disable-animations: false script: echo "Generated AVD snapshot for caching."

  • name: Run espresso tests uses: reactivecircus/android-emulator-runner@v2 with: api-level: 29 avd-name: test force-avd-creation: false emulator-options: -no-snapshot-save -no-window -gpu swiftshader_indirect -noaudio -no-boot-anim -no-metrics -camera-back none disable-animations: true script: ./gradlew connectedMockDebugAndroidTest

Can some one suggest what is wrong here?

[ashishb]

Fixes that worked for me was to use macos-latest instead

1. https://github.com/ashishb/adb-enhanced/pull/246
2. https://github.com/ashishb/adb-enhanced/pull/248

[Bhuvanaarkala07]

We are already using runs-on: macos-13 , but still shwoing above error.

[Braggiouy]

I am facing the same issue. The step does not terminate the emulator, and it stays stuck in the step. I tried @mustalk suggestion, but the workflow is not able to reach the step where it kills the crashpad_handler.

     - name: Set up the Android emulator and run tests
        uses: reactivecircus/android-emulator-runner@v2
        with:
          api-level: 33
          target: google_apis_playstore
          arch: x86_64
          emulator-boot-timeout: 600
          disable-animations: true
          script: ./scripts/run-tests.sh

Added process termination commands within the custom script ./scripts/run-tests.sh, but still no success. This script runs the Appium testing that I have integrated.

In addition, I have :

• ANDROID_EMULATOR_WAIT_TIME_BEFORE_KILL: 60
• And my job has the following values :

runs-on: ubuntu-latest
uses: reactivecircus/android-emulator-runner@v2

Context and Background

Emulator Running Android emulators on GitHub Actions can be challenging due to the lack of KVM support on ubuntu-latest.

macOS vs. Ubuntu The macos-latest runner includes pre-installed Android SDKs and better support for Android emulation. However, using macos-latest is more expensive compared to ubuntu-latest. Setting up a self-hosted macOS runner might be a cost-effective solution, but, I would like to try with the Ubuntu image first, if possible.

Hardware Acceleration Starting on February 23, 2023, GitHub Actions users can leverage hardware acceleration on larger Linux runners, significantly improving Android emulator performance. This requires adding the runner user to the KVM user group:

- name: Enable KVM group perms
  run: |
    echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | sudo tee /etc/udev/rules.d/99-kvm4all.rules
    sudo udevadm control --reload-rules
    sudo udevadm trigger --name-match=kvm

Questions:

Any suggestions or guidance on resolving this issue would be greatly appreciated. Specifically, I need help ensuring the emulator terminates properly, and the workflow can proceed without getting stuck.

I read earlier that this could be fixed by using the macos-latest runner. Is there any possibility to fix this by using the ubuntu-latest one ?

[vaind]

I've faced the same issue and was able to resolve it by making sure the appium instance I created in the test script gets shut down properly by the end of the script (cc @Braggiouy).

you can check pgrep -f appium before and after your script execution

[Braggiouy]

Thanks a million @vaind. That was indeed my issue. Seems that the appium instance was still running in the background, not allowing the Emulator to shut down properly. No need to manually kill the crashpad_handler. Good catch !