jirutka / setup-alpine

Easily use Alpine Linux on GitHub Actions, with support for QEMU user emulator
MIT License
93 stars 14 forks source link

Post Run exits with failure - unmount "target is busy" #7

Closed bamarch closed 9 months ago

bamarch commented 1 year ago

Thanks for the action this is very useful for us to run some tests in Alpine rather than Ubuntu

The action works well for us, but we get this failure during the Post Run

image

Unfortunately this marks the job as failed, so pull requests can't be merged etc.

Is there something we can do here? We're using jirutka/setup-alpine@v1 and branch v3.16

bamarch commented 1 year ago

The yaml from the workflow, for extra context

name: PR

on:
  pull_request:
    branches: [ main ]
    types:
      - opened
      - reopened
      - synchronize

env:
  ALPINE_DOTNET_PACKAGE: dotnet6-sdk
  ALPINE_VERSION: v3.16
  DOTNET_SDK_VERSION: 6.0.101
  SRC_DIR: ./src

jobs:

  dotnet-test:
    name: "Dotnet Test"
    concurrency:
      group: ${{ github.workflow }}_dotnet-test # global lock, not scoped to a specific commit
      cancel-in-progress: false
    runs-on: ubuntu-latest
    timeout-minutes: 30
    if: ${{ github.actor != 'dependabot[bot]' }}
    defaults:
      run:
        working-directory: ${{ env.SRC_DIR }}
    steps:
      - uses: actions/checkout@v3

      - name: "Checkout Infrastructure repo"
        uses: actions/checkout@v3
        with:
          fetch-depth: 0
          repository: BondOrigination/infrastructure
          path: infra-repo
          ssh-key: ${{ secrets.DEPLOY_KEY_INFRASTRUCTURE_READ_ONLY }}

      - uses: jirutka/setup-alpine@v1
        with:
          branch: ${{ env.ALPINE_VERSION }}
          packages: >
            ${{ env.ALPINE_DOTNET_PACKAGE }}
            icu-libs
            tzdata

      - name: Build solution in release mode
        run: dotnet build -c Release
        shell: alpine.sh {0}

      - uses: ./infra-repo/.github/actions/vpn-connect
        timeout-minutes: 3
        with:
          open-vpn-config-file-secret: ${{ secrets.VPN_CONFIG_FILE }}

      - name: Run Tests
        run: dotnet test --no-build -c Release --logger trx --results-directory "${{github.workspace}}/results" --blame-hang-timeout 2m
        shell: alpine.sh {0}

      - uses: ./infra-repo/.github/actions/vpn-disconnect
        if: always()

      - name: Test Report
        uses: dorny/test-reporter@v1
        if: (success() || failure()) && (github.actor != 'dependabot[bot]') # the job requires write access to upload test results artifact
        with:
          name: Tests
          path: "**/*.trx"
          reporter: dotnet-trx

      - name: Upload Test Artifacts
        uses: actions/upload-artifact@v3
        if: failure() # ensures upload even if steps failed
        with:
          name: "System Logs"
          path: ${{ github.workspace }}/**/*.log

      - name: Upload Test Dump Artifacts
        uses: actions/upload-artifact@v3
        if: failure() # ensures upload even if steps failed
        with:
          name: "Test Dumps"
          path: ${{ github.workspace }}/**/*.dmp

and more context for the failure

image

bamarch commented 1 year ago

After some lsof and ps debugging I found that there was a backgrounded dotnet process running that unmount was complaining about.

Inserting a pkill right at the end of the job has prevented the job failures.

      - name: Upload Test Dump Artifacts
        uses: actions/upload-artifact@v3
        if: failure() # ensures upload even if steps failed
        with:
          name: "Test Dumps"
          path: ${{ github.workspace }}/**/*.dmp

      - name: Killed backgrounded dotnet processes # prevent failures from setup-alpine Post Run
        run: pkill -f /usr/lib/dotnet/dotnet
        shell: alpine.sh {0}

What would be the consequences if the action was quiet about errors like this... or didn't do a destroy at the end of the job?

This would be similar to other GHA setup-foo actions, like setup-node. Or could have a flag, destroy=false/true?

Anyway the action is working well for us with this workaround.

Cheers again!

rtrubshaw commented 11 months ago

This has been a thorn in our side for a while.

Thanks @bamarch for the heads up about background processes.

Meanwhile, would issuing theumount with the -l option (lazy unmount) be possible?

Or just ignore the errors? At this point does it really matter?

jirutka commented 9 months ago

Thanks for troubleshooting and sorry it took so long to fix it.