docker/setup-buildx-action@v2 fails with no information

[iquasere]

Behaviour

Steps to reproduce this issue

1. Run the tests at https://github.com/iquasere/MOSCA/actions/runs/5247587094
2. And look at the mgmt-workflow. That is the one causing the problems, as the other grey tests are blocked (for other reasons).
3. First attempt failed, but it made no sense, sense no error was reported, and the Post Set up Docker Buildx step had never failed.
4. So I ran it a second time, and it worked!
5. But a third time (I ran it again because at the master branch the same workflow failed) also failed.

Expected behaviour

The Post Set up Docker Buildx step should be working, as it has always. There was even one time where it worked again, but ever since, the final docker action has failed consistently.

Actual behaviour

The workflow fails the Post Set up Docker Buildx step.

Configuration

• Repository URL (if public): https://github.com/iquasere/MOSCA
• Build URL (if public): https://github.com/iquasere/MOSCA/blob/master/Dockerfile_CI (not sure if this is the intended file)

name: Continuous Integration

on:
  push:
    branches:
      - development
      - master        # master test only useful for badges
  pull_request:
    branches:
      - development
      - master        # master test only useful for badges

  workflow_dispatch:

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2
      - name: Build and push
        uses: docker/build-push-action@v2
        with:
          context: .
          file: ./Dockerfile_CI
          tags: mosca:latest
          outputs: type=docker,dest=/tmp/mosca.tar
      - name: Create named volume
        run: docker volume create named_volume
      - name: Upload artifact
        uses: actions/upload-artifact@v3
        with:
          name: mosca
          path: /tmp/mosca.tar

  preprocess:
    runs-on: ubuntu-latest
    needs: build
    steps:
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2
      - name: Download artifact
        uses: actions/download-artifact@v3
        with:
          name: mosca
          path: /tmp
      - name: Load Docker image
        run: |
          docker load --input /tmp/mosca.tar
          docker image ls -a
      - name: run preprocess
        run: |
          docker run mosca /bin/bash -c "rm -r MOSCARDO/output_rules/Preprocess;
          snakemake --cores all -R --until preprocess -s MOSCA/workflow/Snakefile --configfile MOSCARDO/configs/rules_config.json --use-conda"

  assembly:
    runs-on: ubuntu-latest
    needs: build
    steps:
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2
      - name: Download artifact
        uses: actions/download-artifact@v3
        with:
          name: mosca
          path: /tmp
      - name: Load Docker image
        run: |
          docker load --input /tmp/mosca.tar
          docker image ls -a
      - name: run assembly
        run: |
          docker run mosca /bin/bash -c "rm -r MOSCARDO/output_rules/Assembly; gunzip -r MOSCARDO/output_rules; 
          touch MOSCARDO/output_rules/Preprocess/Trimmomatic/quality_trimmed_mg_forward_paired.fq;
          touch MOSCARDO/output_rules/Preprocess/Trimmomatic/quality_trimmed_mg_reverse_paired.fq;
          snakemake --cores all -R --until assembly -s MOSCA/workflow/Snakefile --configfile MOSCARDO/configs/rules_config.json --use-conda"

  binning:
    runs-on: ubuntu-latest
    needs: build
    if: false  # Blocked for now, since GHA don't provide enough memory to run this step ( 7Gb < 32Gb )
    steps:
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2
      - name: Download artifact
        uses: actions/download-artifact@v3
        with:
          name: mosca
          path: /tmp
      - name: Load Docker image
        run: |
          docker load --input /tmp/mosca.tar
          docker image ls -a
      - name: run binning
        run: |
          docker run mosca /bin/bash -c "gunzip -r MOSCARDO/output_rules; 
          touch MOSCARDO/output_rules/Preprocess/Trimmomatic/quality_trimmed_mg_forward_paired.fq;
          touch MOSCARDO/output_rules/Preprocess/Trimmomatic/quality_trimmed_mg_reverse_paired.fq;
          touch MOSCARDO/output_rules/Preprocess/sample_forward.fastq MOSCARDO/output_rules/Preprocess/sample_reverse.fastq;
          touch MOSCARDO/output_rules/Assembly/sample/contigs.fasta MOSCARDO/output_rules/Assembly/sample/scaffolds.fasta;
          snakemake --cores all -R --until binning -s MOSCA/workflow/Snakefile --configfile MOSCARDO/configs/rules_config.json --use-conda --ignore-incomplete"

  mgmt-workflow:
    runs-on: ubuntu-latest
    needs: build
    steps:
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2
      - name: Download artifact
        uses: actions/download-artifact@v3
        with:
          name: mosca
          path: /tmp
      - name: Load Docker image
        run: |
          docker load --input /tmp/mosca.tar
          docker image ls -a
      - name: MGMT workflow
        run: docker run mosca /bin/bash -c "rm -r MOSCARDO/output_rules; mosca -c MOSCARDO/configs/mgmt_config.json"

  mgmp-workflow:
    runs-on: ubuntu-latest
    needs: build
    if: false  # Blocked for now, since GHA don't provide enough disk space to run this step
    steps:
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2
      - name: Download artifact
        uses: actions/download-artifact@v3
        with:
          name: mosca
          path: /tmp
      - name: Load Docker image
        run: |
          docker load --input /tmp/mosca.tar
          docker image ls -a
      - name: Create named volume
        run: docker volume create named_volume
      - name: MGMP workflow
        run: >
          docker run -v named_volume:/data -v /var/run/docker.sock:/var/run/docker.sock mosca /bin/bash
          -c "rm -r MOSCARDO/output_rules; gunzip -v MOSCARDO/input/*/mp*.RAW.gz; 
          mosca -c MOSCARDO/configs/mgmp_config.json"

Logs

logs_183.zip

[crazy-max]

Doesn't look to be an issue with the action but your workflow.

Don't see any logs in your "MGMT workflow" step: https://github.com/iquasere/MOSCA/actions/runs/5247587094/jobs/9502710801

docker run in this step could have crashed the runner. Maybe out of disk space so post step fails. I don't really know without logs.

Can you try disabling the cleanup post step?:

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2
        with:
        cleanup: false

Also looking at the workflow you don't need docker/setup-buildx-action@v2 for all jobs except the first one as you're not building anything with buildx.

[iquasere]

Thank you for the quick answer! Removing the buildx from where it wasn't needed fixed the failing actions, it likely was because the space was getting filled up indeed.