reproducible-containers / buildkit-cache-dance

Save `RUN --mount=type=cache` caches on GitHub Actions ( Forked from https://github.com/overmindtech/buildkit-cache-dance )
Apache License 2.0
98 stars 25 forks source link

feat!: rewrite in TypeScript with CacheMap support #25

Closed aminya closed 2 months ago

aminya commented 2 months ago

This pull request rewrites the action in TypeScript and adds support for cache-map that gets a string of files that need to be injected as a JSON string. This makes it possible to inject multiple directories in one call and simplifies the usage.

It also makes it possible to run the script outside GitHub Actions (e.g. for S3) or locally using command line arguments.

BREAKING cache-source and cache-target are deprecated in favour of cache-map that expects a JSON string.

I have bumped the version to v3.0.0

Fixes #10 Closes #24 Closes #22
Allows implementation of #16

      - name: Cache
        uses: actions/cache@v3
        id: cache
        with:
          path: |
            var-cache-apt
            var-lib-apt
          key: cache-${{ hashFiles('.github/workflows/test/Dockerfile') }}

      - name: inject cache into docker
        uses: ./
        with:
          cache-map: |
            {
              "var-cache-apt": "/var/cache/apt",
              "var-lib-apt": "/var/lib/apt"
            }
          skip-extraction: ${{ steps.cache.outputs.cache-hit }}
aminya commented 2 months ago

Recovered the cache-source and cache-target options but with deprecation messages. Also, added an example of running this outside GitHub Actions.

AkihiroSuda commented 2 months ago

https://github.com/reproducible-containers/buildkit-cache-dance/actions/runs/8490107723/job/23260959438?pr=25

Post job cleanup.

FROM busybox:1
COPY buildstamp buildstamp
RUN --mount=type=cache,target=/var/cache/apt     mkdir -p /var/dance-cache/     && cp -p -R /var/cache/apt/. /var/dance-cache/ || true

node:internal/process/esm_loader:40
      internalBinding('errors').triggerUncaughtException(
                                ^
tar: Skipping to next header
tar: Exiting with failure status due to previous errors

(Use `node --trace-uncaught ...` to show where the exception was thrown)

Node.js v[2](https://github.com/reproducible-containers/buildkit-cache-dance/actions/runs/8490107723/job/23260959438?pr=25#step:11:2)0.8.1
aminya commented 2 months ago

The error you sent doesn't happen on my fork. But I pushed a fix regardless for more robust error handling https://github.com/aminya/buildkit-cache-dance/actions

AkihiroSuda commented 2 months ago

The test passes when some cache is present on GHA, but it still fails after pruning the cache via https://github.com/reproducible-containers/buildkit-cache-dance/actions/caches

AkihiroSuda commented 2 months ago

Also, please remove the "Merge" commit (use git rebase, not git merge), and consider squashing the commits

aminya commented 2 months ago

The test passes when some cache is present on GHA, but it still fails after pruning the cache via reproducible-containers/buildkit-cache-dance/actions/caches

Okay, found the bug. The stdout was not being captured properly.

aminya commented 2 months ago

Simplified and fixed the docker to tar data piping issue. It now works on my fork with cache pruned as well: https://github.com/aminya/buildkit-cache-dance/actions/runs/8494967106/job/23270766398

strophy commented 2 months ago

Thanks @aminya this is a really significant improvement!

aminya commented 2 months ago

You're welcome!