actions / cache

Cache dependencies and build outputs in GitHub Actions
MIT License
4.5k stars 1.2k forks source link

Caching problem - unpacking tar #198

Closed mikart143 closed 1 year ago

mikart143 commented 4 years ago

Hello ! I found error as shown in #133 I tried to fix this by using windows as system. Cache was made but the new error occured.

Run actions/cache@v1 Cache Size: ~102 MB (107455822 B) C:\windows\System32\tar.exe -xz -f d:\a_temp\5bff5da6-b2e7-439a-9f1e-b8eb52fb391e\cache.tgz -C C:\Users\runneradmin.cargo\registry ./index/github.com-1ecc6299db9ec823/.git/objects/pack/pack-a1b8e2d02676504008fea86ca360367fcee5bd43.idx: Can't unlink already-existing object ./index/github.com-1ecc6299db9ec823/.git/objects/pack/pack-a1b8e2d02676504008fea86ca360367fcee5bd43.pack: Can't unlink already-existing object tar.exe: Error exit delayed from previous errors. [warning]Tar failed with error: The process 'C:\windows\System32\tar.exe' failed with exit code 1. Ensure BSD tar is installed and on the PATH.

I am trying to cache cargo. I am using code from Action examples

dhadka commented 4 years ago

Hi @mikart143, could you please provide your full workflow file? The error Can't unlink already-existing object seems to indicate those files in the cargo registry already exist and the action does not have permission to delete them. Is cargo creating / populating the registry before the cache restore step is performed?

mikart143 commented 4 years ago

It occure on security_audit job

name: Rust

on: 
  push:
   branches:
       - master

jobs:
  security_audit:
    runs-on: windows-latest
    steps:
      - uses: actions/checkout@master
      - name: Cache cargo registry
        uses: actions/cache@v1
        with:
          path: ~/.cargo/registry
          key: ${{ runner.os }}-cargo-registry-${{ hashFiles('**/Cargo.lock') }}
      - name: Cache cargo index
        uses: actions/cache@v1
        with:
          path: ~/.cargo/git
          key: ${{ runner.os }}-cargo-index-${{ hashFiles('**/Cargo.lock') }}
      - name: Cache cargo build
        uses: actions/cache@v1
        with:
          path: target
          key: ${{ runner.os }}-cargo-build-target-${{ hashFiles('**/Cargo.lock') }}
      - uses: actions/checkout@v1
      - uses: actions-rs/audit-check@v1
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
      # - uses: EmbarkStudios/cargo-deny-action@v0

  test:
    needs: [security_audit]
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [ubuntu-latest, windows-latest, macOS-latest]
        rust: [stable]

    steps:
    - uses: hecrj/setup-rust-action@v1
      with:
        rust-version: ${{ matrix.rust }}
    - uses: actions/checkout@master
    - name: Run tests
      run: cargo test --lib --verbose 
  docs:
    needs: [security_audit, test]
    runs-on: ubuntu-latest

    steps: 
    - uses: actions/checkout@v2
    - name: Generating Docs
      run: |
        cargo doc --no-deps
        mv target/doc docs
        echo '<meta http-equiv="refresh" content="0; url=syncer/index.html">' > docs/index.html
  build:
    needs: [security_audit, test]
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2
    - name: Build
      run: cargo build --verbose
    - name: Run tests
      run: cargo test --verbose

image

mzabaluev commented 4 years ago

This workflow run demonstrates the problem. Compared to the tar of the same repository index on Linux and Mac, the archive created on Windows is more than 1.5 times larger and unpacking fails with the "Can't unlink already-existing object" messages.

My guess is that tar on Windows does not handle hardlinks or symlinks properly?

mzabaluev commented 4 years ago

One thing that may help debugging this is if you dump the tar file contents with tar -tzv prior to unpacking.

mzabaluev commented 4 years ago

It's caused by https://github.com/actions/virtual-environments/issues/895 and here's proof.

On Linux the same pre-populated repository checkout exists too, but tar manages to overwrite it without an issue.

mzabaluev commented 4 years ago

It's caused by actions/virtual-environments#895

To clarify, this explains the particular failure as it was reported. This does not explain why tar.exe in Windows fails to overwrite files that were baked into the virtual environment image, when unpacking from an archive that it itself packaged these files into, while having no problems unpacking an archive that was created by GNU tar on Linux.

mzabaluev commented 4 years ago

This does not explain why tar.exe in Windows fails to overwrite files that were baked into the virtual environment image, when unpacking from an archive that it itself packaged these files into, while having no problems unpacking an archive that was created by GNU tar on Linux.

This may be a red herring since there is no intersection in the git pack files between the two checkouts, as tested here:

$ tar -tzv --xattrs --acls -f index-linux.tgz | grep pack
drwxrwxrwx  runner/docker      0 2020-05-25 14:52 ./github.com-1ecc6299db9ec823/.git/objects/pack/
-rwxrwxrwx  runner/docker 3974748 2020-05-18 12:11 ./github.com-1ecc6299db9ec823/.git/objects/pack/pack-e63d82649edaa0ff776e5243a52d12d6f7bf5f0d.idx
-r--r--r--  runner/docker 4679324 2020-05-25 14:52 ./github.com-1ecc6299db9ec823/.git/objects/pack/pack-f3820eff603c81a0e0019112c7acc8a883d383a4.pack
-rwxrwxrwx  runner/docker 43632056 2020-05-18 12:11 ./github.com-1ecc6299db9ec823/.git/objects/pack/pack-e63d82649edaa0ff776e5243a52d12d6f7bf5f0d.pack
-r--r--r--  runner/docker   365156 2020-05-25 14:52 ./github.com-1ecc6299db9ec823/.git/objects/pack/pack-f3820eff603c81a0e0019112c7acc8a883d383a4.idx
$ tar -tzv --xattrs --acls -f index-windows.tgz | grep pack
drwxrwxrwx  0/0               0 2020-05-25 14:52 ./github.com-1ecc6299db9ec823/.git/objects/pack/
-r--r--r--  0/0          410768 2020-05-25 14:52 ./github.com-1ecc6299db9ec823/.git/objects/pack/pack-41eccb34a2a6c9172486aee7624e0889c2612d80.idx
-r--r--r--  0/0         5085807 2020-05-25 14:52 ./github.com-1ecc6299db9ec823/.git/objects/pack/pack-41eccb34a2a6c9172486aee7624e0889c2612d80.pack
-r--r--r--  0/0         3930508 2020-05-17 12:26 ./github.com-1ecc6299db9ec823/.git/objects/pack/pack-4e4ff12a1cfe7b45aa90385fcebdbf6eb08bc15d.idx
-r--r--r--  0/0        43301476 2020-05-17 12:26 ./github.com-1ecc6299db9ec823/.git/objects/pack/pack-4e4ff12a1cfe7b45aa90385fcebdbf6eb08bc15d.pack
mzabaluev commented 4 years ago

Sure enough, repro'd with the Windows content repacked on Linux.

As long as files that come from the virtual environment image are being replaced, tar fails.

mzabaluev commented 4 years ago

I see no difference in ACLs between the pre-populated files:

Path   : Microsoft.PowerShell.Core\FileSystem::C:\Users\runneradmin\.cargo\registry\index\github.com-1ecc6299db9ec823\.
         git\objects\pack
Owner  : BUILTIN\Administrators
Group  : fv-az103\None
Access : BUILTIN\Administrators Allow  FullControl
         BUILTIN\Administrators Allow  FullControl
         NT AUTHORITY\SYSTEM Allow  FullControl
         CREATOR OWNER Allow  268435456
         BUILTIN\Users Allow  ReadAndExecute, Synchronize
         BUILTIN\Users Allow  AppendData
         BUILTIN\Users Allow  CreateFiles
Audit  : 
Sddl   : O:BAG:S-1-5-21-3063699593-698096204-2912911206-513D:(A;;FA;;;BA)(A;OICIID;FA;;;BA)(A;OICIID;FA;;;SY)(A;OICIIOI
         D;GA;;;CO)(A;OICIID;0x1200a9;;;BU)(A;CIID;LC;;;BU)(A;CIID;DC;;;BU)

Path   : Microsoft.PowerShell.Core\FileSystem::C:\Users\runneradmin\.cargo\registry\index\github.com-1ecc6299db9ec823\.
         git\objects\pack\pack-4e4ff12a1cfe7b45aa90385fcebdbf6eb08bc15d.idx
Owner  : BUILTIN\Administrators
Group  : fv-az103\None
Access : BUILTIN\Administrators Allow  FullControl
         NT AUTHORITY\SYSTEM Allow  FullControl
         BUILTIN\Users Allow  ReadAndExecute, Synchronize
Audit  : 
Sddl   : O:BAG:S-1-5-21-3063699593-698096204-2912911206-513D:(A;ID;FA;;;BA)(A;ID;FA;;;SY)(A;ID;0x1200a9;;;BU)

Path   : Microsoft.PowerShell.Core\FileSystem::C:\Users\runneradmin\.cargo\registry\index\github.com-1ecc6299db9ec823\.
         git\objects\pack\pack-4e4ff12a1cfe7b45aa90385fcebdbf6eb08bc15d.pack
Owner  : BUILTIN\Administrators
Group  : fv-az103\None
Access : BUILTIN\Administrators Allow  FullControl
         NT AUTHORITY\SYSTEM Allow  FullControl
         BUILTIN\Users Allow  ReadAndExecute, Synchronize
Audit  : 
Sddl   : O:BAG:S-1-5-21-3063699593-698096204-2912911206-513D:(A;ID;FA;;;BA)(A;ID;FA;;;SY)(A;ID;0x1200a9;;;BU)

and files created by a script in a runner job:

    Directory: C:\Users\runneradmin\.cargo\registry

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a---           5/25/2020  2:51 PM              0 foo

Path   : Microsoft.PowerShell.Core\FileSystem::C:\Users\runneradmin\.cargo\registry\foo
Owner  : BUILTIN\Administrators
Group  : fv-az103\None
Access : BUILTIN\Administrators Allow  FullControl
         NT AUTHORITY\SYSTEM Allow  FullControl
         BUILTIN\Users Allow  ReadAndExecute, Synchronize
Audit  : 
Sddl   : O:BAG:S-1-5-21-3063699593-698096204-2912911206-513D:(A;ID;FA;;;BA)(A;ID;FA;;;SY)(A;ID;0x1200a9;;;BU)

    Directory: C:\Users\runneradmin\.cargo\registry

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d----           5/25/2020  2:51 PM                bar

Path   : Microsoft.PowerShell.Core\FileSystem::C:\Users\runneradmin\.cargo\registry\bar
Owner  : BUILTIN\Administrators
Group  : fv-az103\None
Access : BUILTIN\Administrators Allow  FullControl
         BUILTIN\Administrators Allow  FullControl
         NT AUTHORITY\SYSTEM Allow  FullControl
         CREATOR OWNER Allow  268435456
         BUILTIN\Users Allow  ReadAndExecute, Synchronize
         BUILTIN\Users Allow  AppendData
         BUILTIN\Users Allow  CreateFiles
Audit  : 
Sddl   : O:BAG:S-1-5-21-3063699593-698096204-2912911206-513D:(A;;FA;;;BA)(A;OICIID;FA;;;BA)(A;OICIID;FA;;;SY)(A;OICIIOI
         D;GA;;;CO)(A;OICIID;0x1200a9;;;BU)(A;CIID;LC;;;BU)(A;CIID;DC;;;BU)
mzabaluev commented 4 years ago

This has nothing to do with ACLs or symlinks (AKA junctions), and everything to do with read-only files: https://github.com/actions/virtual-environments/issues/936

mzabaluev commented 4 years ago

A solution suggested in https://github.com/actions/virtual-environments/issues/936#issuecomment-634068778 is to use the tar executable bundled with Git, and use the --force-local option to interpret Windows path names as local file paths rather than remote tape drive servers :old_key:.

lukka commented 3 years ago

Any updates on this long standing issue? Look like @mzabaluev has done the needed investigation and found a potential solution.

This affects any consumer of the GitHub action actions/cache and of the @actions/cache NPM package.

It affects for example run-vpkg.

github-actions[bot] commented 2 years ago

This issue is stale because it has been open for 365 days with no activity. Leave a comment to avoid closing this issue in 5 days.

mzabaluev commented 2 years ago

Yes, we still need tar in the cache action to be fixed.

lvpx commented 1 year ago

Hi @mzabaluev we are tracking this issue in #984. Please check the current proposal there and provide comments/feedback if any. Closing this as duplicate.

Phantsure commented 1 year ago

👋🏼 @mzabaluev We have released a new beta release which should fix this issue. Try tag: actions/cache@3.1.0-beta. Head over to discussion for feedback: https://github.com/actions/cache/discussions/1019