search

awalsh128 / cache-apt-pkgs-action

Cache APT packages in GitHub Actions
Other
205 stars 35 forks source link

ls: cannot access '/home/runner/cache-apt-pkgs/*.tar': No such file or directory #82

Closed marwenguesmii closed 1 year ago

marwenguesmii commented 1 year ago

My problem is quite simple actually when I used this method to cache the package vault. it was compressed into this file vault:1.12.2-1.tar.gz so when this method tries to fetch all the compressed packages it won't find it since the search command is cached_pkg_filepaths=$(ls -1 "${cache_dir}"/*.tar | sort)

schlomo commented 1 year ago

I have the same problem with another package (aha) and the result is that there is no error but the package is actually not installed.

2023-01-03T15:34:46.0329215Z ##[group]Run awalsh128/cache-apt-pkgs-action@latest
2023-01-03T15:34:46.0329541Z with:
2023-01-03T15:34:46.0329836Z   packages: aha
2023-01-03T15:34:46.0330099Z   execute_install_scripts: false
2023-01-03T15:34:46.0330363Z   debug: false
2023-01-03T15:34:46.0330568Z ##[endgroup]
2023-01-03T15:34:46.0747067Z ##[group]Run ${GITHUB_ACTION_PATH}/pre_cache_action.sh \
2023-01-03T15:34:46.0747609Z ${GITHUB_ACTION_PATH}/pre_cache_action.sh \
2023-01-03T15:34:46.0748016Z   ~/cache-apt-pkgs \
2023-01-03T15:34:46.0748274Z   "" \
2023-01-03T15:34:46.0748483Z   "false" \
2023-01-03T15:34:46.0748717Z   "false" \
2023-01-03T15:34:46.0748943Z   aha
2023-01-03T15:34:46.0749255Z echo "CACHE_KEY=$(cat ~/cache-apt-pkgs/cache_key.md5)" >> $GITHUB_ENV
2023-01-03T15:34:46.0829909Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2023-01-03T15:34:46.0830379Z ##[endgroup]
2023-01-03T15:34:46.1320280Z 15:34:46 Validating action arguments (version='', packages='aha ')...
2023-01-03T15:34:46.1354769Z 15:34:46 done
2023-01-03T15:34:46.1355197Z 
2023-01-03T15:34:46.1373853Z 15:34:46 Verifying packages...
2023-01-03T15:34:48.3684881Z 15:34:48 done
2023-01-03T15:34:48.3689625Z 
2023-01-03T15:34:48.3709137Z 15:34:48 Creating cache key...
2023-01-03T15:34:48.3850237Z 15:34:48 - Normalized package list is 'aha=0.5.1-2 '.
2023-01-03T15:34:48.3867632Z 15:34:48 - Value to hash is 'aha=0.5.1-2  @ '.
2023-01-03T15:34:48.3926078Z 15:34:48 - Value hashed as '1c3d96b2f2ed52cd39029ff43dc16877'.
2023-01-03T15:34:48.3941699Z 15:34:48 done
2023-01-03T15:34:48.3961060Z 15:34:48 Hash value written to /home/runner/cache-apt-pkgs/cache_key.md5
2023-01-03T15:34:48.4059989Z ##[group]Run actions/cache@v3
2023-01-03T15:34:48.4060224Z with:
2023-01-03T15:34:48.4060442Z   path: ~/cache-apt-pkgs
2023-01-03T15:34:48.4060898Z   key: cache-apt-pkgs_1c3d96b2f2ed52cd39029ff43dc16877
2023-01-03T15:34:48.4061170Z env:
2023-01-03T15:34:48.4061432Z   CACHE_KEY: 1c3d96b2f2ed52cd39029ff43dc16877
2023-01-03T15:34:48.4061669Z ##[endgroup]
2023-01-03T15:34:54.3136879Z Received 16470 of 16470 (100.0%), 0.3 MBs/sec
2023-01-03T15:34:54.3138170Z Cache Size: ~0 MB (16470 B)
2023-01-03T15:34:54.3170088Z [command]/usr/bin/tar --use-compress-program unzstd -xf /home/runner/work/_temp/9c3ef2ee-1d5a-404a-9890-d8d89704c22f/cache.tzst -P -C /home/runner/work/china-saas-connectivity/china-saas-connectivity
2023-01-03T15:34:54.3256282Z Cache restored successfully
2023-01-03T15:34:54.3554899Z Cache restored from key: cache-apt-pkgs_1c3d96b2f2ed52cd39029ff43dc16877
2023-01-03T15:34:54.3726390Z ##[group]Run ${GITHUB_ACTION_PATH}/post_cache_action.sh \
2023-01-03T15:34:54.3726800Z ${GITHUB_ACTION_PATH}/post_cache_action.sh \
2023-01-03T15:34:54.3727118Z   ~/cache-apt-pkgs \
2023-01-03T15:34:54.3727376Z   / \
2023-01-03T15:34:54.3727608Z   "true" \
2023-01-03T15:34:54.3727846Z   "false" \
2023-01-03T15:34:54.3728081Z   "false" \
2023-01-03T15:34:54.3728313Z   aha
2023-01-03T15:34:54.3728696Z function create_list { local list=$(cat ~/cache-apt-pkgs/manifest_${1}.log | tr '\n' ','); echo ${list:0:-1}; };
2023-01-03T15:34:54.3729384Z echo "package-version-list=$(create_list main)" >> $GITHUB_OUTPUT
2023-01-03T15:34:54.3730143Z echo "all-package-version-list=$(create_list all)" >> $GITHUB_OUTPUT
2023-01-03T15:34:54.3812105Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2023-01-03T15:34:54.3812473Z env:
2023-01-03T15:34:54.3812750Z   CACHE_KEY: 1c3d96b2f2ed52cd39029ff43dc16877
2023-01-03T15:34:54.3813027Z ##[endgroup]
2023-01-03T15:34:54.4102471Z 15:34:54 Found 5 files in the cache.
2023-01-03T15:34:54.4135209Z 15:34:54 - aha:0.5.1-2.tar.gz
2023-01-03T15:34:54.4163016Z 15:34:54 - cache_key.md5
2023-01-03T15:34:54.4193751Z 15:34:54 - install.log
2023-01-03T15:34:54.4221828Z 15:34:54 - manifest_all.log
2023-01-03T15:34:54.4255595Z 15:34:54 - manifest_main.log
2023-01-03T15:34:54.4256607Z 
2023-01-03T15:34:54.4276962Z 15:34:54 Reading from main requested packages manifest...
2023-01-03T15:34:54.4362513Z 15:34:54 - aha 0.5.1-2
2023-01-03T15:34:54.4381122Z 15:34:54 done
2023-01-03T15:34:54.4381611Z 
2023-01-03T15:34:54.4406394Z ls: cannot access '/home/runner/cache-apt-pkgs/*.tar': No such file or directory
2023-01-03T15:34:54.4453228Z 15:34:54 Restoring 0 packages from cache...
2023-01-03T15:34:54.4467577Z 15:34:54 done
2023-01-03T15:34:54.4471058Z 
2023-01-03T15:34:54.4591138Z ##[group]Run actions/upload-artifact@v3
2023-01-03T15:34:54.4592438Z with:
2023-01-03T15:34:54.4592723Z   name: cache-apt-pkgs-logs%aha%
2023-01-03T15:34:54.4593033Z   path: ~/cache-apt-pkgs/*.log
2023-01-03T15:34:54.4593322Z   if-no-files-found: warn
2023-01-03T15:34:54.4593568Z env:
2023-01-03T15:34:54.4593817Z   CACHE_KEY: 1c3d96b2f2ed52cd39029ff43dc16877
2023-01-03T15:34:54.4594093Z ##[endgroup]
2023-01-03T15:34:54.5557427Z With the provided path, there will be 3 files uploaded
2023-01-03T15:34:54.5563816Z Starting artifact upload
2023-01-03T15:34:54.5565304Z For more detailed logs during the artifact upload process, enable step-debugging: https://docs.github.com/actions/monitoring-and-troubleshooting-workflows/enabling-debug-logging#enabling-step-debug-logging
2023-01-03T15:34:54.5615147Z Artifact name is valid!
2023-01-03T15:34:54.6635480Z Container for artifact "cache-apt-pkgs-logs%aha%" successfully created. Starting upload of file(s)
2023-01-03T15:34:55.1055728Z Total size of all the files uploaded is 749 bytes
2023-01-03T15:34:55.1057029Z File upload process has finished. Finalizing the artifact upload
2023-01-03T15:34:55.1726546Z Artifact has been finalized. All files have been successfully uploaded!
2023-01-03T15:34:55.1727289Z 
2023-01-03T15:34:55.1728054Z The raw size of all the files that were specified for upload is 1888 bytes
2023-01-03T15:34:55.1729474Z The size of all the files that were uploaded is 749 bytes. This takes into account any gzip compression used to reduce the upload size, time and storage
2023-01-03T15:34:55.1730057Z 
2023-01-03T15:34:55.1731116Z Note: The size of downloaded zips can differ significantly from the reported size. For more information see: https://github.com/actions/upload-artifact#zipped-artifact-downloads 
2023-01-03T15:34:55.1731782Z 
2023-01-03T15:34:55.1732143Z Artifact cache-apt-pkgs-logs%aha% has been successfully uploaded!

Any ideas how to get this fixed?

awalsh128 commented 1 year ago

Sorry to hear this. Go back to v1.2.1 and confirm if this fixes it. I may revert the latest and v1 tags if this is a new bug introduced in the release.

Also, if you bump the version input, it should reinstall and make it go away. The bug is still there though for those who don't rev their version.

awalsh128 commented 1 year ago

FYI, I reverted the tags. This looks like a dupe of #83 but I am going to leave open until I can confirm.

awalsh128 commented 1 year ago

@schlomo and @Marwennnne I believe this was from an older release awhile ago where compression was pulled since the cache actions already compresses and removing would speed it up. Bumping the input version fixes the issue. Did either of you upgrade recently?

schlomo commented 1 year ago

I tried using latest and then did a "regular" apt-get install step as I didn't have time to debug this and was trying it out to get a one-liner for installing dependencies.

martinling commented 1 year ago

I am using latest and still seeing this issue, despite bumping the cache version after @awalsh128's comment saying the tags had now been reverted.

The failure can be seen in the CI run for my commit bumping the cache version, which is here: https://github.com/greatscottgadgets/packetry/pull/54/commits/238efab32a884048b93a8624494e0a63a32dbcd5

martinling commented 1 year ago

It seems that reverting the latest tag did not cause GitHub Actions to revert workflows using that tag to the older version. I had to explicitly target v1.2.1 and bump the cache version to avoid this bug, as in the PR linked above.

Note that the v1 tag is currently still pointing to the broken v1.2.2 release. Only the latest tag was reverted.

martinling commented 1 year ago

Sorry, my mistake - it's fine to use latest with a cache version bump, I'd just forgotten to bump the version on a second use of the action.

However, the other info in my previous comment is correct - v1 still points to the broken release.

awalsh128 commented 1 year ago

Sorry for the long lag here. To avoid other people encountering this and having to manually fix, I am going to add a feature to automatically refresh the catch when breaking changes are encountered. I'll update this issue once I have something ready.

awalsh128 commented 1 year ago

Okay issued a patch so this doesn't happen again. All tags are rev'd at current release.