search

goto-bus-stop / setup-zig

use a @ziglang compiler in your github actions workflows
Other
151 stars 18 forks source link

Caching not working #72

Open kristoff-it opened 9 months ago

kristoff-it commented 9 months ago

I've noticed that all of my recent Actions run present the same message from setup-zig which sees a cache miss when trying to find Zig, and that fails to save because of a cache hit:

attempting restore of zig-zig-linux-x86_64-0.12.0-dev.2701+d18f52197 to /opt/hostedtoolcache/zig/0.12.0-dev.2701+d18f52197/x64
no cached version found. downloading zig zig-linux-x86_64-0.12.0-dev.2701+d18f52197
/usr/bin/tar x --warning=no-unknown-keyword --overwrite -C /home/runner/work/_temp/a2925824-88c2-4f0b-a132-90b405b75bbe -f /home/runner/work/_temp/23395170-efbe-49b8-b9c7-5fa227cba1e6
adding zig 0.12.0-dev.2701+d18f52197 at /opt/hostedtoolcache/zig/0.12.0-dev.2701/x64 to local cache zig-zig-linux-x86_64-0.12.0-dev.2701+d18f52197
/usr/bin/tar --posix -cf cache.tzst --exclude cache.tzst -P -C /home/runner/work/zine-ssg.io/zine-ssg.io --files-from manifest.txt --use-compress-program zstdmt
Failed to save: Unable to reserve cache with key zig-zig-linux-x86_64-0.12.0-dev.2701+d18f52197, another job may be creating this cache. More details: Cache already exists. Scope: refs/heads/main, Key: zig-zig-linux-x86_64-0.12.0-dev.2701+d18f52197, Version: dfe724fa46fc50cb24d4d622fad3242b17fef35a3fbeeb758b25ad91b726458f
zig installed at /opt/hostedtoolcache/zig/0.12.0-dev.2701/x64

To diagnose the problem a bit I've deleted my cached Zig and run two deployments right after (in sequence, not in parallel):

First run (link):

attempting restore of zig-zig-linux-x86_64-0.12.0-dev.2701+d18f52197 to /opt/hostedtoolcache/zig/0.12.0-dev.2701+d18f52197/x64
no cached version found. downloading zig zig-linux-x86_64-0.12.0-dev.2701+d18f52197
/usr/bin/tar x --warning=no-unknown-keyword --overwrite -C /home/runner/work/_temp/e641dc9f-ee95-4dc5-a868-be3aeb3a46f5 -f /home/runner/work/_temp/7e958929-ff11-4828-9c5f-9502b4edc1a9
adding zig 0.12.0-dev.2701+d18f52197 at /opt/hostedtoolcache/zig/0.12.0-dev.2701/x64 to local cache zig-zig-linux-x86_64-0.12.0-dev.2701+d18f52197
/usr/bin/tar --posix -cf cache.tzst --exclude cache.tzst -P -C /home/runner/work/zine-ssg.io/zine-ssg.io --files-from manifest.txt --use-compress-program zstdmt
Cache Size: ~70 MB (73624515 B)
Cache saved successfully
zig installed at /opt/hostedtoolcache/zig/0.12.0-dev.2701/x64

Second run (link, triggered after the first had finished):

attempting restore of zig-zig-linux-x86_64-0.12.0-dev.2701+d18f52197 to /opt/hostedtoolcache/zig/0.12.0-dev.2701+d18f52197/x64
no cached version found. downloading zig zig-linux-x86_64-0.12.0-dev.2701+d18f52197
/usr/bin/tar x --warning=no-unknown-keyword --overwrite -C /home/runner/work/_temp/df075ae4-e2c2-49b1-b1c0-c09b0720ed1f -f /home/runner/work/_temp/0778b3c5-ce35-4725-b4ca-e8cb0af0b6d9
adding zig 0.12.0-dev.2701+d18f52197 at /opt/hostedtoolcache/zig/0.12.0-dev.2701/x64 to local cache zig-zig-linux-x86_64-0.12.0-dev.2701+d18f52197
/usr/bin/tar --posix -cf cache.tzst --exclude cache.tzst -P -C /home/runner/work/zine-ssg.io/zine-ssg.io --files-from manifest.txt --use-compress-program zstdmt
Failed to save: Unable to reserve cache with key zig-zig-linux-x86_64-0.12.0-dev.2701+d18f52197, another job may be creating this cache. More details: Cache already exists. Scope: refs/heads/main, Key: zig-zig-linux-x86_64-0.12.0-dev.2701+d18f52197, Version: dfe724fa46fc50cb24d4d622fad3242b17fef35a3fbeeb758b25ad91b726458f
zig installed at /opt/hostedtoolcache/zig/0.12.0-dev.2701/x64

Somehow, it seems that the step fails to find existing caches.

This is a big problem because it means that the ZSF incurs in avoidable expenses.

goto-bus-stop commented 8 months ago

Per https://github.com/goto-bus-stop/setup-zig/issues/59 this should only happen when a commit is specified on the version, not for normal releases, so I hope the impact is ultimately not that big (I don't have a good view of how many people are pinning a commit vs. using a stable release though.)

That said it obviuosly still needs to be solved, because a small impact is still an impact. https://github.com/goto-bus-stop/setup-zig/pull/60 contains a potential fix, it would be quite helpful if someone could validate the approach and finish it up.

squeek502 commented 8 months ago

I don't have a good view of how many people are pinning a commit vs. using a stable release though

Many (if not most) people use Zig nightly releases rather than stable.

I've been using the jethrodaniel/setup-zig@fix-caching branch and it seems to be working fine:

https://github.com/squeek502/resinator/actions/runs/8220331040/job/22479316486

Run jethrodaniel/setup-zig@fix-caching
  with:
    version: 0.12.0-dev.3074+ae7f3fc36
    cache: true
attempting restore of zig-zig-windows-x86_64-0.12.0-dev.3074 to C:\hostedtoolcache\windows\zig\0.12.0-dev.3074\x64
Cache Size: ~69 MB (72842377 B)
"C:\Program Files\Git\usr\bin\tar.exe" -xf D:/a/_temp/a707b7dc-b87e-439d-aa6f-3fa78e5f59ac/cache.tzst -P -C D:/a/resinator/resinator --force-local --use-compress-program "zstd -d"
Received 72842377 of 72842377 (100.0%), 68.9 MBs/sec
Cache restored successfully
using cached zig install: C:\hostedtoolcache\windows\zig\0.12.0-dev.3074\x64
zig installed at C:\hostedtoolcache\windows\zig\0.12.0-dev.3074\x64
andy5995 commented 8 months ago

Many (if not most) people use Zig nightly releases rather than stable.

I use the nightly.

2024-03-13T08:59:45.0065955Z ##[group]Run goto-bus-stop/setup-zig@v2
2024-03-13T08:59:45.0066541Z with:
2024-03-13T08:59:45.0066973Z   version: master
2024-03-13T08:59:45.0067385Z   cache: true
2024-03-13T08:59:45.0067755Z ##[endgroup]
2024-03-13T08:59:45.4733514Z attempting restore of zig-zig-linux-x86_64-0.12.0-dev.3282+da5b16f9e to /opt/hostedtoolcache/zig/master/x64
2024-03-13T08:59:46.5489855Z Cache Size: ~66 MB (68872401 B)
2024-03-13T08:59:46.5521350Z [command]/usr/bin/tar -xf /home/runner/work/_temp/f9c89e21-798a-48ce-bbd9-5e190ae91eef/cache.tzst -P -C /home/runner/work/zigpokerhands/zigpokerhands --use-compress-program unzstd
2024-03-13T08:59:46.6897601Z Received 68872401 of 68872401 (100.0%), 65.6 MBs/sec
2024-03-13T08:59:47.6632481Z Cache restored successfully
2024-03-13T08:59:47.6776017Z using cached zig install: /opt/hostedtoolcache/zig/master/x64
2024-03-13T08:59:47.6780982Z zig installed at /opt/hostedtoolcache/zig/master/x64
goto-bus-stop commented 6 months ago

Thanks for the input. I brought it up because i know setup-zig is used by many C projects for zig's convenient C (cross-)compilation. I'm giving this repo some attention today so this should finally be addressed.

@kristoff-it since you are on the zig team ... do you think the ziglang github org or just an individual closer to zig would be interested in taking over this repo? I don't really have the time to maintain it actively (obviously) and also not really the expertise, since I'm not using zig actively at the moment. It would be good to have someone more knowledgeable maintain it

elerch commented 6 months ago

@goto-bus-stop @kristoff-it I am not part of the zig team, but have a fair number of projects using zig now and would be willing to help. I have a fork started with support for nominated mach engines and I am fairly certain of the cause of the some of the issues cited here.