Closed LightAndLight closed 1 year ago
I compared the two versions of uploadToCache
with no environment variables set. callgrind
's instruction count reported ~2 million for ipso and ~7.7 million for bash.
The callgrind
report shows that bash spends most of its time (~6.4 million instructions) inside mblen
, which seems to parse some locale files (/nix/store/scd5n7xsn0hh0lvhhnycr9gx0h8xfzsl-glibc-2.34-210/lib/gconv/gconv-modules.d/gconv-modules-extra.conf
). The slow bash function is here.
If we ignore the cost of set_default_locale
, we get ~1.3 million instructions for the rest of the bash program.
When running with OUT_PATHS="aaa"
, I get ~2.5 million instructions for ipso and ~9.9 million instructions for bash.
set_default_locale
still dominates bash's execution, at ~6.4 million instructions. Subtracting this we get ~3.5 million instructions for bash.
For reference, the Python version has ~89 million instructions:
#! /usr/bin/env python3
import os
import subprocess
BINARY_CACHE_BUCKET = "ipso-binary-cache"
BINARY_CACHE_ENDPOINT = "7065dc7f7d1813a29036535b4c4f4014.r2.cloudflarestorage.com"
OUT_PATHS = os.environ["OUT_PATHS"]
print("Uploading paths", OUT_PATHS)
subprocess.run([
"nix",
"copy",
"--to",
f"s3://{BINARY_CACHE_BUCKET}?scheme=https&endpoint={BINARY_CACHE_ENDPOINT}&secret-key=/run/nix-signing-key",
*OUT_PATHS.split(' ')
],
check=True
)
Blocked until I can find a nice way to provide ipso
to the post-build-hook
script. The Nix daemon runs the script and doesn't have easy access to a Nix-installed version of ipso
. Example failure: https://github.com/LightAndLight/ipso/actions/runs/3501094293/jobs/5864384130#step:9:88.
After using pueue
to queue Nix cache uploads (https://github.com/LightAndLight/ipso/pull/291), it was easy to migrate the scripts. I installed ipso
in the runner's user, and because pueue
is also running in the runner's user, it has access to the Nix-installed program.
Branch: https://github.com/LightAndLight/ipso/tree/271-translate-uploadtocachemacos-to-ipso
Files:
Tasks