makslevental
commented
1 year ago @janisozaur god bless you for figuring this out.
Closed janisozaur closed 1 year ago
makslevental
commented
1 year ago @janisozaur god bless you for figuring this out.
hendrikmuhs
commented
1 year ago Sorry for the late reply.
This sounds to me like an issue of homebrew. For verification can someone with a mac try the following:
mtime of the installed ccache binary (ls -l)mtime has changedmakslevental
commented
1 year ago @hendrikmuhs
(base) mlevental@maksims-MacBook-Pro mlir-python-utils-main % ls -lL $(which ccache)
-rwxr-xr-x 1 mlevental admin 1264576 Aug 29 14:41 /opt/homebrew/bin/ccache
(base) mlevental@maksims-MacBook-Pro mlir-python-utils-main % brew uninstall ccache
Uninstalling /opt/homebrew/Cellar/ccache/4.8.3... (78 files, 1.3MB)
(base) mlevental@maksims-MacBook-Pro mlir-python-utils-main % brew cleanup --prune=1 -s
<LOTS OF THINGS REMOVED>
==> This operation has freed approximately 1.5GB of disk space.
(base) mlevental@maksims-MacBook-Pro mlir-python-utils-main % brew install ccache
==> Downloading https://formulae.brew.sh/api/formula.jws.json
##O=- # #
==> Downloading https://ghcr.io/v2/homebrew/core/ccache/manifests/4.8.3
(base) mlevental@maksims-MacBook-Pro mlir-python-utils-main % ls -lL $(which ccache)
-rwxr-xr-x 1 mlevental admin 1264576 Aug 29 14:41 /opt/homebrew/bin/ccacheI'll say that @janisozaur's solution does work for me but in a hit-or-miss fashion; e.g., you can see here that all the mac builds had high hit rates (because cibuildwheel took ~30 minutes instead of 3 hours) but in a following run, with the same initial conditions, some of the mac jobs didn't hit and some did. Note, I'm not expecting you to dig into my CI, just stating anecdote. And I'm not a ccache expert so maybe there's something else subtly changing betweens my builds but my linux runners consistently hit (with appropriate initial conditions).
hendrikmuhs
commented
1 year ago Thanks @makslevental!
Actually I was still in vacation mode, it is the compiler's mtime that for some reason mismatches, not the one from the ccache binary.
Anyway, I think we should merge the fix. For later it would be nice if the setting can be overwritten in the action.
makslevental
commented
1 year ago Thanks @makslevental!
Actually I was still in vacation mode, it is the compiler's
mtimethat for some reason mismatches, not the one from the ccache binary.Anyway, I think we should merge the fix. For later it would be nice if the setting can be overwritten in the action.
I must sorely need a vacation because I fully understood the patch and still followed your vacation mode instructions like a 🤖. Lol
When executing jobs on macOS, compiler's mtime is different on different workers (or even for individual jobs?), causing cache misses as it is one of elements for key calculation.
The compiler size and contents are actually the same, so ccache can still be effective in those cases, but requires setting
I have verified this to be correct with ccache debugging enabled. Additionally, I have verified setting the option mentioned above made ccache hit all the TUs.
Below is diff from two builds without changing sources (formatted to remove timestamps):
And the logs themselves (only from one run) Audio.cpp.o.20230504_155944_795167.ccache-input-text.txt Audio.cpp.o.20230504_155944_795167.ccache-log.txt