search

serpent-os / moss

The safe, fast and sane package manager for Linux
https://serpentos.com
71 stars 9 forks source link

all: Switch to deterministic BTreeMap/BTreeSet #235

Closed ReillyBrogan closed 3 weeks ago

ReillyBrogan commented 1 month ago

While testing ccache I noticed that cache hit rate was extremely low. After some debugging I noticed that the CFLAGS and other environmental variables and macro scripts were not being set deterministically. This is problematic for ccache as it hashes the compiler arguments as part of the key and the order matters to get the same key on a future run.

The cause of this non-determinism is the Rust stdlib HashMap and HashSet, for which the default hasher used a random seed. To solve this switch to BTreeMap and BTreeSet from the stdlib. Since non-determinism is just generally a bad idea for build tools and package management software replace ALL uses of HashMap and HashSet repo-wide.

Tested by building a package with ccache enabled and using ccache --show-stats in the package recipe to verify that the hit rate was now 100% and that the build stage time had dropped to just the time it spent linking.

ermo commented 1 month ago

@tarkah Your input on this would be appreciated. I don't see any reason why it shouldn't go in, but maybe I'm missing something...?

tarkah commented 1 month ago

This sounds good in theory. Let me dig through each change tomorrow to confirm there's no sneaky regressions due to change from Hash/Eq to Ord

ermo commented 1 month ago

This sounds good in theory. Let me dig through each change tomorrow to confirm there's no sneaky regressions due to change from Hash/Eq to Ord

Any luck with this... @tarkah ?