Closed ghuls closed 2 weeks ago
musl introduced a new malloc implementation called mallocng
in 1.2.0.
Both the oldmalloc
and mallocng
are designed for embedded usage, and have bad performance for multi-thread computing.
You can build and link those malloc implementation as a replacement: https://github.com/microsoft/mimalloc https://github.com/microsoft/snmalloc https://github.com/mjansson/rpmalloc https://github.com/jemalloc/jemalloc
Cluster-Buster is a single threaded program.
Here's the plan to address this:
Zig Version
0.10.0-dev.750+cf5009f9a
Steps to Reproduce
Create input files:
Run Cluster-Buster binaries for each compiled version (with different zig versions) and check execution time.
Expected Behavior
I expect that newer zig versions are faster (or equal in speed) than older ones.
I had a compiled Cluster-Buster version that I compiled 2020-11-05 and this is the fastest version of Cluster-Buster I ever managed to compile. I didn't know with which version and settings ( it was compiled (march, tune) with) but I remembered it was a development version of zig at the time. I never managed to create a faster or same speed binary with any zig version afterwards (0.7.0 => current).
Today I tried to find which zig version it could have been and I managed to find it.
I tracked down the commit that introduced the slowdown: https://github.com/ziglang/zig/commits/master?after=7f91be9c80f8d79e4a2c6cf0b15197cdd454cef6+6335&branch=master
Slow update musl sources to 1.2.1 @andrewrk andrewrk committed on Nov 6, 2020
update musl headers to 1.2.1 @andrewrk andrewrk committed on Nov 6, 2020
update process_headers tool @andrewrk andrewrk committed on Nov 6, 2020
As fast as zig 0.6.0+17837affd std: Introduce SemanticVersion data structure @jayschwa @andrewrk jayschwa authored and andrewrk committed on Nov 6, 2020
Something in the musl lib update seems to cause the slow down.
Actual Behavior
When running all binaries with perf, it seems like the musl update causes a huge number of pagefaults and the sytem time goes to 1.1 second vs to less than 0.1 for the fast binary (zig0.6.0+030f003).