FilipAndersson245
commented
1 month ago The simplest first approach if this seem interesting would be to implement something that randomize the heap allocations. Emery describe a simple approch to that in their paper linked above.
there is also this implementation of it in rust, but it should probably be simpler to do in zig. This could probably be added next to TrackingAllocator
hendriknielaender
Is your feature request related to a problem? Please describe. In his talk "Performance Matters", Emery Berger talk about the importance of randomness when doing benchmarks, as a lot of small layout changes may produce large impacts its hard to find when it is an actual improvement. He proposes that if done repeatedly, a complete random layout cannot bias the result. He compares that doing multiple samples over the same layout can be seen as asking 1 person a question 30 times instead of asking 30 people 1 question.
Describe the solution you'd like Emery Berger brings up these main points of where layout can be randomized to help remove biases.
I do not know how possible all these are in zig, but implementing some of them may help improve benchmark accuracy.
Describe alternatives you've considered We do not do this and ignore layout optimizations in the code.
Additional context https://youtu.be/r-TLSBdHe1A?t=950 https://people.cs.umass.edu/%7Eemery/pubs/stabilizer-asplos13.pdf