The latest run of the backfill script seems to be reporting much lower wall clock/cpu instruction count/etc results for all the commits included in its run (the oldest commit included in the backfill was commit 0c1df96b17fcafdb0aa973dfcb1819929719d650, which is exactly where this performance difference is reported).
This is not a real performance difference, though, as the commit that is reported as the cause is unrelated to any of the current benchmarks. It also only affects certain benchmarks--some don't have any performance difference at this commit.
Really unsure what is causing this, but have noticed one thing:
Every affected benchmark (zig-fmt, ast-check-os, comptime-guid-parse, etc) uses a child process for their benchmark (that is, they call useChildProcess() in their setup fn). Any benchmark that doesn't use a child process does not have the same erroneous performance difference.
The latest run of the backfill script seems to be reporting much lower wall clock/cpu instruction count/etc results for all the commits included in its run (the oldest commit included in the backfill was commit
0c1df96b17fcafdb0aa973dfcb1819929719d650, which is exactly where this performance difference is reported).This is not a real performance difference, though, as the commit that is reported as the cause is unrelated to any of the current benchmarks. It also only affects certain benchmarks--some don't have any performance difference at this commit.
Really unsure what is causing this, but have noticed one thing:
zig-fmt,ast-check-os,comptime-guid-parse, etc) uses a child process for their benchmark (that is, they calluseChildProcess()in their setup fn). Any benchmark that doesn't use a child process does not have the same erroneous performance difference.