Closed carlocab closed 3 years ago
carlocab
commented
3 years ago The error message seems to come from here:
and it might be because the non-x86_64/i386 code is Linux-specific.
However, the code path that runs on Intel macOS doesn't seem too large, so I wonder how difficult it would be to port this to Apple Silicon:
RyanGlScott
commented
3 years ago Indeed, criterion-measurement currently does not build on AArch64 (see also #80). I own neither an AArch64 nor Darwin machine, so I can only speculate on what the fix should be. I doubt that rdtsc will work on AArch64, so we'll likely need a different approach there. Some searching reveals this potential solution, which claims to work on macOS regardless of architecture. Would you be willing to try something similar in cycles.c and see if the criterion test suite gives sensible-looking numbers with it? I'm thinking something like:
diff --git a/criterion-measurement/cbits/cycles.c b/criterion-measurement/cbits/cycles.c
index 73ba7f9..6767878 100644
--- a/criterion-measurement/cbits/cycles.c
+++ b/criterion-measurement/cbits/cycles.c
@@ -1,6 +1,15 @@
#include "Rts.h"
-#if x86_64_HOST_ARCH || i386_HOST_ARCH
+#if darwin_HOST_OS
+
+#include <mach/mach_time.h>
+
+StgWord64 criterion_rdtsc(void)
+{
+ return mach_absolute_time();
+}
+
+#elif x86_64_HOST_ARCH || i386_HOST_ARCH
StgWord64 criterion_rdtsc(void)
{I don't have an AArch64 machine, but I can certainly test this on an Intel one. Is there a unit test for this code fragment that I can run, or some other way to test this patch? Or will successful compilation be enough?
carlocab
commented
3 years ago Seems to build fine with your patch after replacing darwin_HOST_OS with __APPLE__. darwin_HOST_OS wasn't defined when I tried it for some reason.
RyanGlScott
commented
3 years ago Seems to build fine with your patch after replacing
darwin_HOST_OSwith__APPLE__.darwin_HOST_OSwasn't defined when I tried it for some reason.
That's really strange, since all of the macros used in cycles.c (x86_64_HOST_ARCH, i386_HOST_ARCH, and linux_HOST_OS) are provided by GHC itself, per this section of the User's Guide. Moreover, other libraries seem to be using darwin_HOST_OS without issue, such as base.
I suppose I'm glad that __APPLE__ works as a backup, but I'd prefer to use the official GHC macros if at all possible. One other thing to check: does it work if you write #if defined(darwin_HOST_OS) instead of #if darwin_HOST_OS? It's possible that darwin_HOST_OS actually is defined but not set to a "truthy" value, which would cause #if darwin_HOST_OS ... not to be invoked.
Is there a unit test for this code fragment that I can run, or some other way to test this patch?
Can you run cabal test test:sanity --test-show-details=direct on criterion? That will run a simple benchmark suite and output some figures whose timing ought to depend on what criterion_rdtsc returns. Here is what I get on my machine, for instance:
$ cabal test test:sanity --test-show-details=direct
Resolving dependencies...
Build profile: -w ghc-9.0.1 -O1
In order, the following will be built (use -v for more details):
- criterion-1.5.8.0 (test:sanity) (first run)
Preprocessing test suite 'sanity' for criterion-1.5.8.0..
Building test suite 'sanity' for criterion-1.5.8.0..
Running 1 test suites...
Test suite sanity: RUNNING...
sanity: benchmarking fib/fib 10
time 2.302 μs (2.279 μs .. 2.323 μs)
1.000 R² (0.999 R² .. 1.000 R²)
mean 2.300 μs (2.285 μs .. 2.313 μs)
std dev 47.22 ns (39.04 ns .. 53.08 ns)
variance introduced by outliers: 23% (moderately inflated)
benchmarking fib/fib 22
time 1.467 ms (1.457 ms .. 1.479 ms)
1.000 R² (1.000 R² .. 1.000 R²)
mean 1.472 ms (1.467 ms .. 1.475 ms)
std dev 14.32 μs (11.78 μs .. 16.19 μs)
benchmarking length . filter/string
time 2.729 μs (2.728 μs .. 2.730 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 2.729 μs (2.729 μs .. 2.730 μs)
std dev 2.859 ns (2.356 ns .. 3.541 ns)
benchmarking length . filter/bytestring
time 547.4 ns (547.2 ns .. 547.6 ns)
1.000 R² (1.000 R² .. 1.000 R²)
mean 547.1 ns (547.1 ns .. 547.3 ns)
std dev 368.9 ps (254.0 ps .. 647.8 ps)
OK (20.75s)
All 1 tests passed (20.75s)
Test suite sanity: PASS
Test suite logged to:
/home/ryanglscott/Documents/Hacking/Haskell/criterion/dist-newstyle/build/x86_64-linux/ghc-9.0.1/criterion-1.5.8.0/t/sanity/test/criterion-1.5.8.0-sanity.log
1 of 1 test suites (1 of 1 test cases) passed.Your numbers will be different, of course, but I just want to make sure it's in the same general area as the numbers above.
csstaub
commented
3 years ago I tried the patch above on an Apple Silicon machine, but it timed out for me:
sanity: benchmarking fib/fib 10
FAIL
Exception: HUnitFailure (Just (SrcLoc {srcLocPackage = "main", srcLocModule = "Main", srcLocFile = "tests/Sanity.hs", srcLocStartLine = 50, srcLocStartCol = 18, srcLocEndLine = 51, srcLocEndCol = 61})) (Reason "killed for running longer than 30 seconds!")
1 out of 1 tests failed (30.00s)I also tried it using an (adapted) code snippet from here and got the same error: https://stackoverflow.com/questions/40454157/is-there-an-equivalent-instruction-to-rdtsc-in-arm/40455065
StgWord64 criterion_rdtsc(void)
{
StgWord64 val;
/*
* According to ARM DDI 0487F.c, from Armv8.0 to Armv8.5 inclusive, the
* system counter is at least 56 bits wide; from Armv8.6, the counter
* must be 64 bits wide. So the system counter could be less than 64
* bits wide and it is attributed with the flag 'cap_user_time_short'
* is true.
*/
__asm__ __volatile__ ("mrs %0, cntvct_el0" : "=r" (val));
return val;
}Interesting. It's not clear to me if the timeout is due to criterion_rdtsc or due to something else in criterion. If you run cabal test test:sanity --enable-profiling --test-options="+RTS -p", which cost centres does sanity.prof show as taking up the most time?
csstaub
commented
3 years ago Here's the contents of the sanity.prof file:
Sun Jun 27 12:49 2021 Time and Allocation Profiling Report (Final)
sanity +RTS -p -RTS
total time = 23.55 secs (23549 ticks @ 1000 us, 1 processor)
total alloc = 223,527,537,904 bytes (excludes profiling overheads)
COST CENTRE MODULE SRC %time %alloc
fib Main tests/Sanity.hs:(20,1)-(23,35) 99.9 100.0
individual inherited
COST CENTRE MODULE SRC no. entries %time %alloc %time %alloc
MAIN MAIN <built-in> 513 0 0.0 0.0 100.0 100.0
CAF Main <entire-module> 1025 0 0.0 0.0 0.0 0.0
fib Main tests/Sanity.hs:(20,1)-(23,35) 1454 1 0.0 0.0 0.0 0.0
getArgEnv Main tests/Sanity.hs:(59,1)-(61,38) 1240 1 0.0 0.0 0.0 0.0
main Main tests/Sanity.hs:54:1-45 1026 1 0.0 0.0 0.0 0.0
testCase Test.Tasty.HUnit Test/Tasty/HUnit.hs:81:1-62 1054 1 0.0 0.0 0.0 0.0
singleTest Test.Tasty.Providers Test/Tasty/Providers.hs:20:1-23 1055 1 0.0 0.0 0.0 0.0
defaultMain Test.Tasty Test/Tasty.hs:101:1-59 1028 0 0.0 0.0 0.0 0.0
defaultMainWithIngredients Test.Tasty.CmdLine Test/Tasty/CmdLine.hs:(170,1)-(181,45) 1029 1 0.0 0.0 0.0 0.0
sanity Main tests/Sanity.hs:(31,1)-(51,60) 1243 0 0.0 0.0 0.0 0.0
whnf Criterion.Measurement.Types src/Criterion/Measurement/Types.hs:277:1-38 1452 1 0.0 0.0 0.0 0.0
bench Criterion.Measurement.Types src/Criterion/Measurement/Types.hs:591:1-17 1442 0 0.0 0.0 0.0 0.0
bgroup Criterion.Measurement.Types src/Criterion/Measurement/Types.hs:597:1-19 1440 0 0.0 0.0 0.0 0.0
defaultMain Criterion.Main Criterion/Main.hs:95:1-43 1245 0 0.0 0.0 0.0 0.0
defaultMainWith Criterion.Main Criterion/Main.hs:(149,1)-(151,16) 1246 1 0.0 0.0 0.0 0.0
CAF Test.Tasty.HUnit <entire-module> 1024 0 0.0 0.0 0.0 0.0
CAF Test.Tasty.HUnit.Orig <entire-module> 1023 0 0.0 0.0 0.0 0.0
CAF Test.Tasty <entire-module> 1022 0 0.0 0.0 0.0 0.0
defaultIngredients Test.Tasty Test/Tasty.hs:75:1-56 1046 1 0.0 0.0 0.0 0.0
defaultMain Test.Tasty Test/Tasty.hs:101:1-59 1027 1 0.0 0.0 0.0 0.0
CAF Test.Tasty.Options <entire-module> 1021 0 0.0 0.0 0.0 0.0
uniqueOptionDescriptions Test.Tasty.Options Test/Tasty/Options.hs:(138,1)-(143,62) 1040 1 0.0 0.0 0.0 0.0
CAF Test.Tasty.Providers.ConsoleFormat <entire-module> 1019 0 0.0 0.0 0.0 0.0
noResultDetails Test.Tasty.Providers.ConsoleFormat Test/Tasty/Providers/ConsoleFormat.hs:41:1-66 1478 1 0.0 0.0 0.0 0.0
CAF Test.Tasty.Ingredients <entire-module> 1018 0 0.0 0.0 0.0 0.0
ingredientsOptions Test.Tasty.Ingredients Test/Tasty/Ingredients.hs:115:1-75 1043 1 0.0 0.0 0.0 0.0
CAF Test.Tasty.Ingredients.ConsoleReporter <entire-module> 1017 0 0.0 0.0 0.0 0.0
mkFlagCLParser Test.Tasty.Options Test/Tasty/Options.hs:(159,1)-(163,3) 1100 2 0.0 0.0 0.0 0.0
flag' Options.Applicative.Builder src/Options/Applicative/Builder.hs:(327,1)-(331,43) 1105 2 0.0 0.0 0.0 0.0
mkParser Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:(165,1)-(169,36) 1106 2 0.0 0.0 0.0 0.0
help Options.Applicative.Builder src/Options/Applicative/Builder.hs:190:1-55 1103 2 0.0 0.0 0.0 0.0
optionMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:130:1-25 1104 0 0.0 0.0 0.0 0.0
long Options.Applicative.Builder src/Options/Applicative/Builder.hs:166:1-32 1101 2 0.0 0.0 0.0 0.0
fieldMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:133:1-28 1102 2 0.0 0.0 0.0 0.0
mkOptionCLParser Test.Tasty.Options Test/Tasty/Options.hs:(167,1)-(176,74) 1109 2 0.0 0.0 0.0 0.0
help Options.Applicative.Builder src/Options/Applicative/Builder.hs:190:1-55 1112 2 0.0 0.0 0.0 0.0
optionMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:130:1-25 1113 0 0.0 0.0 0.0 0.0
long Options.Applicative.Builder src/Options/Applicative/Builder.hs:166:1-32 1110 2 0.0 0.0 0.0 0.0
fieldMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:133:1-28 1111 2 0.0 0.0 0.0 0.0
option Options.Applicative.Builder src/Options/Applicative/Builder.hs:(370,1)-(375,65) 1114 2 0.0 0.0 0.0 0.0
mkParser Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:(165,1)-(169,36) 1115 2 0.0 0.0 0.0 0.0
computeStatistics Test.Tasty.Ingredients.ConsoleReporter Test/Tasty/Ingredients/ConsoleReporter.hs:(291,1)-(293,30) 1479 1 0.0 0.0 0.0 0.0
consoleTestReporter Test.Tasty.Ingredients.ConsoleReporter Test/Tasty/Ingredients/ConsoleReporter.hs:(380,1)-(429,44) 1049 1 0.0 0.0 0.0 0.0
hSupportsANSI System.Console.ANSI.Unix src/System/Console/ANSI/Unix.hs:(74,1)-(77,65) 1214 1 0.0 0.0 0.0 0.0
hSupportsANSIColor System.Console.ANSI.Unix src/includes/Common-Include.hs:(144,1)-(148,49) 1222 1 0.0 0.0 0.0 0.0
lookupOption Test.Tasty.Options Test/Tasty/Options.hs:(115,1)-(119,27) 1213 0 0.0 0.0 0.0 0.0
metavar Options.Applicative.Builder src/Options/Applicative/Builder.hs:206:1-55 1107 1 0.0 0.0 0.0 0.0
optionMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:130:1-25 1108 0 0.0 0.0 0.0 0.0
short Options.Applicative.Builder src/Options/Applicative/Builder.hs:162:1-34 1098 1 0.0 0.0 0.0 0.0
fieldMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:133:1-28 1099 1 0.0 0.0 0.0 0.0
printStatistics Test.Tasty.Ingredients.ConsoleReporter Test/Tasty/Ingredients/ConsoleReporter.hs:(307,1)-(312,40) 1483 0 0.0 0.0 0.0 0.0
CAF Control.Concurrent.Async <entire-module> 1013 0 0.0 0.0 0.0 0.0
async Control.Concurrent.Async Control/Concurrent/Async.hs:69:1-35 1207 1 0.0 0.0 0.0 0.0
withAsync Control.Concurrent.Async Control/Concurrent/Async.hs:97:1-43 1219 1 0.0 0.0 0.0 0.0
CAF Test.Tasty.Core <entire-module> 1011 0 0.0 0.0 0.0 0.0
treeOptions Test.Tasty.Core Test/Tasty/Core.hs:(385,1)-(400,31) 1050 1 0.0 0.0 0.0 0.0
trivialFold Test.Tasty.Core Test/Tasty/Core.hs:(330,1)-(335,3) 1052 1 0.0 0.0 0.0 0.0
foldTestTree Test.Tasty.Core Test/Tasty/Core.hs:(364,1)-(381,46) 1058 0 0.0 0.0 0.0 0.0
lookupOption Test.Tasty.Options Test/Tasty/Options.hs:(115,1)-(119,27) 1059 0 0.0 0.0 0.0 0.0
CAF Test.Tasty.Options.Core <entire-module> 1010 0 0.0 0.0 0.0 0.0
metavar Options.Applicative.Builder src/Options/Applicative/Builder.hs:206:1-55 1080 2 0.0 0.0 0.0 0.0
optionMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:130:1-25 1081 0 0.0 0.0 0.0 0.0
mkOptionCLParser Test.Tasty.Options Test/Tasty/Options.hs:(167,1)-(176,74) 1082 2 0.0 0.0 0.0 0.0
help Options.Applicative.Builder src/Options/Applicative/Builder.hs:190:1-55 1085 2 0.0 0.0 0.0 0.0
optionMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:130:1-25 1086 0 0.0 0.0 0.0 0.0
long Options.Applicative.Builder src/Options/Applicative/Builder.hs:166:1-32 1083 2 0.0 0.0 0.0 0.0
fieldMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:133:1-28 1084 2 0.0 0.0 0.0 0.0
option Options.Applicative.Builder src/Options/Applicative/Builder.hs:(370,1)-(375,65) 1087 2 0.0 0.0 0.0 0.0
mkParser Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:(165,1)-(169,36) 1088 2 0.0 0.0 0.0 0.0
short Options.Applicative.Builder src/Options/Applicative/Builder.hs:162:1-34 1078 2 0.0 0.0 0.0 0.0
fieldMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:133:1-28 1079 2 0.0 0.0 0.0 0.0
coreOptions Test.Tasty.Options.Core Test/Tasty/Options/Core.hs:(93,1)-(96,3) 1042 1 0.0 0.0 0.0 0.0
CAF Test.Tasty.Options.Env <entire-module> 1009 0 0.0 0.0 0.0 0.0
getEnvOptions Test.Tasty.Options.Env Test/Tasty/Options/Env.hs:(38,1)-(56,57) 1174 1 0.0 0.0 0.0 0.0
CAF Test.Tasty.Patterns <entire-module> 1008 0 0.0 0.0 0.0 0.0
metavar Options.Applicative.Builder src/Options/Applicative/Builder.hs:206:1-55 1063 1 0.0 0.0 0.0 0.0
optionMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:130:1-25 1065 0 0.0 0.0 0.0 0.0
mkOptionCLParser Test.Tasty.Options Test/Tasty/Options.hs:(167,1)-(176,74) 1066 1 0.0 0.0 0.0 0.0
help Options.Applicative.Builder src/Options/Applicative/Builder.hs:190:1-55 1069 1 0.0 0.0 0.0 0.0
optionMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:130:1-25 1070 0 0.0 0.0 0.0 0.0
long Options.Applicative.Builder src/Options/Applicative/Builder.hs:166:1-32 1067 1 0.0 0.0 0.0 0.0
fieldMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:133:1-28 1068 1 0.0 0.0 0.0 0.0
option Options.Applicative.Builder src/Options/Applicative/Builder.hs:(370,1)-(375,65) 1071 1 0.0 0.0 0.0 0.0
mkParser Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:(165,1)-(169,36) 1075 1 0.0 0.0 0.0 0.0
noPattern Test.Tasty.Patterns Test/Tasty/Patterns.hs:31:1-31 1060 1 0.0 0.0 0.0 0.0
short Options.Applicative.Builder src/Options/Applicative/Builder.hs:162:1-34 1061 1 0.0 0.0 0.0 0.0
fieldMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:133:1-28 1062 1 0.0 0.0 0.0 0.0
CAF Test.Tasty.Run <entire-module> 1006 0 0.0 0.0 0.0 0.0
launchTestTree Test.Tasty.Run Test/Tasty/Run.hs:(421,1)-(452,31) 1204 0 0.0 0.0 0.0 0.0
lookupOption Test.Tasty.Options Test/Tasty/Options.hs:(115,1)-(119,27) 1251 0 0.0 0.0 0.0 0.0
timed Test.Tasty.Runners.Utils Test/Tasty/Runners/Utils.hs:(99,1)-(103,23) 1205 0 0.0 0.0 0.0 0.0
lookupOption Test.Tasty.Options Test/Tasty/Options.hs:(115,1)-(119,27) 1206 0 0.0 0.0 0.0 0.0
CAF Test.Tasty.Runners.Utils <entire-module> 1004 0 0.0 0.0 0.0 0.0
forceElements Test.Tasty.Runners.Utils Test/Tasty/Runners/Utils.hs:57:1-28 1465 1 0.0 0.0 0.0 0.0
formatMessage Test.Tasty.Runners.Utils Test/Tasty/Runners/Utils.hs:(43,1)-(52,107) 1463 1 0.0 0.0 0.0 0.0
getTime Test.Tasty.Runners.Utils Test/Tasty/Runners/Utils.hs:(111,1)-(119,31) 1199 1 0.0 0.0 0.0 0.0
installSignalHandlers Test.Tasty.Runners.Utils Test/Tasty/Runners/Utils.hs:(71,1)-(82,67) 1033 1 0.0 0.0 0.0 0.0
CAF Test.Tasty.CmdLine <entire-module> 1003 0 0.0 0.0 0.0 0.0
optionParser Test.Tasty.CmdLine Test/Tasty/CmdLine.hs:(39,1)-(43,39) 1037 1 0.0 0.0 0.0 0.0
parseOptions Test.Tasty.CmdLine Test/Tasty/CmdLine.hs:(152,1)-(161,33) 1127 0 0.0 0.0 0.0 0.0
header Options.Applicative.Builder src/Options/Applicative/Builder.hs:398:1-57 1128 1 0.0 0.0 0.0 0.0
CAF Test.Tasty.Ingredients.ListTests <entire-module> 1002 0 0.0 0.0 0.0 0.0
listingTests Test.Tasty.Ingredients.ListTests Test/Tasty/Ingredients/ListTests.hs:(39,1)-(45,19) 1048 1 0.0 0.0 0.0 0.0
lookupOption Test.Tasty.Options Test/Tasty/Options.hs:(115,1)-(119,27) 1189 0 0.0 0.0 0.0 0.0
mkFlagCLParser Test.Tasty.Options Test/Tasty/Options.hs:(159,1)-(163,3) 1091 1 0.0 0.0 0.0 0.0
flag' Options.Applicative.Builder src/Options/Applicative/Builder.hs:(327,1)-(331,43) 1096 1 0.0 0.0 0.0 0.0
mkParser Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:(165,1)-(169,36) 1097 1 0.0 0.0 0.0 0.0
help Options.Applicative.Builder src/Options/Applicative/Builder.hs:190:1-55 1094 1 0.0 0.0 0.0 0.0
optionMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:130:1-25 1095 0 0.0 0.0 0.0 0.0
long Options.Applicative.Builder src/Options/Applicative/Builder.hs:166:1-32 1092 1 0.0 0.0 0.0 0.0
fieldMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:133:1-28 1093 1 0.0 0.0 0.0 0.0
short Options.Applicative.Builder src/Options/Applicative/Builder.hs:162:1-34 1089 1 0.0 0.0 0.0 0.0
fieldMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:133:1-28 1090 1 0.0 0.0 0.0 0.0
CAF Data.Char.WCWidth <entire-module> 1001 0 0.0 0.0 0.0 0.0
wcwidth Data.Char.WCWidth Data/Char/WCWidth.hs:69:1-69 1231 1 0.0 0.0 0.0 0.0
CAF System.Clock <entire-module> 998 0 0.0 0.0 0.0 0.0
toNanoSecs System.Clock System/Clock.hsc:262:1-73 1490 0 0.0 0.0 0.0 0.0
CAF Criterion.IO <entire-module> 996 0 0.0 0.0 0.0 0.0
critVersion Criterion.IO Criterion/IO.hs:60:1-64 1437 1 0.0 0.0 0.0 0.0
headerRoot Criterion.IO Criterion/IO.hs:55:1-24 1436 1 0.0 0.0 0.0 0.0
CAF Criterion.IO.Printf <entire-module> 995 0 0.0 0.0 0.0 0.0
writeCsv Criterion.IO.Printf Criterion/IO/Printf.hs:(99,1)-(102,49) 1418 0 0.0 0.0 0.0 0.0
encDelimiter Data.Csv.Encoding src/Data/Csv/Encoding.hs:202:7-18 1419 2 0.0 0.0 0.0 0.0
encQuoting Data.Csv.Encoding src/Data/Csv/Encoding.hs:213:7-16 1427 1 0.0 0.0 0.0 0.0
encUseCrLf Data.Csv.Encoding src/Data/Csv/Encoding.hs:206:7-16 1429 1 0.0 0.0 0.0 0.0
recordSep Data.Csv.Encoding src/Data/Csv/Encoding.hs:(342,1)-(343,32) 1428 1 0.0 0.0 0.0 0.0
CAF Criterion.Internal <entire-module> 994 0 0.0 0.0 0.0 0.0
runAndAnalyse Criterion.Internal Criterion/Internal.hs:(119,1)-(157,12) 1435 0 0.0 0.0 0.0 0.0
CAF Criterion.Main <entire-module> 993 0 0.0 0.0 0.0 0.0
defaultMain Criterion.Main Criterion/Main.hs:95:1-43 1244 1 0.0 0.0 0.0 0.0
runMode Criterion.Main Criterion/Main.hs:(158,1)-(173,34) 1412 0 0.0 0.0 0.0 0.0
writeCsv Criterion.IO.Printf Criterion/IO/Printf.hs:(99,1)-(102,49) 1413 1 0.0 0.0 0.0 0.0
CAF Criterion.Main.Options <entire-module> 992 0 0.0 0.0 0.0 0.0
defaultConfig Criterion.Main.Options Criterion/Main/Options.hs:(79,1)-(91,5) 1308 1 0.0 0.0 0.0 0.0
metavar Options.Applicative.Builder src/Options/Applicative/Builder.hs:206:1-55 1341 1 0.0 0.0 0.0 0.0
optionMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:130:1-25 1342 0 0.0 0.0 0.0 0.0
config Criterion.Main.Options Criterion/Main/Options.hs:(126,1)-(155,51) 1310 0 0.0 0.0 0.0 0.0
help Options.Applicative.Builder src/Options/Applicative/Builder.hs:190:1-55 1318 11 0.0 0.0 0.0 0.0
optionMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:130:1-25 1319 0 0.0 0.0 0.0 0.0
long Options.Applicative.Builder src/Options/Applicative/Builder.hs:166:1-32 1311 11 0.0 0.0 0.0 0.0
fieldMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:133:1-28 1312 11 0.0 0.0 0.0 0.0
metavar Options.Applicative.Builder src/Options/Applicative/Builder.hs:206:1-55 1315 6 0.0 0.0 0.0 0.0
optionMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:130:1-25 1316 0 0.0 0.0 0.0 0.0
short Options.Applicative.Builder src/Options/Applicative/Builder.hs:162:1-34 1313 5 0.0 0.0 0.0 0.0
fieldMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:133:1-28 1314 5 0.0 0.0 0.0 0.0
fromM Options.Applicative.Types src/Options/Applicative/Types.hs:287:1-26 1327 1 0.0 0.0 0.0 0.0
manyM Options.Applicative.Types src/Options/Applicative/Types.hs:(293,1)-(297,30) 1328 1 0.0 0.0 0.0 0.0
oneM Options.Applicative.Types src/Options/Applicative/Types.hs:290:1-26 1329 1 0.0 0.0 0.0 0.0
option Options.Applicative.Builder src/Options/Applicative/Builder.hs:(370,1)-(375,65) 1330 1 0.0 0.0 0.0 0.0
mkParser Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:(165,1)-(169,36) 1331 1 0.0 0.0 0.0 0.0
mkOption Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:175:1-43 1332 1 0.0 0.0 0.0 0.0
optNames Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:36:5-12 1333 1 0.0 0.0 0.0 0.0
long Options.Applicative.Builder src/Options/Applicative/Builder.hs:166:1-32 1334 0 0.0 0.0 0.0 0.0
fieldMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:133:1-28 1335 0 0.0 0.0 0.0 0.0
optNames Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:36:5-12 1336 1 0.0 0.0 0.0 0.0
describeWith Criterion.Main.Options Criterion/Main/Options.hs:(204,1)-(207,38) 1270 0 0.0 0.0 0.0 0.0
footerDoc Options.Applicative.Builder src/Options/Applicative/Builder.hs:412:1-60 1271 1 0.0 0.0 0.0 0.0
header Options.Applicative.Builder src/Options/Applicative/Builder.hs:398:1-57 1274 1 0.0 0.0 0.0 0.0
parseWith Criterion.Main.Options Criterion/Main/Options.hs:(98,1)-(122,49) 1374 0 0.0 0.0 0.0 0.0
metavar Options.Applicative.Builder src/Options/Applicative/Builder.hs:206:1-55 1379 2 0.0 0.0 0.0 0.0
optionMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:130:1-25 1380 0 0.0 0.0 0.0 0.0
argument Options.Applicative.Builder src/Options/Applicative/Builder.hs:(285,1)-(289,25) 1389 1 0.0 0.0 0.0 0.0
mkParser Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:(165,1)-(169,36) 1390 1 0.0 0.0 0.0 0.0
fromM Options.Applicative.Types src/Options/Applicative/Types.hs:287:1-26 1386 1 0.0 0.0 0.0 0.0
manyM Options.Applicative.Types src/Options/Applicative/Types.hs:(293,1)-(297,30) 1387 1 0.0 0.0 0.0 0.0
oneM Options.Applicative.Types src/Options/Applicative/Types.hs:290:1-26 1388 1 0.0 0.0 0.0 0.0
help Options.Applicative.Builder src/Options/Applicative/Builder.hs:190:1-55 1382 1 0.0 0.0 0.0 0.0
optionMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:130:1-25 1383 0 0.0 0.0 0.0 0.0
long Options.Applicative.Builder src/Options/Applicative/Builder.hs:166:1-32 1375 1 0.0 0.0 0.0 0.0
fieldMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:133:1-28 1376 1 0.0 0.0 0.0 0.0
option Options.Applicative.Builder src/Options/Applicative/Builder.hs:(370,1)-(375,65) 1384 1 0.0 0.0 0.0 0.0
mkParser Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:(165,1)-(169,36) 1385 1 0.0 0.0 0.0 0.0
short Options.Applicative.Builder src/Options/Applicative/Builder.hs:162:1-34 1377 1 0.0 0.0 0.0 0.0
fieldMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:133:1-28 1378 1 0.0 0.0 0.0 0.0
value Options.Applicative.Builder src/Options/Applicative/Builder.hs:178:1-50 1381 1 0.0 0.0 0.0 0.0
CAF Paths_criterion <entire-module> 987 0 0.0 0.0 0.0 0.0
version Paths_criterion /Users/cs/Development/criterion/dist-newstyle/build/aarch64-osx/ghc-8.10.5/criterion-1.5.9.0/build/autogen/Paths_criterion.hs:30:1-30 1438 1 0.0 0.0 0.0 0.0
CAF Options.Applicative.BashCompletion <entire-module> 961 0 0.0 0.0 0.0 0.0
bashCompletionParser Options.Applicative.BashCompletion src/Options/Applicative/BashCompletion.hs:(35,1)-(65,7) 1133 0 0.0 0.0 0.0 0.0
long Options.Applicative.Builder src/Options/Applicative/Builder.hs:166:1-32 1134 8 0.0 0.0 0.0 0.0
fieldMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:133:1-28 1135 8 0.0 0.0 0.0 0.0
strOption Options.Applicative.Builder src/Options/Applicative/Builder.hs:360:1-22 1143 4 0.0 0.0 0.0 0.0
option Options.Applicative.Builder src/Options/Applicative/Builder.hs:(370,1)-(375,65) 1144 4 0.0 0.0 0.0 0.0
mkParser Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:(165,1)-(169,36) 1145 4 0.0 0.0 0.0 0.0
mkOption Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:175:1-43 1294 4 0.0 0.0 0.0 0.0
optNames Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:36:5-12 1295 4 0.0 0.0 0.0 0.0
long Options.Applicative.Builder src/Options/Applicative/Builder.hs:166:1-32 1296 0 0.0 0.0 0.0 0.0
fieldMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:133:1-28 1297 0 0.0 0.0 0.0 0.0
optNames Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:36:5-12 1298 4 0.0 0.0 0.0 0.0
option Options.Applicative.Builder src/Options/Applicative/Builder.hs:(370,1)-(375,65) 1150 3 0.0 0.0 0.0 0.0
mkParser Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:(165,1)-(169,36) 1151 3 0.0 0.0 0.0 0.0
mkOption Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:175:1-43 1289 3 0.0 0.0 0.0 0.0
optNames Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:36:5-12 1290 3 0.0 0.0 0.0 0.0
long Options.Applicative.Builder src/Options/Applicative/Builder.hs:166:1-32 1291 0 0.0 0.0 0.0 0.0
fieldMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:133:1-28 1292 0 0.0 0.0 0.0 0.0
optNames Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:36:5-12 1293 3 0.0 0.0 0.0 0.0
value Options.Applicative.Builder src/Options/Applicative/Builder.hs:178:1-50 1288 2 0.0 0.0 0.0 0.0
flag' Options.Applicative.Builder src/Options/Applicative/Builder.hs:(327,1)-(331,43) 1138 1 0.0 0.0 0.0 0.0
flagActive Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:42:5-14 1282 1 0.0 0.0 0.0 0.0
flagNames Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:41:5-13 1286 1 0.0 0.0 0.0 0.0
mkParser Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:(165,1)-(169,36) 1139 1 0.0 0.0 0.0 0.0
mkOption Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:175:1-43 1281 1 0.0 0.0 0.0 0.0
long Options.Applicative.Builder src/Options/Applicative/Builder.hs:166:1-32 1284 0 0.0 0.0 0.0 0.0
fieldMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:133:1-28 1285 0 0.0 0.0 0.0 0.0
flagNames Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:41:5-13 1287 1 0.0 0.0 0.0 0.0
fromM Options.Applicative.Types src/Options/Applicative/Types.hs:287:1-26 1140 1 0.0 0.0 0.0 0.0
manyM Options.Applicative.Types src/Options/Applicative/Types.hs:(293,1)-(297,30) 1141 1 0.0 0.0 0.0 0.0
oneM Options.Applicative.Types src/Options/Applicative/Types.hs:290:1-26 1142 1 0.0 0.0 0.0 0.0
CAF Options.Applicative.Builder <entire-module> 960 0 0.0 0.0 0.0 0.0
defaultPrefs Options.Applicative.Builder src/Options/Applicative/Builder.hs:547:1-24 1261 1 0.0 0.0 0.0 0.0
idm Options.Applicative.Builder src/Options/Applicative/Builder.hs:543:1-12 1263 1 0.0 0.0 0.0 0.0
prefs Options.Applicative.Builder src/Options/Applicative/Builder.hs:(527,1)-(537,36) 1262 1 0.0 0.0 0.0 0.0
fullDesc Options.Applicative.Builder src/Options/Applicative/Builder.hs:390:1-52 1130 1 0.0 0.0 0.0 0.0
hidden Options.Applicative.Builder src/Options/Applicative/Builder.hs:(212,1)-(213,54) 1167 1 0.0 0.0 0.0 0.0
optionMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:130:1-25 1168 0 0.0 0.0 0.0 0.0
option Options.Applicative.Builder src/Options/Applicative/Builder.hs:(370,1)-(375,65) 1072 0 0.0 0.0 0.0 0.0
metavar Options.Applicative.Builder src/Options/Applicative/Builder.hs:206:1-55 1073 1 0.0 0.0 0.0 0.0
optionMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:130:1-25 1074 0 0.0 0.0 0.0 0.0
CAF Options.Applicative.Builder.Internal <entire-module> 958 0 0.0 0.0 0.0 0.0
internal Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:189:1-60 1136 1 0.0 0.0 0.0 0.0
optionMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:130:1-25 1137 0 0.0 0.0 0.0 0.0
noGlobal Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:194:1-59 1163 1 0.0 0.0 0.0 0.0
optionMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:130:1-25 1164 0 0.0 0.0 0.0 0.0
optionMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:130:1-25 1064 1 0.0 0.0 0.0 0.0
CAF Options.Applicative.Common <entire-module> 957 0 0.0 0.0 0.0 0.0
liftOpt Options.Applicative.Common src/Options/Applicative/Common.hs:81:1-14 1076 1 0.0 0.0 0.0 0.0
CAF Options.Applicative.Extra <entire-module> 956 0 0.0 0.0 0.0 0.0
execParser Options.Applicative.Extra src/Options/Applicative/Extra.hs:84:1-42 1116 1 0.0 0.0 0.0 0.0
helper Options.Applicative.Extra src/Options/Applicative/Extra.hs:(49,1)-(65,44) 1153 1 0.0 0.0 0.0 0.0
help Options.Applicative.Builder src/Options/Applicative/Builder.hs:190:1-55 1158 1 0.0 0.0 0.0 0.0
optionMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:130:1-25 1159 0 0.0 0.0 0.0 0.0
long Options.Applicative.Builder src/Options/Applicative/Builder.hs:166:1-32 1154 1 0.0 0.0 0.0 0.0
fieldMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:133:1-28 1155 1 0.0 0.0 0.0 0.0
metavar Options.Applicative.Builder src/Options/Applicative/Builder.hs:206:1-55 1161 1 0.0 0.0 0.0 0.0
optionMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:130:1-25 1162 0 0.0 0.0 0.0 0.0
noArgError Options.Applicative.Builder src/Options/Applicative/Builder.hs:199:1-61 1165 1 0.0 0.0 0.0 0.0
fieldMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:133:1-28 1166 1 0.0 0.0 0.0 0.0
option Options.Applicative.Builder src/Options/Applicative/Builder.hs:(370,1)-(375,65) 1169 1 0.0 0.0 0.0 0.0
optNames Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:36:5-12 1306 3 0.0 0.0 0.0 0.0
mkParser Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:(165,1)-(169,36) 1170 1 0.0 0.0 0.0 0.0
mkOption Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:175:1-43 1305 1 0.0 0.0 0.0 0.0
short Options.Applicative.Builder src/Options/Applicative/Builder.hs:162:1-34 1156 1 0.0 0.0 0.0 0.0
fieldMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:133:1-28 1157 1 0.0 0.0 0.0 0.0
value Options.Applicative.Builder src/Options/Applicative/Builder.hs:178:1-50 1160 1 0.0 0.0 0.0 0.0
CAF Options.Applicative.Types <entire-module> 950 0 0.0 0.0 0.0 0.0
readerAsk Options.Applicative.Types src/Options/Applicative/Types.hs:219:1-21 1362 1 0.0 0.0 0.0 0.0
CAF Criterion.Measurement <entire-module> 922 0 0.0 0.0 0.0 0.0
getGCStatistics Criterion.Measurement src/Criterion/Measurement.hs:(124,1)-(151,43) 1457 1 0.0 0.0 0.0 0.0
measured Criterion.Measurement src/Criterion/Measurement.hs:(330,1)-(343,22) 1459 1 0.0 0.0 0.0 0.0
runBenchmark Criterion.Measurement src/Criterion/Measurement.hs:(292,1)-(316,41) 1456 0 0.0 0.0 0.0 0.0
CAF Criterion.Measurement.Types <entire-module> 921 0 0.0 0.0 0.0 0.0
bench Criterion.Measurement.Types src/Criterion/Measurement/Types.hs:591:1-17 1441 1 0.0 0.0 0.0 0.0
bgroup Criterion.Measurement.Types src/Criterion/Measurement/Types.hs:597:1-19 1439 1 0.0 0.0 0.0 0.0
CAF System.IO.CodePage <entire-module> 919 0 0.0 0.0 0.0 0.0
withCP65001 System.IO.CodePage src/System/IO/CodePage.hs:68:1-34 1401 1 0.0 0.0 0.0 0.0
withCodePage System.IO.CodePage src/System/IO/CodePage.hs:99:1-49 1402 1 0.0 0.0 0.0 0.0
CAF System.IO.CodePage.Internal <entire-module> 918 0 0.0 0.0 0.0 0.0
cp65001 System.IO.CodePage.Internal src/System/IO/CodePage/Internal.hs:36:1-15 1410 1 0.0 0.0 0.0 0.0
defaultOptions System.IO.CodePage.Internal src/System/IO/CodePage/Internal.hs:(78,1)-(82,3) 1406 1 0.0 0.0 0.0 0.0
nonWindowsFallbackCodePageEncoding System.IO.CodePage.Internal src/System/IO/CodePage/Internal.hs:101:1-71 1409 0 0.0 0.0 0.0 0.0
nonWindowsFallbackCodePageEncoding System.IO.CodePage.Internal src/System/IO/CodePage/Internal.hs:101:1-71 1408 1 0.0 0.0 0.0 0.0
CAF Data.Csv.Encoding <entire-module> 914 0 0.0 0.0 0.0 0.0
defaultEncodeOptions Data.Csv.Encoding src/Data/Csv/Encoding.hs:(218,1)-(223,5) 1420 1 0.0 0.0 0.0 0.0
CAF System.Console.ANSI.Unix <entire-module> 906 0 0.0 0.0 0.0 0.0
CAF Test.HUnit.Lang <entire-module> 684 0 0.0 0.0 0.0 0.0
CAF Data.Unique <entire-module> 641 0 0.0 0.0 0.0 0.0
CAF GHC.Conc.Signal <entire-module> 625 0 0.0 0.0 0.0 0.0
CAF GHC.Conc.Sync <entire-module> 624 0 0.0 0.0 0.0 0.0
CAF GHC.Float <entire-module> 616 0 0.0 0.0 0.0 0.0
CAF GHC.IO.Encoding <entire-module> 606 0 0.0 0.0 0.0 0.0
CAF GHC.IO.Encoding.Iconv <entire-module> 604 0 0.0 0.0 0.0 0.0
CAF GHC.IO.Encoding.UTF8 <entire-module> 599 0 0.0 0.0 0.0 0.0
CAF GHC.IO.Exception <entire-module> 598 0 0.0 0.0 0.0 0.0
CAF GHC.IO.FD <entire-module> 597 0 0.0 0.0 0.0 0.0
CAF GHC.IO.Handle.FD <entire-module> 595 0 0.0 0.0 0.0 0.0
CAF GHC.Show <entire-module> 574 0 0.0 0.0 0.0 0.0
CAF System.Exit <entire-module> 558 0 0.0 0.0 0.0 0.0
CAF System.Timeout <entire-module> 553 0 0.0 0.0 0.0 0.0
CAF Text.Printf <entire-module> 550 0 0.0 0.0 0.0 0.0
CAF Text.Read.Lex <entire-module> 548 0 0.0 0.0 0.0 0.0
CAF Data.Typeable.Internal <entire-module> 542 0 0.0 0.0 0.0 0.0
main Main tests/Sanity.hs:54:1-45 1030 0 0.0 0.0 100.0 100.0
defaultMain Test.Tasty Test/Tasty.hs:101:1-59 1031 0 0.0 0.0 100.0 100.0
defaultMainWithIngredients Test.Tasty.CmdLine Test/Tasty/CmdLine.hs:(170,1)-(181,45) 1032 0 0.0 0.0 100.0 100.0
parseOptions Test.Tasty.CmdLine Test/Tasty/CmdLine.hs:(152,1)-(161,33) 1035 1 0.0 0.0 0.0 0.0
info Options.Applicative.Builder src/Options/Applicative/Builder.hs:(454,1)-(463,34) 1126 1 0.0 0.0 0.0 0.0
fullDesc Options.Applicative.Builder src/Options/Applicative/Builder.hs:390:1-52 1131 0 0.0 0.0 0.0 0.0
suiteEnvOptions Test.Tasty.Options.Env Test/Tasty/Options/Env.hs:60:1-64 1173 1 0.0 0.0 0.0 0.0
suiteOptions Test.Tasty.Ingredients Test/Tasty/Ingredients.hs:(120,1)-(123,18) 1177 1 0.0 0.0 0.0 0.0
ingredientsOptions Test.Tasty.Ingredients Test/Tasty/Ingredients.hs:115:1-75 1179 0 0.0 0.0 0.0 0.0
ingredientOptions Test.Tasty.Ingredients Test/Tasty/Ingredients.hs:(109,1)-(111,45) 1181 2 0.0 0.0 0.0 0.0
uniqueOptionDescriptions Test.Tasty.Options Test/Tasty/Options.hs:(138,1)-(143,62) 1180 0 0.0 0.0 0.0 0.0
treeOptions Test.Tasty.Core Test/Tasty/Core.hs:(385,1)-(400,31) 1182 0 0.0 0.0 0.0 0.0
foldTestTree Test.Tasty.Core Test/Tasty/Core.hs:(364,1)-(381,46) 1183 1 0.0 0.0 0.0 0.0
lookupOption Test.Tasty.Options Test/Tasty/Options.hs:(115,1)-(119,27) 1185 1 0.0 0.0 0.0 0.0
testPatternMatches Test.Tasty.Patterns Test/Tasty/Patterns.hs:(58,1)-(61,48) 1184 1 0.0 0.0 0.0 0.0
uniqueOptionDescriptions Test.Tasty.Options Test/Tasty/Options.hs:(138,1)-(143,62) 1178 0 0.0 0.0 0.0 0.0
getEnvOptions Test.Tasty.Options.Env Test/Tasty/Options/Env.hs:(38,1)-(56,57) 1175 0 0.0 0.0 0.0 0.0
getApp Test.Tasty.Runners.Reducers Test/Tasty/Runners/Reducers.hs:64:23-28 1176 1 0.0 0.0 0.0 0.0
suiteOptionParser Test.Tasty.CmdLine Test/Tasty/CmdLine.hs:140:1-65 1036 1 0.0 0.0 0.0 0.0
suiteOptions Test.Tasty.Ingredients Test/Tasty/Ingredients.hs:(120,1)-(123,18) 1039 1 0.0 0.0 0.0 0.0
ingredientsOptions Test.Tasty.Ingredients Test/Tasty/Ingredients.hs:115:1-75 1044 0 0.0 0.0 0.0 0.0
ingredientOptions Test.Tasty.Ingredients Test/Tasty/Ingredients.hs:(109,1)-(111,45) 1047 2 0.0 0.0 0.0 0.0
uniqueOptionDescriptions Test.Tasty.Options Test/Tasty/Options.hs:(138,1)-(143,62) 1045 0 0.0 0.0 0.0 0.0
treeOptions Test.Tasty.Core Test/Tasty/Core.hs:(385,1)-(400,31) 1051 0 0.0 0.0 0.0 0.0
foldTestTree Test.Tasty.Core Test/Tasty/Core.hs:(364,1)-(381,46) 1053 1 0.0 0.0 0.0 0.0
lookupOption Test.Tasty.Options Test/Tasty/Options.hs:(115,1)-(119,27) 1057 1 0.0 0.0 0.0 0.0
testPatternMatches Test.Tasty.Patterns Test/Tasty/Patterns.hs:(58,1)-(61,48) 1056 1 0.0 0.0 0.0 0.0
uniqueOptionDescriptions Test.Tasty.Options Test/Tasty/Options.hs:(138,1)-(143,62) 1041 0 0.0 0.0 0.0 0.0
optionParser Test.Tasty.CmdLine Test/Tasty/CmdLine.hs:(39,1)-(43,39) 1038 0 0.0 0.0 0.0 0.0
evalParser Options.Applicative.Common src/Options/Applicative/Common.hs:(245,1)-(249,56) 1077 8 0.0 0.0 0.0 0.0
getApp Test.Tasty.Runners.Reducers Test/Tasty/Runners/Reducers.hs:64:23-28 1171 1 0.0 0.0 0.0 0.0
execParser Options.Applicative.Extra src/Options/Applicative/Extra.hs:84:1-42 1117 0 0.0 0.0 0.0 0.0
customExecParser Options.Applicative.Extra src/Options/Applicative/Extra.hs:(88,1)-(90,23) 1118 1 0.0 0.0 0.0 0.0
execParserPure Options.Applicative.Extra src/Options/Applicative/Extra.hs:(124,1)-(133,33) 1119 1 0.0 0.0 0.0 0.0
runP Options.Applicative.Internal src/Options/Applicative/Internal.hs:91:1-59 1120 1 0.0 0.0 0.0 0.0
bashCompletionParser Options.Applicative.BashCompletion src/Options/Applicative/BashCompletion.hs:(35,1)-(65,7) 1132 1 0.0 0.0 0.0 0.0
infoParser Options.Applicative.Types src/Options/Applicative/Types.hs:90:5-14 1152 1 0.0 0.0 0.0 0.0
runParserInfo Options.Applicative.Common src/Options/Applicative/Common.hs:233:1-62 1121 1 0.0 0.0 0.0 0.0
infoParser Options.Applicative.Types src/Options/Applicative/Types.hs:90:5-14 1125 1 0.0 0.0 0.0 0.0
runParserFully Options.Applicative.Common src/Options/Applicative/Common.hs:(236,1)-(240,33) 1122 1 0.0 0.0 0.0 0.0
runParser Options.Applicative.Common src/Options/Applicative/Common.hs:(203,1)-(220,24) 1123 1 0.0 0.0 0.0 0.0
evalParser Options.Applicative.Common src/Options/Applicative/Common.hs:(245,1)-(249,56) 1124 59 0.0 0.0 0.0 0.0
bashCompletionParser Options.Applicative.BashCompletion src/Options/Applicative/BashCompletion.hs:(35,1)-(65,7) 1146 0 0.0 0.0 0.0 0.0
fromM Options.Applicative.Types src/Options/Applicative/Types.hs:287:1-26 1147 0 0.0 0.0 0.0 0.0
manyM Options.Applicative.Types src/Options/Applicative/Types.hs:(293,1)-(297,30) 1148 0 0.0 0.0 0.0 0.0
runParserM Options.Applicative.Types src/Options/Applicative/Types.hs:273:5-14 1149 1 0.0 0.0 0.0 0.0
handleParseResult Options.Applicative.Extra src/Options/Applicative/Extra.hs:(94,1)-(106,17) 1172 1 0.0 0.0 0.0 0.0
header Options.Applicative.Builder src/Options/Applicative/Builder.hs:398:1-57 1129 0 0.0 0.0 0.0 0.0
tryIngredients Test.Tasty.Ingredients Test/Tasty/Ingredients.hs:(100,1)-(101,50) 1186 1 0.0 0.0 100.0 100.0
launchTestTree Test.Tasty.Run Test/Tasty/Run.hs:(421,1)-(452,31) 1192 1 0.0 0.0 100.0 100.0
timed Test.Tasty.Runners.Utils Test/Tasty/Runners/Utils.hs:(99,1)-(103,23) 1198 2 0.0 0.0 100.0 100.0
lookupOption Test.Tasty.Options Test/Tasty/Options.hs:(115,1)-(119,27) 1203 1 0.0 0.0 0.0 0.0
runInParallel Test.Tasty.Parallel Test/Tasty/Parallel.hs:(44,1)-(68,21) 1202 1 0.0 0.0 100.0 100.0
async Control.Concurrent.Async Control/Concurrent/Async.hs:69:1-35 1208 0 0.0 0.0 100.0 100.0
actionRun Test.Tasty.Parallel Test/Tasty/Parallel.hs:23:5-13 1217 1 0.0 0.0 100.0 100.0
foldTestTree Test.Tasty.Core Test/Tasty/Core.hs:(364,1)-(381,46) 1218 0 0.0 0.0 100.0 100.0
asyncThreadId Control.Concurrent.Async Control/Concurrent/Async.hs:61:5-17 1221 1 0.0 0.0 0.0 0.0
exceptionResult Test.Tasty.Core Test/Tasty/Core.hs:(114,1)-(120,3) 1467 1 0.0 0.0 0.0 0.0
getTraversal Test.Tasty.Runners.Reducers Test/Tasty/Runners/Reducers.hs:53:35-46 1462 1 0.0 0.0 0.0 0.0
sanity Main tests/Sanity.hs:(31,1)-(51,60) 1239 1 0.0 0.0 100.0 100.0
assertFailure Test.HUnit.Lang src/Test/HUnit/Lang.hs:59:1-80 1461 1 0.0 0.0 0.0 0.0
callStack Data.CallStack src/Data/CallStack.hs:(45,1)-(47,41) 1468 1 0.0 0.0 0.0 0.0
extraArgs Main tests/Sanity.hs:(27,1)-(28,62) 1242 1 0.0 0.0 0.0 0.0
defaultMain Criterion.Main Criterion/Main.hs:95:1-43 1247 0 0.0 0.0 100.0 100.0
defaultMainWith Criterion.Main Criterion/Main.hs:(149,1)-(151,16) 1248 0 0.0 0.0 100.0 100.0
describe Criterion.Main.Options Criterion/Main/Options.hs:200:1-43 1267 1 0.0 0.0 0.0 0.0
describeWith Criterion.Main.Options Criterion/Main/Options.hs:(204,1)-(207,38) 1268 1 0.0 0.0 0.0 0.0
info Options.Applicative.Builder src/Options/Applicative/Builder.hs:(454,1)-(463,34) 1269 1 0.0 0.0 0.0 0.0
fullDesc Options.Applicative.Builder src/Options/Applicative/Builder.hs:390:1-52 1273 0 0.0 0.0 0.0 0.0
footerDoc Options.Applicative.Builder src/Options/Applicative/Builder.hs:412:1-60 1272 0 0.0 0.0 0.0 0.0
header Options.Applicative.Builder src/Options/Applicative/Builder.hs:398:1-57 1275 0 0.0 0.0 0.0 0.0
parseWith Criterion.Main.Options Criterion/Main/Options.hs:(98,1)-(122,49) 1307 1 0.0 0.0 0.0 0.0
config Criterion.Main.Options Criterion/Main/Options.hs:(126,1)-(155,51) 1309 1 0.0 0.0 0.0 0.0
strOption Options.Applicative.Builder src/Options/Applicative/Builder.hs:360:1-22 1343 6 0.0 0.0 0.0 0.0
option Options.Applicative.Builder src/Options/Applicative/Builder.hs:(370,1)-(375,65) 1344 6 0.0 0.0 0.0 0.0
mkParser Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:(165,1)-(169,36) 1345 6 0.0 0.0 0.0 0.0
mkOption Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:175:1-43 1346 5 0.0 0.0 0.0 0.0
optNames Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:36:5-12 1347 5 0.0 0.0 0.0 0.0
str Options.Applicative.Builder src/Options/Applicative/Builder.hs:134:1-30 1359 5 0.0 0.0 0.0 0.0
value Options.Applicative.Builder src/Options/Applicative/Builder.hs:178:1-50 1317 5 0.0 0.0 0.0 0.0
option Options.Applicative.Builder src/Options/Applicative/Builder.hs:(370,1)-(375,65) 1320 4 0.0 0.0 0.0 0.0
mkParser Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:(165,1)-(169,36) 1321 4 0.0 0.0 0.0 0.0
mkOption Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:175:1-43 1322 3 0.0 0.0 0.0 0.0
optNames Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:36:5-12 1323 3 0.0 0.0 0.0 0.0
fieldMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:133:1-28 1325 0 0.0 0.0 0.0 0.0
optNames Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:36:5-12 1326 5 0.0 0.0 0.0 0.0
long Options.Applicative.Builder src/Options/Applicative/Builder.hs:166:1-32 1348 0 0.0 0.0 0.0 0.0
fieldMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:133:1-28 1349 0 0.0 0.0 0.0 0.0
optNames Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:36:5-12 1368 2 0.0 0.0 0.0 0.0
short Options.Applicative.Builder src/Options/Applicative/Builder.hs:162:1-34 1365 0 0.0 0.0 0.0 0.0
fieldMod Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:133:1-28 1366 0 0.0 0.0 0.0 0.0
optNames Options.Applicative.Builder.Internal src/Options/Applicative/Builder/Internal.hs:36:5-12 1367 1 0.0 0.0 0.0 0.0
runMode Criterion.Main Criterion/Main.hs:(158,1)-(173,34) 1396 1 0.0 0.0 100.0 100.0
unId Data.Vector.Fusion.Util Data/Vector/Fusion/Util.hs:25:21-24 1424 16 0.0 0.0 0.0 0.0
marray# Data.Primitive.Array Data/Primitive/Array.hs:104:5-11 1425 8 0.0 0.0 0.0 0.0
array# Data.Primitive.Array Data/Primitive/Array.hs:91:5-10 1426 7 0.0 0.0 0.0 0.0
makeMatcher Criterion.Main Criterion/Main.hs:(103,1)-(113,99) 1399 1 0.0 0.0 0.0 0.0
runAndAnalyse Criterion.Internal Criterion/Internal.hs:(119,1)-(157,12) 1431 1 0.0 0.0 0.0 0.0
sChunks Data.Vector.Fusion.Bundle.Monadic Data/Vector/Fusion/Bundle/Monadic.hs:123:30-36 1423 1 0.0 0.0 0.0 0.0
sSize Data.Vector.Fusion.Bundle.Monadic Data/Vector/Fusion/Bundle/Monadic.hs:125:30-34 1422 1 0.0 0.0 0.0 0.0
upperBound Data.Vector.Fusion.Bundle.Size Data/Vector/Fusion/Bundle/Size.hs:(126,1)-(128,30) 1421 1 0.0 0.0 0.0 0.0
withConfig Criterion.Monad Criterion/Monad.hs:(29,1)-(31,29) 1400 1 0.0 0.0 100.0 100.0
withCP65001 System.IO.CodePage src/System/IO/CodePage.hs:68:1-34 1404 0 0.0 0.0 100.0 100.0
withCodePage System.IO.CodePage src/System/IO/CodePage.hs:99:1-49 1405 0 0.0 0.0 100.0 100.0
withCodePageOptions System.IO.CodePage src/System/IO/CodePage.hs:(109,1)-(194,44) 1407 1 0.0 0.0 100.0 100.0
defaultFallbackCodePageEncoding System.IO.CodePage.Internal src/System/IO/CodePage/Internal.hs:(107,1)-(121,65) 1411 1 0.0 0.0 0.0 0.0
runAndAnalyse Criterion.Internal Criterion/Internal.hs:(119,1)-(157,12) 1432 0 0.0 0.0 100.0 100.0
addPrefix Criterion.Measurement.Types src/Criterion/Measurement/Types.hs:(605,1)-(606,38) 1447 3 0.0 0.0 0.0 0.0
config Criterion.Monad.Internal Criterion/Monad/Internal.hs:32:5-10 1433 1 0.0 0.0 0.0 0.0
jsonFile Criterion.Types Criterion/Types.hs:119:7-14 1434 1 0.0 0.0 0.0 0.0
note Criterion.IO.Printf Criterion/IO/Printf.hs:87:1-46 1444 1 0.0 0.0 0.0 0.0
config Criterion.Monad.Internal Criterion/Monad/Internal.hs:32:5-10 1446 1 0.0 0.0 0.0 0.0
verbosity Criterion.Types Criterion/Types.hs:123:7-15 1445 1 0.0 0.0 0.0 0.0
runAndAnalyseOne Criterion.Internal Criterion/Internal.hs:(109,1)-(111,24) 1448 1 0.0 0.0 100.0 100.0
runOne Criterion.Internal Criterion/Internal.hs:(49,1)-(54,34) 1449 1 0.0 0.0 100.0 100.0
config Criterion.Monad.Internal Criterion/Monad/Internal.hs:32:5-10 1450 1 0.0 0.0 0.0 0.0
runBenchmark Criterion.Measurement src/Criterion/Measurement.hs:(292,1)-(316,41) 1451 1 0.0 0.0 100.0 100.0
applyGCStatistics Criterion.Measurement src/Criterion/Measurement.hs:(356,1)-(370,29) 1460 225 0.0 0.0 0.0 0.0
getGCStatistics Criterion.Measurement src/Criterion/Measurement.hs:(124,1)-(151,43) 1458 0 0.0 0.0 0.0 0.0
whnf Criterion.Measurement.Types src/Criterion/Measurement/Types.hs:277:1-38 1453 0 0.1 0.0 100.0 100.0
fib Main tests/Sanity.hs:(20,1)-(23,35) 1455 0 99.9 100.0 99.9 100.0
writeCsv Criterion.IO.Printf Criterion/IO/Printf.hs:(99,1)-(102,49) 1414 0 0.0 0.0 0.0 0.0
config Criterion.Monad.Internal Criterion/Monad/Internal.hs:32:5-10 1416 1 0.0 0.0 0.0 0.0
csvFile Criterion.Types Criterion/Types.hs:117:7-13 1415 1 0.0 0.0 0.0 0.0
recordSep Data.Csv.Encoding src/Data/Csv/Encoding.hs:(342,1)-(343,32) 1430 0 0.0 0.0 0.0 0.0
execParser Options.Applicative.Extra src/Options/Applicative/Extra.hs:84:1-42 1249 0 0.0 0.0 0.0 0.0
customExecParser Options.Applicative.Extra src/Options/Applicative/Extra.hs:(88,1)-(90,23) 1250 1 0.0 0.0 0.0 0.0
execParserPure Options.Applicative.Extra src/Options/Applicative/Extra.hs:(124,1)-(133,33) 1252 1 0.0 0.0 0.0 0.0
runP Options.Applicative.Internal src/Options/Applicative/Internal.hs:91:1-59 1253 1 0.0 0.0 0.0 0.0
bashCompletionParser Options.Applicative.BashCompletion src/Options/Applicative/BashCompletion.hs:(35,1)-(65,7) 1277 1 0.0 0.0 0.0 0.0
infoParser Options.Applicative.Types src/Options/Applicative/Types.hs:90:5-14 1304 1 0.0 0.0 0.0 0.0
runParserInfo Options.Applicative.Common src/Options/Applicative/Common.hs:233:1-62 1254 1 0.0 0.0 0.0 0.0
infoParser Options.Applicative.Types src/Options/Applicative/Types.hs:90:5-14 1276 1 0.0 0.0 0.0 0.0
infoPolicy Options.Applicative.Types src/Options/Applicative/Types.hs:97:5-14 1266 1 0.0 0.0 0.0 0.0
runParserFully Options.Applicative.Common src/Options/Applicative/Common.hs:(236,1)-(240,33) 1255 1 0.0 0.0 0.0 0.0
runParser Options.Applicative.Common src/Options/Applicative/Common.hs:(203,1)-(220,24) 1256 6 0.0 0.0 0.0 0.0
evalParser Options.Applicative.Common src/Options/Applicative/Common.hs:(245,1)-(249,56) 1369 54 0.0 0.0 0.0 0.0
config Criterion.Main.Options Criterion/Main/Options.hs:(126,1)-(155,51) 1370 0 0.0 0.0 0.0 0.0
fromM Options.Applicative.Types src/Options/Applicative/Types.hs:287:1-26 1371 0 0.0 0.0 0.0 0.0
manyM Options.Applicative.Types src/Options/Applicative/Types.hs:(293,1)-(297,30) 1372 0 0.0 0.0 0.0 0.0
runParserM Options.Applicative.Types src/Options/Applicative/Types.hs:273:5-14 1373 1 0.0 0.0 0.0 0.0
describe Criterion.Main.Options Criterion/Main/Options.hs:200:1-43 1397 0 0.0 0.0 0.0 0.0
parseWith Criterion.Main.Options Criterion/Main/Options.hs:(98,1)-(122,49) 1398 0 0.0 0.0 0.0 0.0
config Criterion.Main.Options Criterion/Main/Options.hs:(126,1)-(155,51) 1417 0 0.0 0.0 0.0 0.0
parseWith Criterion.Main.Options Criterion/Main/Options.hs:(98,1)-(122,49) 1391 0 0.0 0.0 0.0 0.0
fromM Options.Applicative.Types src/Options/Applicative/Types.hs:287:1-26 1392 0 0.0 0.0 0.0 0.0
manyM Options.Applicative.Types src/Options/Applicative/Types.hs:(293,1)-(297,30) 1393 0 0.0 0.0 0.0 0.0
runParserM Options.Applicative.Types src/Options/Applicative/Types.hs:273:5-14 1394 1 0.0 0.0 0.0 0.0
runParserStep Options.Applicative.Common src/Options/Applicative/Common.hs:(223,1)-(227,35) 1257 5 0.0 0.0 0.0 0.0
<!> Options.Applicative.Internal src/Options/Applicative/Internal.hs:(249,1)-(252,15) 1278 64 0.0 0.0 0.0 0.0
prefDisambiguate Options.Applicative.Types src/Options/Applicative/Types.hs:113:5-20 1260 46 0.0 0.0 0.0 0.0
optMain Options.Applicative.Types src/Options/Applicative/Types.hs:171:5-11 1280 41 0.0 0.0 0.0 0.0
evalParser Options.Applicative.Common src/Options/Applicative/Common.hs:(245,1)-(249,56) 1299 18 0.0 0.0 0.0 0.0
disamb Options.Applicative.Internal src/Options/Applicative/Internal.hs:(258,1)-(265,18) 1258 5 0.0 0.0 0.0 0.0
runListT Options.Applicative.Internal src/Options/Applicative/Internal.hs:(183,1)-(187,43) 1259 10 0.0 0.0 0.0 0.0
takeListT Options.Applicative.Internal src/Options/Applicative/Internal.hs:(179,1)-(180,75) 1265 0 0.0 0.0 0.0 0.0
<!> Options.Applicative.Internal src/Options/Applicative/Internal.hs:(249,1)-(252,15) 1279 0 0.0 0.0 0.0 0.0
runReadM Options.Applicative.Internal src/Options/Applicative/Internal.hs:98:1-63 1351 5 0.0 0.0 0.0 0.0
hoistEither Options.Applicative.Internal src/Options/Applicative/Internal.hs:88:1-34 1363 5 0.0 0.0 0.0 0.0
withReadM Options.Applicative.Internal src/Options/Applicative/Internal.hs:(101,1)-(104,12) 1353 0 0.0 0.0 0.0 0.0
crReader Options.Applicative.Types src/Options/Applicative/Types.hs:231:5-12 1355 5 0.0 0.0 0.0 0.0
describe Criterion.Main.Options Criterion/Main/Options.hs:200:1-43 1356 0 0.0 0.0 0.0 0.0
parseWith Criterion.Main.Options Criterion/Main/Options.hs:(98,1)-(122,49) 1357 0 0.0 0.0 0.0 0.0
config Criterion.Main.Options Criterion/Main/Options.hs:(126,1)-(155,51) 1358 0 0.0 0.0 0.0 0.0
strOption Options.Applicative.Builder src/Options/Applicative/Builder.hs:360:1-22 1360 0 0.0 0.0 0.0 0.0
str Options.Applicative.Builder src/Options/Applicative/Builder.hs:134:1-30 1361 0 0.0 0.0 0.0 0.0
unReadM Options.Applicative.Types src/Options/Applicative/Types.hs:189:5-11 1354 5 0.0 0.0 0.0 0.0
uncons Options.Applicative.Internal src/Options/Applicative/Internal.hs:(94,1)-(95,30) 1350 5 0.0 0.0 0.0 0.0
takeListT Options.Applicative.Internal src/Options/Applicative/Internal.hs:(179,1)-(180,75) 1264 10 0.0 0.0 0.0 0.0
withReadM Options.Applicative.Internal src/Options/Applicative/Internal.hs:(101,1)-(104,12) 1352 5 0.0 0.0 0.0 0.0
bashCompletionParser Options.Applicative.BashCompletion src/Options/Applicative/BashCompletion.hs:(35,1)-(65,7) 1300 0 0.0 0.0 0.0 0.0
fromM Options.Applicative.Types src/Options/Applicative/Types.hs:287:1-26 1301 0 0.0 0.0 0.0 0.0
manyM Options.Applicative.Types src/Options/Applicative/Types.hs:(293,1)-(297,30) 1302 0 0.0 0.0 0.0 0.0
runParserM Options.Applicative.Types src/Options/Applicative/Types.hs:273:5-14 1303 1 0.0 0.0 0.0 0.0
config Criterion.Main.Options Criterion/Main/Options.hs:(126,1)-(155,51) 1337 0 0.0 0.0 0.0 0.0
fromM Options.Applicative.Types src/Options/Applicative/Types.hs:287:1-26 1338 0 0.0 0.0 0.0 0.0
manyM Options.Applicative.Types src/Options/Applicative/Types.hs:(293,1)-(297,30) 1339 0 0.0 0.0 0.0 0.0
runParserM Options.Applicative.Types src/Options/Applicative/Types.hs:273:5-14 1340 5 0.0 0.0 0.0 0.0
handleParseResult Options.Applicative.Extra src/Options/Applicative/Extra.hs:(94,1)-(106,17) 1395 1 0.0 0.0 0.0 0.0
getArgEnv Main tests/Sanity.hs:(59,1)-(61,38) 1241 0 0.0 0.0 0.0 0.0
withAsync Control.Concurrent.Async Control/Concurrent/Async.hs:97:1-43 1220 0 0.0 0.0 0.0 0.0
actionStatus Test.Tasty.Parallel Test/Tasty/Parallel.hs:22:5-16 1216 1 0.0 0.0 0.0 0.0
consoleTestReporter Test.Tasty.Ingredients.ConsoleReporter Test/Tasty/Ingredients/ConsoleReporter.hs:(380,1)-(429,44) 1209 0 0.0 0.0 0.0 0.0
buildTestOutput Test.Tasty.Ingredients.ConsoleReporter Test/Tasty/Ingredients/ConsoleReporter.hs:(97,1)-(154,19) 1226 1 0.0 0.0 0.0 0.0
foldTestTree Test.Tasty.Core Test/Tasty/Core.hs:(364,1)-(381,46) 1228 2 0.0 0.0 0.0 0.0
lookupOption Test.Tasty.Options Test/Tasty/Options.hs:(115,1)-(119,27) 1230 2 0.0 0.0 0.0 0.0
testPatternMatches Test.Tasty.Patterns Test/Tasty/Patterns.hs:(58,1)-(61,48) 1229 2 0.0 0.0 0.0 0.0
wcwidth Data.Char.WCWidth Data/Char/WCWidth.hs:69:1-69 1232 0 0.0 0.0 0.0 0.0
trivialFold Test.Tasty.Core Test/Tasty/Core.hs:(330,1)-(335,3) 1227 2 0.0 0.0 0.0 0.0
getApp Test.Tasty.Runners.Reducers Test/Tasty/Runners/Reducers.hs:64:23-28 1233 1 0.0 0.0 0.0 0.0
foldTestOutput Test.Tasty.Ingredients.ConsoleReporter Test/Tasty/Ingredients/ConsoleReporter.hs:(171,1)-(185,18) 1234 1 0.0 0.0 0.0 0.0
getApp Test.Tasty.Runners.Reducers Test/Tasty/Runners/Reducers.hs:64:23-28 1235 1 0.0 0.0 0.0 0.0
buildTestOutput Test.Tasty.Ingredients.ConsoleReporter Test/Tasty/Ingredients/ConsoleReporter.hs:(97,1)-(154,19) 1236 0 0.0 0.0 0.0 0.0
foldTestTree Test.Tasty.Core Test/Tasty/Core.hs:(364,1)-(381,46) 1237 0 0.0 0.0 0.0 0.0
withConsoleFormat Test.Tasty.Ingredients.ConsoleReporter Test/Tasty/Ingredients/ConsoleReporter.hs:(649,1)-(658,22) 1471 3 0.0 0.0 0.0 0.0
resultDescription Test.Tasty.Core Test/Tasty/Core.hs:55:5-21 1466 1 0.0 0.0 0.0 0.0
resultDetailsPrinter Test.Tasty.Core Test/Tasty/Core.hs:69:5-24 1477 1 0.0 0.0 0.0 0.0
resultOutcome Test.Tasty.Core Test/Tasty/Core.hs:53:5-17 1470 1 0.0 0.0 0.0 0.0
resultShortDescription Test.Tasty.Core Test/Tasty/Core.hs:64:5-26 1473 1 0.0 0.0 0.0 0.0
resultSuccessful Test.Tasty.Core Test/Tasty/Core.hs:(107,1)-(110,23) 1475 1 0.0 0.0 0.0 0.0
resultOutcome Test.Tasty.Core Test/Tasty/Core.hs:53:5-17 1476 1 0.0 0.0 0.0 0.0
resultTime Test.Tasty.Core Test/Tasty/Core.hs:67:5-14 1474 1 0.0 0.0 0.0 0.0
formatMessage Test.Tasty.Runners.Utils Test/Tasty/Runners/Utils.hs:(43,1)-(52,107) 1464 0 0.0 0.0 0.0 0.0
wcwidth Data.Char.WCWidth Data/Char/WCWidth.hs:69:1-69 1238 0 0.0 0.0 0.0 0.0
getTraversal Test.Tasty.Runners.Reducers Test/Tasty/Runners/Reducers.hs:53:35-46 1225 1 0.0 0.0 0.0 0.0
lookupOption Test.Tasty.Options Test/Tasty/Options.hs:(115,1)-(119,27) 1210 1 0.0 0.0 0.0 0.0
printStatistics Test.Tasty.Ingredients.ConsoleReporter Test/Tasty/Ingredients/ConsoleReporter.hs:(307,1)-(312,40) 1482 1 0.0 0.0 0.0 0.0
statFailures Test.Tasty.Ingredients.ConsoleReporter Test/Tasty/Ingredients/ConsoleReporter.hs:278:5-16 1486 2 0.0 0.0 0.0 0.0
withConsoleFormat Test.Tasty.Ingredients.ConsoleReporter Test/Tasty/Ingredients/ConsoleReporter.hs:(649,1)-(658,22) 1487 2 0.0 0.0 0.0 0.0
statTotal Test.Tasty.Ingredients.ConsoleReporter Test/Tasty/Ingredients/ConsoleReporter.hs:275:5-13 1488 1 0.0 0.0 0.0 0.0
statFailures Test.Tasty.Ingredients.ConsoleReporter Test/Tasty/Ingredients/ConsoleReporter.hs:278:5-16 1491 1 0.0 0.0 0.0 0.0
useColor Test.Tasty.Ingredients.ConsoleReporter Test/Tasty/Ingredients/ConsoleReporter.hs:(507,1)-(511,20) 1472 1 0.0 0.0 0.0 0.0
computeStatistics Test.Tasty.Ingredients.ConsoleReporter Test/Tasty/Ingredients/ConsoleReporter.hs:(291,1)-(293,30) 1480 0 0.0 0.0 0.0 0.0
getApp Test.Tasty.Runners.Reducers Test/Tasty/Runners/Reducers.hs:64:23-28 1481 1 0.0 0.0 0.0 0.0
resultSuccessful Test.Tasty.Core Test/Tasty/Core.hs:(107,1)-(110,23) 1484 1 0.0 0.0 0.0 0.0
resultOutcome Test.Tasty.Core Test/Tasty/Core.hs:53:5-17 1485 1 0.0 0.0 0.0 0.0
hSupportsANSI System.Console.ANSI.Unix src/System/Console/ANSI/Unix.hs:(74,1)-(77,65) 1215 0 0.0 0.0 0.0 0.0
hSupportsANSIColor System.Console.ANSI.Unix src/includes/Common-Include.hs:(144,1)-(148,49) 1223 0 0.0 0.0 0.0 0.0
hSupportsANSI System.Console.ANSI.Unix src/System/Console/ANSI/Unix.hs:(74,1)-(77,65) 1224 1 0.0 0.0 0.0 0.0
getTime Test.Tasty.Runners.Utils Test/Tasty/Runners/Utils.hs:(111,1)-(119,31) 1200 0 0.0 0.0 0.0 0.0
getTime System.Clock System/Clock.hsc:175:1-101 1201 3 0.0 0.0 0.0 0.0
toNanoSecs System.Clock System/Clock.hsc:262:1-73 1489 2 0.0 0.0 0.0 0.0
foldTestTree Test.Tasty.Core Test/Tasty/Core.hs:(364,1)-(381,46) 1195 1 0.0 0.0 0.0 0.0
lookupOption Test.Tasty.Options Test/Tasty/Options.hs:(115,1)-(119,27) 1197 2 0.0 0.0 0.0 0.0
testPatternMatches Test.Tasty.Patterns Test/Tasty/Patterns.hs:(58,1)-(61,48) 1196 1 0.0 0.0 0.0 0.0
getTraversal Test.Tasty.Runners.Reducers Test/Tasty/Runners/Reducers.hs:53:35-46 1193 1 0.0 0.0 0.0 0.0
trivialFold Test.Tasty.Core Test/Tasty/Core.hs:(330,1)-(335,3) 1194 1 0.0 0.0 0.0 0.0
consoleTestReporter Test.Tasty.Ingredients.ConsoleReporter Test/Tasty/Ingredients/ConsoleReporter.hs:(380,1)-(429,44) 1211 0 0.0 0.0 0.0 0.0
lookupOption Test.Tasty.Options Test/Tasty/Options.hs:(115,1)-(119,27) 1212 2 0.0 0.0 0.0 0.0
listingTests Test.Tasty.Ingredients.ListTests Test/Tasty/Ingredients/ListTests.hs:(39,1)-(45,19) 1187 0 0.0 0.0 0.0 0.0
lookupOption Test.Tasty.Options Test/Tasty/Options.hs:(115,1)-(119,27) 1188 1 0.0 0.0 0.0 0.0
installSignalHandlers Test.Tasty.Runners.Utils Test/Tasty/Runners/Utils.hs:(71,1)-(82,67) 1034 0 0.0 0.0 0.0 0.0Also, if relevant, I'm using GHC 8.10.5 from Homebrew.
csstaub
commented
3 years ago mach_absolute_time by itself seems to work fine:
~/D/rdtsc-test ❯❯❯ cat main.c
#include <stdio.h>
#include <mach/mach_time.h>
int main(void) {
unsigned long val1 = mach_absolute_time();
printf("%lu\n", val1);
unsigned long val2 = mach_absolute_time();
printf("%lu\n", val2);
return 0;
}
~/D/rdtsc-test ❯❯❯ ./main
1037131009860
1037131010500Peculiar. I was hoping that the profiling report would point to an obvious suspect, but that doesn't seem to be the case. I can't help but wonder if there's some other aspect of timing in the measure function that's going horribly wrong. Would you mind doing some Debug.Trace printing in criterion-measurement:Criterion.Measurement to see what might be going wrong? Something like this would be a good place to start:
diff --git a/criterion-measurement/src/Criterion/Measurement.hs b/criterion-measurement/src/Criterion/Measurement.hs
index 8cff06b..917b0ea 100644
--- a/criterion-measurement/src/Criterion/Measurement.hs
+++ b/criterion-measurement/src/Criterion/Measurement.hs
@@ -59,6 +59,8 @@ import qualified Control.Exception as Exc
import qualified Data.Vector as V
import qualified GHC.Stats as Stats
+import Debug.Trace
+
#if !(MIN_VERSION_base(4,7,0))
foreign import ccall "performGC" performMinorGC :: IO ()
#endif
@@ -206,7 +208,7 @@ measure bm iters = runBenchmarkable bm iters addResults $ \ !n act -> do
-- From these we can derive all other deltas, and performGC guarantees they
-- are up-to-date.
endStatsPostGC <- getGCStatistics
- let !m = applyGCStatistics endStatsPostGC endStatsPreGC startStats $ measured {
+ let !m = traceShowId $ applyGCStatistics endStatsPostGC endStatsPreGC startStats $ measured {
measTime = max 0 (endTime - startTime)
, measCpuTime = max 0 (endCpuTime - startCpuTime)
, measCycles = max 0 (fromIntegral (endCycles - startCycles))
Ideally, this should print out a bunch of Measureds where measTime, measCpuTime, measCycles, and measIters are all increasing. Something like:
Measured {measTime = 6.009540000000002e-2, measCpuTime = 6.25e-2, measCycles = 215938260, measIters = 25549, ...}
Measured {measTime = 6.339399999999995e-2, measCpuTime = 6.25e-2, measCycles = 227794572, measIters = 26826, ...}
Measured {measTime = 6.762709999999994e-2, measCpuTime = 6.25e-2, measCycles = 243007704, measIters = 28167, ...}If that isn't the case, that could hint at other problems.
csstaub
commented
3 years ago Applied patch, re-ran cabal test test:sanity --enable-profiling --test-options="+RTS -p":
First few lines of output:
Measured {measTime = 0.0, measCpuTime = 9.999999999999593e-6, measCycles = 189, measIters = 1, measAllocated = -9223372036854775808, measNumGcs = -9223372036854775808, measBytesCopied = -9223372036854775808, measMutatorWallSeconds = -Infinity, measMutatorCpuSeconds = -Infinity, measGcWallSeconds = -Infinity, measGcCpuSeconds = -Infinity}
Measured {measTime = 0.0, measCpuTime = 9.000000000002062e-6, measCycles = 215, measIters = 2, measAllocated = -9223372036854775808, measNumGcs = -9223372036854775808, measBytesCopied = -9223372036854775808, measMutatorWallSeconds = -Infinity, measMutatorCpuSeconds = -Infinity, measGcWallSeconds = -Infinity, measGcCpuSeconds = -Infinity}
Measured {measTime = 0.0, measCpuTime = 1.2000000000001593e-5, measCycles = 286, measIters = 3, measAllocated = -9223372036854775808, measNumGcs = -9223372036854775808, measBytesCopied = -9223372036854775808, measMutatorWallSeconds = -Infinity, measMutatorCpuSeconds = -Infinity, measGcWallSeconds = -Infinity, measGcCpuSeconds = -Infinity}
Measured {measTime = 0.0, measCpuTime = 1.6000000000002124e-5, measCycles = 369, measIters = 4, measAllocated = -9223372036854775808, measNumGcs = -9223372036854775808, measBytesCopied = -9223372036854775808, measMutatorWallSeconds = -Infinity, measMutatorCpuSeconds = -Infinity, measGcWallSeconds = -Infinity, measGcCpuSeconds = -Infinity}
Measured {measTime = 0.0, measCpuTime = 1.8000000000000654e-5, measCycles = 419, measIters = 5, measAllocated = -9223372036854775808, measNumGcs = -9223372036854775808, measBytesCopied = -9223372036854775808, measMutatorWallSeconds = -Infinity, measMutatorCpuSeconds = -Infinity, measGcWallSeconds = -Infinity, measGcCpuSeconds = -Infinity}
Measured {measTime = 0.0, measCpuTime = 2.1000000000003655e-5, measCycles = 484, measIters = 6, measAllocated = -9223372036854775808, measNumGcs = -9223372036854775808, measBytesCopied = -9223372036854775808, measMutatorWallSeconds = -Infinity, measMutatorCpuSeconds = -Infinity, measGcWallSeconds = -Infinity, measGcCpuSeconds = -Infinity}Last few lines of output:
Measured {measTime = 0.0, measCpuTime = 1.0741919999999965, measCycles = 25741743, measIters = 356119, measAllocated = -9223372036854775808, measNumGcs = -9223372036854775808, measBytesCopied = -9223372036854775808, measMutatorWallSeconds = -Infinity, measMutatorCpuSeconds = -Infinity, measGcWallSeconds = -Infinity, measGcCpuSeconds = -Infinity}
Measured {measTime = 0.0, measCpuTime = 1.1290440000000004, measCycles = 27058875, measIters = 373925, measAllocated = -9223372036854775808, measNumGcs = -9223372036854775808, measBytesCopied = -9223372036854775808, measMutatorWallSeconds = -Infinity, measMutatorCpuSeconds = -Infinity, measGcWallSeconds = -Infinity, measGcCpuSeconds = -Infinity}
Measured {measTime = 0.0, measCpuTime = 1.1826769999999982, measCycles = 28341294, measIters = 392622, measAllocated = -9223372036854775808, measNumGcs = -9223372036854775808, measBytesCopied = -9223372036854775808, measMutatorWallSeconds = -Infinity, measMutatorCpuSeconds = -Infinity, measGcWallSeconds = -Infinity, measGcCpuSeconds = -Infinity}
Measured {measTime = 0.0, measCpuTime = 1.2424420000000005, measCycles = 29773527, measIters = 412253, measAllocated = -9223372036854775808, measNumGcs = -9223372036854775808, measBytesCopied = -9223372036854775808, measMutatorWallSeconds = -Infinity, measMutatorCpuSeconds = -Infinity, measGcWallSeconds = -Infinity, measGcCpuSeconds = -Infinity}
Measured {measTime = 0.0, measCpuTime = 1.3038469999999975, measCycles = 31246545, measIters = 432866, measAllocated = -9223372036854775808, measNumGcs = -9223372036854775808, measBytesCopied = -9223372036854775808, measMutatorWallSeconds = -Infinity, measMutatorCpuSeconds = -Infinity, measGcWallSeconds = -Infinity, measGcCpuSeconds = -Infinity}
Measured {measTime = 0.0, measCpuTime = 1.3686839999999982, measCycles = 32799150, measIters = 454509, measAllocated = -9223372036854775808, measNumGcs = -9223372036854775808, measBytesCopied = -9223372036854775808, measMutatorWallSeconds = -Infinity, measMutatorCpuSeconds = -Infinity, measGcWallSeconds = -Infinity, measGcCpuSeconds = -Infinity}Seems like it's not working as expected, measTime is not increasing.
RyanGlScott
commented
3 years ago Ah! Indeed, measTime not increasing is a red flag. The function which populates this field is criterion_gettime, which is defined in this file on macOS. Does anything in that function (or in its sibling function criterion_inittime) look out of the ordinary on AArch64?
csstaub
commented
3 years ago Yes, it seems timebase_recip isn't being initialized correctly?
I annotated the code with some printfs and compiled it separately, this is what I got:
❯❯❯ gcc -o time time.c && ./time
=> criterion_inittime
timebase_recip = 0.000000
calling mach_timebase_info
timebase_info.denom = 3
timebase_info.numer = 125
timebase_recip = 0.000000
=> criterion_gettime
mach_absolute_time = 1744303326048
timebase_recip = 0.000000
mach_absolute_time * timebase_recip = 0.000000Alright, I can fix the measTime issue by casting timebase_info.denom and timebase_info.numer to double before doing the divide with 1e9. Doing that and re-running the sanity test suite shows an increasing measTime value. However the fib test still times out after 30s even with that fix.
csstaub
commented
3 years ago It kinda seems like cpu time is wonky at the beginning for small values:
Measured {measTime = 7.3200041583731945e-9, measCpuTime = 8.000000000001062e-6, measCycles = 123, measIters = 1, measAllocated = -9223372036854775808, measNumGcs = -9223372036854775808, measBytesCopied = -9223372036854775808, measMutatorWallSeconds = -Infinity, measMutatorCpuSeconds = -Infinity, measGcWallSeconds = -Infinity, measGcCpuSeconds = -Infinity}
Measured {measTime = 5.495998323112872e-9, measCpuTime = 9.000000000000327e-6, measCycles = 200, measIters = 2, measAllocated = -9223372036854775808, measNumGcs = -9223372036854775808, measBytesCopied = -9223372036854775808, measMutatorWallSeconds = -Infinity, measMutatorCpuSeconds = -Infinity, measGcWallSeconds = -Infinity, measGcCpuSeconds = -Infinity}
Measured {measTime = 4.151999632995285e-9, measCpuTime = 7.0000000000017965e-6, measCycles = 162, measIters = 3, measAllocated = -9223372036854775808, measNumGcs = -9223372036854775808, measBytesCopied = -9223372036854775808, measMutatorWallSeconds = -Infinity, measMutatorCpuSeconds = -Infinity, measGcWallSeconds = -Infinity, measGcCpuSeconds = -Infinity}
Measured {measTime = 4.631999672710663e-9, measCpuTime = 7.999999999997592e-6, measCycles = 184, measIters = 4, measAllocated = -9223372036854775808, measNumGcs = -9223372036854775808, measBytesCopied = -9223372036854775808, measMutatorWallSeconds = -Infinity, measMutatorCpuSeconds = -Infinity, measGcWallSeconds = -Infinity, measGcCpuSeconds = -Infinity}
Measured {measTime = 5.35199973228373e-9, measCpuTime = 8.999999999998592e-6, measCycles = 222, measIters = 5, measAllocated = -9223372036854775808, measNumGcs = -9223372036854775808, measBytesCopied = -9223372036854775808, measMutatorWallSeconds = -Infinity, measMutatorCpuSeconds = -Infinity, measGcWallSeconds = -Infinity, measGcCpuSeconds = -Infinity}Note we go from 8.000000000001062e-6 to to 9.000000000000327e-6 to 7.0000000000017965e-6. Unless I'm misunderstanding that's not expected, right?
RyanGlScott
commented
3 years ago Some searching reveals that criterion-measurement's use of mach_absolute_time to measure nanoseconds is suspect. See this page (in particular, the "Apply Timebase Information to Mach Absolute Time Values" section.) While mach_absolute_time has nanosecond resolution on Intel Macs, this is not the case on Apple silicon. That article recommends using clock_gettime_nsec_np instead (which was introduced in macOS 10.02) as a portable alternative to measuring the number of nanoseconds. See also https://github.com/dotnet/runtime/pull/43343, which hit a similar issue.
Can you try the following patch to see if it gives more reasonable results?
diff --git a/criterion-measurement/cbits/time-osx.c b/criterion-measurement/cbits/time-osx.c
index 60d1404..d7f8ab4 100644
--- a/criterion-measurement/cbits/time-osx.c
+++ b/criterion-measurement/cbits/time-osx.c
@@ -1,20 +1,13 @@
#include <mach/mach.h>
-#include <mach/mach_time.h>
-
-static mach_timebase_info_data_t timebase_info;
-static double timebase_recip;
+#include <time.h>
void criterion_inittime(void)
{
- if (timebase_recip == 0) {
- mach_timebase_info(&timebase_info);
- timebase_recip = (timebase_info.denom / timebase_info.numer) / 1e9;
- }
}
double criterion_gettime(void)
{
- return mach_absolute_time() * timebase_recip;
+ return clock_gettime_nsec_np(CLOCK_UPTIME_RAW) / 1e9;
}
static double to_double(time_value_t time)That seems to work!
Here's the full diff I have applied, for posterity:
diff --git a/criterion-measurement/cbits/cycles.c b/criterion-measurement/cbits/cycles.c
index 73ba7f9..6767878 100644
--- a/criterion-measurement/cbits/cycles.c
+++ b/criterion-measurement/cbits/cycles.c
@@ -1,6 +1,15 @@
#include "Rts.h"
-#if x86_64_HOST_ARCH || i386_HOST_ARCH
+#if darwin_HOST_OS
+
+#include <mach/mach_time.h>
+
+StgWord64 criterion_rdtsc(void)
+{
+ return mach_absolute_time();
+}
+
+#elif x86_64_HOST_ARCH || i386_HOST_ARCH
StgWord64 criterion_rdtsc(void)
{
diff --git a/criterion-measurement/cbits/time-osx.c b/criterion-measurement/cbits/time-osx.c
index 60d1404..31b1205 100644
--- a/criterion-measurement/cbits/time-osx.c
+++ b/criterion-measurement/cbits/time-osx.c
@@ -1,20 +1,11 @@
#include <mach/mach.h>
-#include <mach/mach_time.h>
+#include <time.h>
-static mach_timebase_info_data_t timebase_info;
-static double timebase_recip;
-
-void criterion_inittime(void)
-{
- if (timebase_recip == 0) {
- mach_timebase_info(&timebase_info);
- timebase_recip = (timebase_info.denom / timebase_info.numer) / 1e9;
- }
-}
+void criterion_inittime(void) {}
double criterion_gettime(void)
{
- return mach_absolute_time() * timebase_recip;
+ return clock_gettime_nsec_np(CLOCK_UPTIME_RAW) / 1e9;
}
static double to_double(time_value_t time)Output from the sanity test suite:
Running 1 test suites...
Test suite sanity: RUNNING...
sanity: benchmarking fib/fib 10
time 1.464 μs (1.463 μs .. 1.465 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 1.464 μs (1.463 μs .. 1.465 μs)
std dev 1.916 ns (1.555 ns .. 2.511 ns)
benchmarking fib/fib 22
time 1.016 ms (1.013 ms .. 1.020 ms)
1.000 R² (1.000 R² .. 1.000 R²)
mean 1.016 ms (1.015 ms .. 1.017 ms)
std dev 2.628 μs (1.599 μs .. 4.530 μs)
benchmarking length . filter/string
time 2.077 μs (2.076 μs .. 2.078 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 2.083 μs (2.079 μs .. 2.090 μs)
std dev 17.76 ns (10.29 ns .. 26.96 ns)
benchmarking length . filter/bytestring
time 664.9 ns (664.7 ns .. 665.2 ns)
1.000 R² (1.000 R² .. 1.000 R²)
mean 664.9 ns (664.8 ns .. 665.2 ns)
std dev 547.0 ps (319.4 ps .. 963.9 ps)
OK (20.47s)
All 1 tests passed (20.47s)
Test suite sanity: PASSFantastic. I've opened #240 using the patch in https://github.com/haskell/criterion/issues/238#issuecomment-870983679. As a sanity check: @carlocab, can you confirm that you can run the test suite on an Intel-based Mac using the patch in #240? You mentioned having some trouble in https://github.com/haskell/criterion/issues/238#issuecomment-865740867, so I want to make extra sure that the issue is resolved now.
csstaub
commented
3 years ago I can confirm that branch T238 with the patch compiles & executes the sanity test suite on an Intel-based MacBook @RyanGlScott, but shows some warnings about inflated variance:
Running 1 test suites...
Test suite sanity: RUNNING...
sanity: benchmarking fib/fib 10
time 2.441 μs (2.375 μs .. 2.498 μs)
0.995 R² (0.991 R² .. 0.998 R²)
mean 2.477 μs (2.417 μs .. 2.626 μs)
std dev 305.8 ns (132.2 ns .. 577.9 ns)
variance introduced by outliers: 92% (severely inflated)
benchmarking fib/fib 22
time 1.676 ms (1.648 ms .. 1.703 ms)
0.993 R² (0.985 R² .. 0.998 R²)
mean 1.791 ms (1.751 ms .. 1.901 ms)
std dev 211.1 μs (99.65 μs .. 438.7 μs)
variance introduced by outliers: 76% (severely inflated)
benchmarking length . filter/string
time 2.213 μs (2.168 μs .. 2.248 μs)
0.998 R² (0.997 R² .. 0.999 R²)
mean 2.176 μs (2.156 μs .. 2.197 μs)
std dev 68.84 ns (56.52 ns .. 93.13 ns)
variance introduced by outliers: 42% (moderately inflated)
benchmarking length . filter/bytestring
time 1.646 μs (1.632 μs .. 1.661 μs)
0.999 R² (0.999 R² .. 1.000 R²)
mean 1.663 μs (1.651 μs .. 1.676 μs)
std dev 40.25 ns (33.70 ns .. 50.34 ns)
variance introduced by outliers: 30% (moderately inflated)
OK (20.69s)
All 1 tests passed (20.69s)
Test suite sanity: PASSSticking with rdtsc on x86/x86-64 and using the mach_absolute_time() only as a fallback for arm64 machines seems to reduce the variance (tested a few times, consistently better this way):
diff --git a/criterion-measurement/cbits/cycles.c b/criterion-measurement/cbits/cycles.c
index 73ba7f9..51074a3 100644
--- a/criterion-measurement/cbits/cycles.c
+++ b/criterion-measurement/cbits/cycles.c
@@ -9,6 +9,15 @@ StgWord64 criterion_rdtsc(void)
return ((StgWord64) lo) | (((StgWord64) hi)<<32);
}
+#elif darwin_HOST_OS
+
+#include <mach/mach_time.h>
+
+StgWord64 criterion_rdtsc(void)
+{
+ return mach_absolute_time();
+}
+
#elif linux_HOST_OS
/*Sample output:
Running 1 test suites...
Test suite sanity: RUNNING...
sanity: benchmarking fib/fib 10
time 2.272 μs (2.254 μs .. 2.297 μs)
0.999 R² (0.998 R² .. 1.000 R²)
mean 2.310 μs (2.289 μs .. 2.340 μs)
std dev 85.75 ns (62.74 ns .. 115.8 ns)
variance introduced by outliers: 50% (moderately inflated)
benchmarking fib/fib 22
time 1.647 ms (1.636 ms .. 1.665 ms)
0.999 R² (0.998 R² .. 1.000 R²)
mean 1.657 ms (1.650 ms .. 1.668 ms)
std dev 31.66 μs (23.24 μs .. 47.75 μs)
benchmarking length . filter/string
time 2.100 μs (2.090 μs .. 2.108 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 2.117 μs (2.106 μs .. 2.146 μs)
std dev 54.42 ns (27.29 ns .. 104.4 ns)
variance introduced by outliers: 32% (moderately inflated)
benchmarking length . filter/bytestring
time 1.574 μs (1.567 μs .. 1.581 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 1.572 μs (1.567 μs .. 1.578 μs)
std dev 18.70 ns (13.40 ns .. 26.06 ns)
OK (20.70s)
All 1 tests passed (20.70s)
Test suite sanity: PASSOops, I missed my notifications for this thread. Thanks for stepping in for me here, @csstaub. Is there any further testing I need to do? Looks like @csstaub also did the testing on x86-64.
RyanGlScott
commented
3 years ago Thanks for the very thorough testing, @csstaub! I'm pretty confident that #240 is the right fix now. Also, I wouldn't worry too much about the variance reported—this is an extremely simplistic microbenchmark that runs quickly enough that it's somewhat susceptible to variance from outlying measurements.
For the sake of uniformity across all versions of macOS, I'm going to always use mach_absolute_time as the implementation of criterion_rdtsc. If this proves to be an issue later, we can always refine things in a future patch.
I've uploaded criterion-measurement-0.1.3.0 to Hackage, which should resolve everything. If not, please reopen this issue.
Hi there, Homebrew maintainer here. Since GHC 8.10.5 now works on Apple Silicon, we're trying to build some GHC-dependent packages for it. We've failed to build git-annex and that seems to be from a failure to build one of its (recursive) dependencies, criterion-measurement.
This is the error log:
The warning about the LLVM version appears to be spurious. It seems that the released version of criterion-measurement just doesn't recognise the attempt to build for Darwin on ARM64/AArch64.
Build logs for git-annex are available at https://github.com/Homebrew/homebrew-core/actions/runs/958143882, and the error from criterion-measurement starts at https://github.com/Homebrew/homebrew-core/runs/2878261603?check_suite_focus=true#step:6:1653.
Assistance would be appreciated.