This PR is the result of an investigation into why eval tests were so slow in #81. Turned out the prelude was being evaluated once for each test (and also once for every request in vp-calc and other users of inferno). This PR refactors the mkInferno function to be an IO action that evaluates the prelude once and shares the computation. (Thanks to @Daniel-Diaz for the assist)
Before:
INFERNO_EXE=/home/sid/inferno/dist-newstyle/build/x86_64-linux/ghc-9.2.5/inferno-core-0.5.0.0/x/inferno/build/inferno/inferno HSPEC_PRINT_SLOW_ITEMS=10 cabal test inferno-tests --test-show-details=direct
...
Finished in 29.5532 seconds
434 examples, 0 failures
Slow spec items:
test/Parse/Spec.hs:80:3: /pretty printing/parsing/parseExpr and pretty are inverse up to normalizeExpr/ (7368ms)
src/Test/Aeson/Internal/ADT/GoldenSpecs.hs:78:3: /JSON encoding of VCObject/produces the same JSON as is found in golden/VCObject/VCModule.json/ (1784ms)
src/Test/Aeson/Internal/ADT/GoldenSpecs.hs:78:3: /JSON encoding of VCObject/produces the same JSON as is found in golden/VCObject/VCFunction.json/ (1555ms)
src/Test/Aeson/Internal/ADT/RoundtripSpecs.hs:55:5: /JSON encoding of VCObject/allows to encode values with aeson and read them back/ (201ms)
test/Eval/Spec.hs:375:7: /evaluate/"(Array.reduce (fun x y -> x + max 0 y) 0 ((-3) .. 3)) == (Array.reduceRight (fun x y -> y + max 0 x) 0 ((-3) .. 3))" should evaluate to #true/ (91ms)
test/Eval/Spec.hs:375:7: /evaluate/"open Time in let ?now = (toTime (hours 66666)) in hoursBefore ?now 44 == ?now - (hours 44)" should evaluate to #true/ (85ms)
test/Eval/Spec.hs:375:7: /evaluate/"Array.findFirstSome [None, Some 3.0, None, Some 4.0]" should evaluate to Some 3.0/ (85ms)
test/Eval/Spec.hs:375:7: /evaluate/"[x | x <- 1 .. 10, if x % 2 == 0]" should evaluate to [2,4,6,8,10]/ (84ms)
test/Eval/Spec.hs:375:7: /evaluate/"open Time in let ?now = (toTime (minutes 66666)) in minutesBefore ?now 44 == ?now - (minutes 44)" should evaluate to #true/ (84ms)
test/Eval/Spec.hs:375:7: /evaluate/"open Time in let ?now = (toTime (weeks 66666)) in weeksBefore ?now 44 == ?now - (weeks 44)" should evaluate to #true/ (83ms)
After:
INFERNO_EXE=/home/sid/inferno/dist-newstyle/build/x86_64-linux/ghc-9.2.5/inferno-core-0.5.0.0/x/inferno/build/inferno/inferno HSPEC_PRINT_SLOW_ITEMS=10 cabal test inferno-tests --test-show-details=direct
...
Finished in 11.9918 seconds
434 examples, 0 failures
Slow spec items:
test/Parse/Spec.hs:80:3: /pretty printing/parsing/parseExpr and pretty are inverse up to normalizeExpr/ (6470ms)
src/Test/Aeson/Internal/ADT/GoldenSpecs.hs:78:3: /JSON encoding of VCObject/produces the same JSON as is found in golden/VCObject/VCModule.json/ (1759ms)
src/Test/Aeson/Internal/ADT/GoldenSpecs.hs:78:3: /JSON encoding of VCObject/produces the same JSON as is found in golden/VCObject/VCFunction.json/ (1534ms)
src/Test/Aeson/Internal/ADT/RoundtripSpecs.hs:55:5: /JSON encoding of VCObject/allows to encode values with aeson and read them back/ (69ms)
test/Parse/Spec.hs:250:7: /pretty printing/parsing/parsing array builder/should succeed for "[(x,y) | x <- someList, y <- otherList]"/ (31ms)
test/Eval/Spec.hs:53:11: /evaluate/"(Array.reduce (fun x y -> x + max 0 y) 0 ((-3) .. 3)) == (Array.reduceRight (fun x y -> y + max 0 x) 0 ((-3) .. 3))" should evaluate to #true/ (29ms)
test/Eval/Spec.hs:53:11: /evaluate/"match [1.2, 3, 3] with { | [x, y, z] -> 2*x+3*y+z | _ -> 3 }" should evaluate to 14.4/ (22ms)
test/Eval/Spec.hs:53:11: /evaluate/"open Time in let ?now = (toTime (days 66666)) in daysBefore ?now 44 == ?now - (days 44)" should evaluate to #true/ (22ms)
test/Eval/Spec.hs:53:11: /evaluate/"Array.findLastSome [None, Some 3.0, None, Some 4.0]" should evaluate to Some 4.0/ (22ms)
test/Eval/Spec.hs:53:11: /evaluate/"Array.findFirstSome [None, Some 3.0, None, Some 4.0]" should evaluate to Some 3.0/ (22ms)
The difference was even starker after adding the primitives of #81 -- each eval test was ~1000ms, and this PR brings it down to ~20ms!
This PR is the result of an investigation into why eval tests were so slow in #81. Turned out the prelude was being evaluated once for each test (and also once for every request in vp-calc and other users of inferno). This PR refactors the
mkInferno
function to be an IO action that evaluates the prelude once and shares the computation. (Thanks to @Daniel-Diaz for the assist)Before:
After:
The difference was even starker after adding the primitives of #81 -- each eval test was ~1000ms, and this PR brings it down to ~20ms!