Add benchmarks

[brandonchinn178]

Can use the Continuous Benchmark GitHub action to track performance over time:

• Using customSmallerIsBetter template
• The Haskell criterion library for benchmarks already has a --template json option
• Just need to write a converter from foo.json to the benchmark.json format that customSmallerIsBetter expects (break out into separate library?)

[willbasky]

What about benchmarks against tomland?

[brandonchinn178]

Sure, that would be good too

[arp242]

I added toml-reader over here: https://arp242.github.io/toml-test-matrix, which also includes some benchmarks.

Adding the test verification wasn't too hard as toml-reader already includes a script for this, but I'm having a bit more trouble with the benchmark program; I added (adapted from toml-parser):

import Control.Exception (evaluate)
import Data.Time (diffUTCTime, getCurrentTime)
import System.Environment (getArgs)
import TOML.Parser (parseTOML)

main :: IO ()
main =
 do args <- getArgs
    filename <- case args of
      [filename] -> pure filename
      _ -> fail "Usage: perf <file.toml>"
    txt <- readFile filename
    evaluate (length txt) -- readFile uses lazy IO, force it to load
    start <- getCurrentTime
    evaluate (parseTOML txt)
    stop <- getCurrentTime
    print (stop `diffUTCTime` start)

But it's always very fast:

% ls -lh a.toml
-rw-r--r-- 1 martin martin 4.9M Dec 11 05:19 a.toml

% ./perf-bin a.toml
0.000000079s

79 nanoseconds to parse a 5M file would be quite impressive.

It's more realistic for toml-parser – I guess something in toml-reader evaluates more lazy?

It's probably a very simple fix, but I don't really know Haskell so not so easy to fix for me. Would you mind telling me how to fix it? 😅

[brandonchinn178]

@arp242 Well it seems like you're passing one argument to parseTOML, when it takes two arguments. So evaluate forces the thunk resulting in a function that's never actually called. You probably want parseTOML "input.toml" txt

[arp242]

Ah right; it now fails with:

perf/perf.hs:15:34: error:
    • Couldn't match type ‘[Char]’
                     with ‘text-2.0.1:Data.Text.Internal.Text’
      Expected: text-2.0.1:Data.Text.Internal.Text
        Actual: String
    • In the second argument of ‘parseTOML’, namely ‘txt’
      In the first argument of ‘evaluate’, namely
        ‘(parseTOML filename txt)’
      In a stmt of a 'do' block: evaluate (parseTOML filename txt)
   |
15 |     evaluate (parseTOML filename txt)
   |                                  ^^^

I tried a few things to get the type of txt correct, but it's all failing on me 😅

[brandonchinn178]

-- top of file
import qualified Data.Text as Text

-- here
parseTOML filename (Text.pack txt)

[arp242]

Thanks, I got it working.