UnkindPartition / tasty

Modern and extensible testing framework for Haskell
638 stars 108 forks source link

Prepare test deps based on command line options in IO #329

Closed robinp closed 2 years ago

robinp commented 2 years ago

Hello! My usecase is to set up some runtime deps for the tests ahead of running them. For example, I would like to open a postgres connection pool, and pass in the pool to the tests, instead of having the tests create a connection or pool each time when they run.

Normally I use optparse-applicative to pass in arguments to my programs. I tried this for the test binary as well, so I could parse the connection spec, create the pool and just pass as arg to tasty like defaultMain (tests conn).

The problem with that is tasty would err out due to arguments present that it doesn't know.

Some options I thought about: a) After running my parsing, remove the used options, so tasty only sees the leftovers. But then if I eventually would want to pass tasty options, my parser would complain (unless I can instruct optparse-applicative to skip unknown options, which I didn't find a way to)

b) Use tasty's option parsing instead my standard way. But askOption is not run in IO, so I can get an argument, but can't instantiate the pool ahead of time. Could there be a variant of askOption running in IO maybe?

c) Could tasty expose helpers where caller can access the optparse-applicative parser it generates? Then I could run that as part of my larger parser, and pass the results back to tasty.

d) Some hack with globals and unsafePerformIO.

Do you have any suggestions? I would actually prefer c) if that is possible / makes sense, but open to other suggestions as well. Thank you!

Bodigrim commented 2 years ago

@robinp see https://hackage.haskell.org/package/tasty-1.4.2.1/docs/Test-Tasty-Runners.html#g:6

robinp commented 2 years ago

@Bodigrim thank you for the pointer! Got it working along these lines:

import           Test.Tasty.Ingredients (ingredientsOptions)
import           Test.Tasty.Options (uniqueOptionDescriptions, OptionSet)
import           Test.Tasty.Runners (coreOptions, optionParser, tryIngredients)

data Config = Config { cfgPgConnString :: Text, cfgTasty :: OptionSet }

configParser :: O.Parser OptionSet -> O.Parser Config
configParser topt = Config
    <$> ....
    <*> topt

main :: IO ()
main = let (_warns, p) = optionParser . uniqueOptionDescriptions $ coreOptions ++ ingredientsOptions defaultIngredients
       in myMainWithOpts (configParser p) $ \cfg -> do
    conn <- PG.connectPostgreSQL (toS (cfgPgConnString cfg))
    case tryIngredients defaultIngredients (cfgTasty cfg) (tests conn) of
      Nothing -> panic "argh, nothing from ingredients"
      Just act -> do
        ok <- act
        if ok then exitSuccess else exitFailure