wai: document how to use the same application context for multiple tests

[guibou]

Using sydtest-wai, how do I use the same application (and keep its current context) between multiples it, possibly forcing the ordering of tests using sequential?

I would like some kind of aroundAll in order to start once the application and setup the WaiClient and some kind of beforeWith in order to change / setup the client for a subset of test. My understanding is that I need to pass a WaiClient as an "outer" resource and use it in my test cases with runWaiClientM

However, the api does not provide any helpers in order to create a WaiClient as an "outer" resource.

Should I create them (I can open an MR with that + additionnal documentation) or am I in the wrong direction ?

What I'm currently doing is:

A SetupFunc in order to setup my app:

setupWaiApp
  :: SetupFunc Application
  -> SetupFunc (WaiClient ())
setupWaiApp setupApp = do
  man <- liftIO $ HTTP.newManager HTTP.defaultManagerSettings
  app <- setupApp
  p <- applicationSetupFunc app
  let client =
        WaiClient
          { waiClientManager = man,
            waiClientEnv = (),
            waiClientPort = p
          }
  pure client

Two utilities function:

withSharedWaiApp
  :: SetupFunc Application
  -> TestDefM (WaiClient () : outers) () ()
  -> TestDefM outers () ()
withSharedWaiApp srv spec = do
  setupAroundAll (setupWaiApp srv) spec

withWaiApp
  ::  SetupFunc Application
  -> TestDefM outers (WaiClient ()) ()
  -> TestDefM outers () ()
withWaiApp srv spec = do
  setupAround (setupWaiApp srv) spec

Note how withWaiApp behaves as what is currently available with sydtest-wai: it sets a WaiClient () as inner resource: each "it" will create a new Application from the setup func and will run in isolation. However, withSharedWaiApp setups a shared WaiClient () as "outer" resource. The same client will be shared between the different it is the test group.

I then have:

itWaiShared
  :: String
  -> WaiClientM env ()
  -> TestDefM (WaiClient env : outers) () ()
itWaiShared desc spec = itWithOuter desc $ \client -> runWaiClientM client spec

Which acts as a special version of itWithOuter and runs the inner WaiClientM.

Now my code can look like:

spec :: HasCallStack => TestDefM outers () ()
spect = do
  -- These tests are independent and it is OK to run them in isolation with a new app each time
  withWaiApp setupApp $  do
    runTestScenariiV2 scenarios
    runTrialPipelineScenario
    runVpopGeneratorsTests

  -- These tests run with the same app
  withSharedWaiApp setupApp $ do
       -- We run them sequentially because we want to be sure that the first test is run BEFORE the setup of the resource
      sequential $ do
          itWaiShared "test that the resource does not exists" $ do
                _ -- query the server and wait for a 404
          describe "with existing resource" $ do
               -- We create the resource and now we can run different "supposed to be independent" queries in parallel
               beforeAllWith createResource $ do
                     parallel $ do
                           _ -- others tests

So I have a few questions:

• Am I completely wrong? What did I miss?
• Is there a way to transform an outer resource to inner resource on the fly, so I could get rid of the itWaiShared
• Are you interested by an MR with these helpers + documentation?

[NorfairKing]

Using sydtest-wai, how do I use the same application (and keep its current context) between multiples it, possibly forcing the ordering of tests using sequential?

The intention is that that's not possible. The intention is that tests are isolated so you can run them in parallel. If you want that you'll need some variable like an IORef or more likely a TVar.

Is there a way to transform an outer resource to inner resource on the fly, so I could get rid of the itWaiShared

Yes you can use beforeWith' to make the outer resource an inner resource. But you'll still need a variable wrapper to make them modifiable of course.

Basically what you're asking is not explicitly supported because it's a bad idea. If you want to run multiple tests or sessions, run them in a single it.

[guibou]

Yes you can use beforeWith' to make the outer resource an inner resource. But you'll still need a variable wrapper to make them modifiable of course.

Thank you, indeed, this helps.

Basically what you're asking is not explicitly supported because it's a bad idea. If you want to run multiple tests or sessions, run them in a single it.

Yes, after writing everything and realizing that it saves 10s over our 15 minutes unit test suit run, I realized that simpler is better and removed everything.

I'm keeping the idea (and beforeWith') in order to do that in limited context where it may really be relevant.

Closing. Thank you.