NorfairKing / sydtest

A modern testing framework for Haskell with good defaults and advanced testing features.
113 stars 25 forks source link

Multiples `sydTest` segfault in parallel #91

Open guibou opened 4 weeks ago

guibou commented 4 weeks ago

sydTest (and most related functions, such as sydTestResults, ...) cannot be run twice in a concurrent environment because they internally relies on non thread safe features.

Especially (but maybe not exhaustive):

Since this library is built on top of the curses interface, it is not thread-safe.

The following code will hence hapilly segfault:

import Test.Syd

main = sydTest $ do
  it "test 1" $ do
    sydTest $ it "inner" $ 1 `shouldBe` 1
  it "test 2" $ do
    sydTest $ it "inner" $ 1 `shouldBe` 1

Built with -O2 -threaded and run with --continuous +RTS -N and you'll eventually get a segfault.

This is not a problem in traditionnal uses of sydtest, however, it is a problem when using sydtest in the sydtest test suite itself. That's what I tried when working on #80 (especially, see comment https://github.com/NorfairKing/sydtest/issues/80#issuecomment-2286446001).

Note that a workaround is to use "internal" functions which does not do the detection, such as:

  specForest <- execTestDefM defaultSettings spec
  resultForest <- runSpecForestAsynchronously defaultSettings 8 specForest

I wanted to track in a ticket that sydTest cannot be run concurrently, so if someone hit a segmentation fault, maybe they'll find this ticket. I'm unsure sydtest should be fixed (e.g. the fix would be highly invasive and complex), however maybe the documentation may list the "unsafe" functions.

NorfairKing commented 4 weeks ago

Huh, I didn't expect a segfault. I thought one of them would fail while the other passed or so. Thanks for pointing this out!