Open brandonchinn178 opened 12 months ago
@brandonchinn178 expected: yes. PR welcome: also yes. I wouldn't mind supporting both.
FWIW: there's a really good reason why the default is true
and you probably want to use sequential
instead.
Do you want to show me where you're using sydtest
? Maybe I can make a helpful suggestion.
I understand why the default is true; in general, I agree that randomizing the test order is good to prevent a implicit dependency on the order of tests.
In my case, all my tests are unit tests and I want the output order to match the order in the source code, to match the spec. It's just aesthetics, so I don't think sequential
is the right thing here.
Also, can you explain why this is expected? I do see this: https://github.com/NorfairKing/sydtest/blob/a3ba5f3096c3dcc58920f61ba8a45191d9020eb4/sydtest/src/Test/Syd/OptParse.hs#L268-L269
And the help text does say it's allowed:
# any of
[ randomise-execution-order: # optional
# Randomise the execution order of the tests in the test suite
<boolean>
, randomize-execution-order: # optional
# American spelling
<boolean>
]
--no-randomize-execution-order
also works, it just doesn't work in the config file
@brandonchinn178 It looks like I made two mistakes in one message:
doNotRandomiseExecutionOrder
instead of sequential
.It also looks like there's an issue with the parsing.
I'm investigating now if it's autodocodec
that's broken.
@brandonchinn178 I just managed to reproduce this failure. Autodocodec has tests against this, so I'll continue digging.
I've added another test to autodocodec and haven't found the issue, but now I think I have an idea what the issue is.
I figured out why this happened:
It turns out that I misunderstood (my own) autodocodec API and caused this bug. I'll have to revisit the autodocodec API to make this kind of issue solveable.
randomize-execution-order: false
doesn't seem to work in the config file, butrandomise-execution-order: false
does. Is that expected?