skinniderlab / CLM

MIT License
0 stars 0 forks source link

setting seed in a fixture at the function level #225

Closed vineetbansal closed 3 months ago

vineetbansal commented 3 months ago

Implemented a fixture that is called automatically before each test function call, which sets the random seed using our set_seed function.

This is not just a convenience such that we don't have to use set_seed(0) in individual test functions, though I'm also leveraging that in this PR, but is also important to make sure that one test function using set_seed doesn't affect other test functions' behavior (since all test functions run in the same process).

Currently any tests that use set_seed will affect the behavior of any tests that come after it and don't use a set_seed, which makes it impossible to reorder tests, introduce new tests, delete tests etc without behavior being affected across the board in unpredictable ways.

Individual tests can still use set_seed (like in the several cases of set_seed(5831) that I see in the code) if they want to.