search

UDST / choicemodels

Python library for discrete choice modeling
https://udst.github.io/choicemodels
BSD 3-Clause "New" or "Revised" License
74 stars 33 forks source link

Improving probabilistic tests #42

Open smmaurer opened 6 years ago

smmaurer commented 6 years ago

Some of our unit tests are probabilistic, so they sometimes fail.

The tests are intended to verify that things like sampling weights are being applied correctly. But since the sampling is random, occasionally the tests don't pass. We should make this more sophisticated :slightly_smiling_face:

Here's an example of a probabilistic test: https://github.com/UDST/choicemodels/blob/master/tests/test_mct.py#L46

And a failure that was resolved simply by re-running the test: https://travis-ci.org/UDST/choicemodels/jobs/427893157

bridwell commented 6 years ago

I wrote a simple context manager that manages the numpy and python random states. Might be applicable here:

https://github.com/AZMAG/smartpy_core/blob/master/smartpy_core/wrangling.py#L12 https://github.com/AZMAG/smartpy_core/blob/master/smartpy_core/tests/test_wrangling.py#L163