search

kitware-resonant / dkc-next

Apache License 2.0
3 stars 0 forks source link

Use pytest-randomly to set a consistent random seed #126

Closed jbeezley closed 3 years ago

jbeezley commented 3 years ago

Should prevent the random testing errors that we get due to name constraints.

jbeezley commented 3 years ago

You can use -p no:randomly if you really want no seed.

brianhelba commented 3 years ago

I think a lot of the benefit just comes from having values which we didn't choose, which prevents some amount of overfitting to test cases. Additionally, having a large number of different values is nice for the same reasons, and we do loose that aspect a bit here.

The alternative to this change is to update fields which are in danger of collision to use a value of a Faker + Sequence, which will prevent collisions but give us the full diversity of the random pool across other fields.

brianhelba commented 3 years ago

I'd advocate my approach (Faker + Sequence), but I'm open to others' thoughts. I imagine that most fields can remain only with Faker, it's just the unique + large cardinality ones that'd need to be changed.