We want property testing to "eventually" catch errors, but this shouldn't come at the cost of non-deterministic CI. Ideally we make this deterministic and run property testing nightly or some similar solution.
Describe alternatives you've considered
Status quo.
Run property tests long enough that we convince ourselves future failures happen less than 1% of the time (or whatever bound we choose) .
Additional context
No response
Is this something that you'd be interested in working on?
[X] 👋 I may be able to implement this feature request
Category
Other
Describe the feature you'd like to request
CI is non-deterministic. This failure: https://github.com/cedar-policy/cedar/actions/runs/7464644346/job/20312068201?pr=576 is the same issue Victor noticed here: https://github.com/cedar-policy/cedar-java/pull/72/files#r1437156141 and only sometimes occurs (PR that doesn't catch issue: https://github.com/cedar-policy/cedar/actions/runs/7465002067/job/20313236263?pr=577)
We want property testing to "eventually" catch errors, but this shouldn't come at the cost of non-deterministic CI. Ideally we make this deterministic and run property testing nightly or some similar solution.
Describe alternatives you've considered
Status quo.
Run property tests long enough that we convince ourselves future failures happen less than 1% of the time (or whatever bound we choose) .
Additional context
No response
Is this something that you'd be interested in working on?