shaobo-he-aws
commented
3 weeks ago CI seems to stabilize. Close it for now.
Closed andrewmwells-amazon closed 3 weeks ago
shaobo-he-aws
commented
3 weeks ago CI seems to stabilize. Close it for now.
Category
Other
Describe the feature you'd like to request
CI is non-deterministic. This failure: https://github.com/cedar-policy/cedar/actions/runs/7464644346/job/20312068201?pr=576 is the same issue Victor noticed here: https://github.com/cedar-policy/cedar-java/pull/72/files#r1437156141 and only sometimes occurs (PR that doesn't catch issue: https://github.com/cedar-policy/cedar/actions/runs/7465002067/job/20313236263?pr=577)
We want property testing to "eventually" catch errors, but this shouldn't come at the cost of non-deterministic CI. Ideally we make this deterministic and run property testing nightly or some similar solution.
Describe alternatives you've considered
Status quo.
Run property tests long enough that we convince ourselves future failures happen less than 1% of the time (or whatever bound we choose) .
Additional context
No response
Is this something that you'd be interested in working on?