Currently, Avian has a cross-platform determinism test that produces insta snapshots, which are compared between platforms in CI with GitHub Actions.
However, these snapshots are quite large, and always pollute diffs in PRs (such as this one) whenever changes affecting simulation behavior are made. Insta must also be installed to generate these snapshots, making it annoying for new contributors to deal with. Overall, it's a hassle.
The test itself is also very simplistic and only tests collisions.
Instead of using snapshots, we should just run the simulation for a while, and compute a single transform hash based on the position and rotation of all bodies. This is much simpler and more convenient.
Solution
Add a 2D test for cross-platform determinism, simulating pairs of objects constrained via revolute joints falling to the ground. After 500 steps, a transform hash is computed and compared against the expected value. If they don't match, the test will fail with a message indicating that the expected hash should be updated if the change in behavior was intended.
test tests::determinism_2d::cross_platform_determinism_2d ... FAILED
failures:
---- tests::determinism_2d::cross_platform_determinism_2d stdout ----
thread 'tests::determinism_2d::cross_platform_determinism_2d' panicked at crates\avian2d\../../src\tests\determinism_2d.rs:63:5:
Expected hash 0xa1c6b39, found hash 0xa38b5132 instead.
If changes in behavior were expected, update the hash in src/tests/determinism_2d.rs on line 61.
This test is based on Box2D's FallingHinges test and sample.
There is also a new determinism_2d example, which is a visual representation of the test.
While making this, I stumbled upon some bugs as well:
The max correction for revolute joints is way too small, often causing explosive behavior when it is exceeded.
When the PhysicsSchedule runs, Time<Substeps> is only set once the substepping loop starts. Earlier in the schedule, the previous value is used. However, some systems like update_contact_softness might need the substep time before that. Using the previous value can lead to seemingly indeterministic results e.g. when resetting scenes.
Objective
Currently, Avian has a cross-platform determinism test that produces insta snapshots, which are compared between platforms in CI with GitHub Actions.
However, these snapshots are quite large, and always pollute diffs in PRs (such as this one) whenever changes affecting simulation behavior are made. Insta must also be installed to generate these snapshots, making it annoying for new contributors to deal with. Overall, it's a hassle.
The test itself is also very simplistic and only tests collisions.
Instead of using snapshots, we should just run the simulation for a while, and compute a single transform hash based on the position and rotation of all bodies. This is much simpler and more convenient.
Solution
Add a 2D test for cross-platform determinism, simulating pairs of objects constrained via revolute joints falling to the ground. After 500 steps, a transform hash is computed and compared against the expected value. If they don't match, the test will fail with a message indicating that the expected hash should be updated if the change in behavior was intended.
This test is based on Box2D's
FallingHingestest and sample.There is also a new
determinism_2dexample, which is a visual representation of the test.https://github.com/user-attachments/assets/2d212fe7-14b3-4dff-b1d0-96c531da6918
While making this, I stumbled upon some bugs as well:
PhysicsScheduleruns,Time<Substeps>is only set once the substepping loop starts. Earlier in the schedule, the previous value is used. However, some systems likeupdate_contact_softnessmight need the substep time before that. Using the previous value can lead to seemingly indeterministic results e.g. when resetting scenes.I have fixed both of these issues.