dimforge / bevy_rapier

Official Rapier plugin for the Bevy game engine.
https://rapier.rs
Apache License 2.0
1.28k stars 259 forks source link

Added initial benchmarks #551

Closed Vrixyz closed 4 months ago

Vrixyz commented 4 months ago

Objective

bevy_rapier currently has no benchmarks, making it difficult to be confident with refactoring, or consider performance improvements. This PR aims to implement foundation to address this.

Avian comparison

I doubt the benchmarks are comparable with avian's, as the time management from avian is customized, it might not be running full speed in avian.

bevy_rapier

- results as of https://github.com/dimforge/bevy_rapier/pull/551/commits/6e6131839b10229c59950fef827a64a9722fbbea: ``` Benchmarking cubes 3x3, 30 steps: Warming up for 3.0000 s Warning: Unable to complete 100 samples in 20.0s. You may wish to increase target time to 22.0s, enable flat sampling, or reduce sample count to 60. cubes 3x3, 30 steps time: [2.8347 ms 2.8435 ms 2.8534 ms] Found 11 outliers among 100 measurements (11.00%) 1 (1.00%) low mild 3 (3.00%) high mild 7 (7.00%) high severe cubes 5x5, 30 steps time: [8.0240 ms 8.0784 ms 8.1379 ms] Found 6 outliers among 100 measurements (6.00%) 5 (5.00%) high mild 1 (1.00%) high severe cubes 10x10, 30 steps time: [105.89 ms 107.98 ms 110.40 ms] Found 8 outliers among 100 measurements (8.00%) 4 (4.00%) high mild 4 (4.00%) high severe ```

Avian

- results as of https://github.com/Jondolf/bevy_xpbd/commit/81290423e146264120cf9711af716f6faf669717 ``` cubes 3x3, 30 steps time: [49.782 ms 50.203 ms 50.708 ms] change: [+1638.8% +1656.7% +1676.8%] (p = 0.00 < 0.05) Performance has regressed. Found 5 outliers among 100 measurements (5.00%) 2 (2.00%) high mild 3 (3.00%) high severe cubes 5x5, 30 steps time: [109.87 ms 110.38 ms 111.03 ms] change: [+1254.3% +1266.3% +1278.4%] (p = 0.00 < 0.05) Performance has regressed. Found 12 outliers among 100 measurements (12.00%) 7 (7.00%) high mild 5 (5.00%) high severe Benchmarking cubes 10x10, 30 steps: Warming up for 3.0000 s Warning: Unable to complete 100 samples in 20.0s. You may wish to increase target time to 39.4s, or reduce sample count to 50. cubes 10x10, 30 steps time: [390.36 ms 391.82 ms 393.67 ms] change: [+254.70% +262.86% +270.08%] (p = 0.00 < 0.05) Performance has regressed. Found 9 outliers among 100 measurements (9.00%) 3 (3.00%) high mild 6 (6.00%) high severe ```


Suspicious bad performance

I looked into pipeline.counters.step_time.time(), originally to compare with the total update time, in order to get an idea of the overhead from bevy/bevy_rapier, but I had surprising results:

Flamegraph

[flamegraph.zip](https://github.com/user-attachments/files/16095207/flamegraph.zip) ![flamegraph](https://github.com/dimforge/bevy_rapier/assets/2290685/5fbf491c-9af9-4624-920d-ce1eb970523e)

^ above numbers reported with f3a3d79d6314e9

sebcrozet commented 4 months ago

Both approaches make sense: using divan for smaller benchmarks to quickly detect performance regression; and the custom bench for larger simulations and per-step analysis (ideally outputting some sort of graph).

Is there any way to initialize the scene, and then have divan benchmark each frame of the bevy world update?

Vrixyz commented 4 months ago

Is there any way to initialize the scene, and then have divan benchmark each frame of the bevy world update?

I did implement that with 2cda622