Strange performance issue with many colliders

[paholg]

I wanted a quick and dirty way of generating levels (with vertical walls, so I believe a heightfield will not work). So, I have a 4-color image, and I create a cuboid collider for each pixel, with color determining height -- the heights are "nothing", "floor", "short wall", and "wall". That code exists here, if you're curious: https://github.com/paholg/gam/blob/6ad7186ebbb1502a94f623f0c28e5f19c4785175/crates/engine/src/level.rs#L62-L120

However, I am noticing significant frame drops for a 100x100 grid. What's more, I'm not sure where those frames are going -- none of my CPU cores are close to maxed out.

I have run a few experiments, and pretty consistently see a frame drop from ~60 fps to half that, even when eliminating most of my game.

If there's any more information I can provide, I would be happy to, or if you'd like me to produce a minimal reproducible example, I can do that. Perhaps it would make sense to reproduce this without bevy_rapier, though I don't believe that's contributing.

Experiment

Experimental setup: I disabled rendering the terrain cubes (just in case that matters), as well as spawning characters and ran my game for 20 seconds with cargo flamegraph. The game is just some cuboids and some lights at this point, and a bunch of systems with empty queries. I did this with a 50x50 terrain grid, and a 100x100 grid.

With a 50x50 grid (2,500 cuboid colliders):

• FPS: ~54-70
• Flamegraph:
  • 3.52% of samples are in bevy_rapier3d::plugin::systems::step_simulation

With a 100x100 grid (10,000 cuboid colliders):

• FPS: ~19-34
• Flamegraph:
  • 11.33% of samples are in bevy_rapier3d::plugin::systems::step_simulation
  • I am also seeing some calls into bevy_rapier3d in the rendering pipeline here, which are not present with the 50x50 grid.

Observations:

• While much more time is spent in rapier with the larger grid, going from 3.5% to 11.33% should not result in a 50% frame drop, especially when none of my CPU cores are working especially hard.
• A lot of the time is spent in rapier3d::pipeline::query_pipeline::QueryPipeline::update_with_mode, for both setups. Basically all of that time is used by parry3d::partitioning::qbvh::GenericQbvh::clear_and_rebuild. This seems strange to me, as there only exist non-moving colliders attached to fixed rigid bodies. But I don't know enough about rapier/parry internals to know if it makes sense. Maybe there could be a flag to indicate a collider/fixed-body will never move, so this could be avoided, or maybe that suggestion makes no sense.

Questions

• Any thoughts on what's going on here? Maybe a memory bandwidth issue?
• Is it reasonable to want to build a floor and walls with this many colliders? If I were to continue this approach, I would want to combine groups of pixels into larger cuboids, at least, but I don't really want to spend that effort now as there's a good chance I'll end up doing something else long-term.

Flamegraphs

Note: GitHub does not seem to support nice navigation of these. If you'd like to inspect them, I recommend downloading then opening with a browser. Then, for example, searching for "rapier" will highlight everything from this library (and from bevy_rapier).

50x50 grid

100x100 grid

[sebcrozet]

The GenericQbvh::clear_and_rebuild function completely rebuilds the tree used for scene queries. So even if colliders don’t move, it is rebuilt in full, which explains the poor scaling you are observing. QueryPipeline::update_incremental has been introduced to avoid this problem by only rebuild based on a user-defined set of modified colliders.

[cybersoulK]

QueryPipeline::update_incremental (by passing Some(&mut query_pipeline) to Physics::step instead of manually calling update) is currently triggering a panic.

@Ralith i don't think it's related to converting an entity from Kinematic to Dynamic, since in my scene, i only have Static and Dynamic rigid bodies and i don't do any conversion whatsoever, i am guessing it's due to the high quantity of rigid bodies + attached colliders.

https://github.com/dimforge/rapier/issues/532 The issue got solved by this PR. Please merge it! https://github.com/dimforge/parry/pull/169

But still, QueryPipeline::update_incremental has a huge issue, and should not be recommended? https://github.com/dimforge/rapier/issues/466

Also my benchmark of rapier3d and PhysX is likely suffering from these issue (a lot of static rigid bodies / with respective colliders) https://youtu.be/ufKnFjdmuyc

even with the PR, and using QueryPipeline::update_incremental, it didn't solve the performance issue. I would love to run the tests again if things improve in favour of Rapier3d, but i am not sure where the issue might be