Rust is notorious for slow compile times so i suspect many devs use debug builds most of the time. If an ECS is fast with optimizations but unusably slow without them, it would be good to find out early. For example, in my unscientific expriments, hecs and legion were about the same in release mode but legion was several times slower is debug mode (don't have exact numbers anymore).
I suggest this bench suite should test 3 configurations:
Everything in release mode (this is what it does currently I believe)
Everything in debug mode
Deps (which in normal usage change infrequently) in release mode but the main crate (typically the gamelogic which changes often) in debug mode - this is what macroquad suggests (expand Tips):
Rust is notorious for slow compile times so i suspect many devs use debug builds most of the time. If an ECS is fast with optimizations but unusably slow without them, it would be good to find out early. For example, in my unscientific expriments, hecs and legion were about the same in release mode but legion was several times slower is debug mode (don't have exact numbers anymore).
I suggest this bench suite should test 3 configurations: