Open reduz opened 11 months ago
A few boxes in the Math section can be ticked off :)
Can any checkbox be marked as done after #65 was merged?
Can any checkbox be marked as done after #65 was merged?
Yes, I've just done that :slightly_smiling_face:
Seems like some more benchmarks can be ticked off:
Implemented by #41:
Implemented by #47:
Implemented by #48:
Implemented by #49:
Implemented by #53:
Implemented by #54:
Missed this in my previous comment, but the scene nodes benchmarks also need to be ticked off, as they were implemented by #44.
Creation of benchmarks for the benchmarks server
This document is a compilation of benchmarks suggested by maintainers and contributors from all areas of the engine.
Our goal is to create a limited amount of benchmarks where we can track the progress and optimization improvements we do to the engine every day and how they reflect on benchmarks. Likewise it will help us better spot when we have a performance regression.
The benchmarks should be relatively short and running all the benchmarks should not take a very long time (an hour at much).
Because creating all these benchmarks is laborious, we are asking for help to community and contributors, so we can organize the creation of all these (and obviously suggest more, but we need to ensure the ones listed here are completed first).
For this, we have created a #benchmarks channel in our Rocket Chat (chat.godotengine.org). If you are interested in lending us a hand and implementing the benchmarks from this list, please feel free to join and say hi.
Ultimately the goal is that these benchmarks are contributed to this repository as pull requests.
Benchmark Methodologies
Depending on the type of benchmark, different methodologies need to be used:
Benchmark format
Benchmarks are written like this:
When a benchmark belongs to groups, the benchmark (graph) will be named after the group and contain all the plots of the individual benchmarks belonging to it, helping to contextualize and compare.
Otherwise, if no group is specified, the benchmark is standalone with a single plot.
List of Benchmarks
These are all the benchmarks that have to be created. As they are submitted, the checkboxes will be ticked.
Core
main
).Math
3D Rendering
CPU Benchmarks
[ ] π₯CPUπ₯ [Cull]: Static: Cull 10k objects
[ ] π₯CPUπ₯ [Cull]: Rotating: Cull 10k rotating objects
[ ] π₯CPUπ₯ Cull Directional Shadows : Cull 10k objects lit by directional shadow (4 splits)
[ ] π₯CPUπ₯ [Cull Omni Shadows]: Static: Cull 10k objects, 200 omni lights with shadows, static
[ ] π₯CPUπ₯ [Cull Omni Shadows]: Dynamic: Cull 10k objects, 200 omni lights with shadows, lights move around.
[ ] π₯CPUπ₯ [Cull Spot Shadows]: Static: Cull 10k objects, 200 omni lights with shadows, static
[ ] π₯CPUπ₯ [Cull Spot Shadows]: Dynamic: Cull 10k objects, 200 omni lights with shadows, lights move around.
[ ] π₯CPUπ₯ Lightmap probe influence : Simple scene baked with lightmaps (and using tons of probes, like 500), 1000 objects moving around receiving influence from probes.
GPU Benchmarks
[x] π©GPUπ© [GI,Lighting,Effects,AA,DOF]: Sponza with Ambient Light: Sponza only lit by ambient light.
[x] π©GPUπ© [Lighting]: Sponza with directional light: Sponza lit with a directional light and shadows.
[x] π©GPUπ© [Lighting]: Sponza with omni lights: Sponza lit by several omni lights casting shadows.
[x] π©GPUπ© [GI]: Sponza lightmapped: Sponza lit with directional light, lightmapped only (lights all baked static).
[x] π©GPUπ© [GI]: Sponza with RefProbe: Sponza with directional light, with GI reflection probe.
[x] π©GPUπ© [GI]: Sponza with VoxelGI: Sponza with directional light, with GI from VoxelGI
[x] π©GPUπ© [GI]: Sponza with SDFGI: Sponza with directional light, with GI from SDFGI
[x] π©GPUπ© [GI]: Sponza with SSGI: Sponza with directional light, with GI from SSGI
[x] π©GPUπ© [Effects]: Sponza with SSR: Sponza with directional light, with SSR
[x] π©GPUπ© [Effects]: Sponza with Volumetric Fog: Sponza lit with directional light and volumetric fog.
[x] π©GPUπ© [Effects]: Sponza with SSAO: Sponza with constant ambient light (no lights) and SSAO.
[x] π©GPUπ© [Effects] GLOW: Sponza with ambient light, all glow levels enabled.
[x] π©GPUπ© [AA]: FXAA: Sponza with ambient light (not lights), and only FXAA
[x] π©GPUπ© [AA]: MSAA4x: Sponza with ambient light (not lights), and MSAA4x
[x] π©GPUπ© [AA]: MSAA8x: Sponza with ambient light (not lights), and MSAA8x
[x] π©GPUπ© [AA]: TAA: Sponza with ambient light (not lights), and TAA
[x] π©GPUπ© [AA]: FSR2_100: Sponza with ambient light (not lights), and FSR2 at 100%
[x] π©GPUπ© [AA]: FSR2_50: Sponza with ambient light (not lights), and FSR2 at 50% (or whathever)
[x] π©GPUπ© [DOF]: Box: Sponza with ambient light, running Box DOF effect (near and far)
[x] π©GPUπ© [DOF]: Hex: Sponza with ambient light, running Hex DOF effect (near and far)
[x] π©GPUπ© [DOF]: Circle: Sponza with ambient light, running Circle DOF effect (near and far)
[x] π©GPUπ© [SDFGI Motion]: OFF: Very large scene with directional light and without SDFGI off, camera moving around.
[x] π©GPUπ© [SDFGI Motion] ON: Very large scene with directional light and without SDFGI on, camera moving around.
2D Rendering
[x] π₯CPUπ₯ [CanvasItem]: Rendering: Draw different shapes (images, circles, etc) using the CanvasItem 2D drawing API. Draw 5000 elements. Measure performance.
[x] π₯CPUπ₯ [CanvasItem]: Re-Rendering: Same as above, but every frame call queue_update() so it redraws. Measure performance.
[x] π₯CPUπ₯ Polygon : Draw complex polygons (1000 points) every frame
[x] π₯CPUπ₯ [BunnyMark]: nodes: Run a bunnymark style benchmark with 5000 nodes.
[x] π₯CPUπ₯ [BunnyMark]: CanvasItem: Run a bunnymark style benchmark with 5000 bunnies drawn using CanvasItem API
[x] π₯CPUπ₯[BunnyMark]: MeshInstance2D: Run a bunnymark style benchmark with 5000 bunnies drawn using MeshInstance2D, drawing directly into the 2D array.
[x] π₯CPUπ₯ [2D Lights]: Lights: Benchmark running several 2D lights on-screen
[x] π₯CPUπ₯ [2D Lights]: Lights & Shadows: Benchmark running several 2D lights and shadows on-screen
2D Physics
3D Physics
GDScript
C
GodotCPP (C++)
Asset importing
[ ] π¦Startupπ¦ [Mesh Import]: GLTF: Import a large GLTF2 scene (lots of geometry) but no textures.
[ ] π¦Startupπ¦ [Mesh Import]: FBX: Import a large FBX file (same as above, FBX format)
[ ] π¦Startupπ¦ [Mesh Import]: OBJ : Same as above, OBJ format.
[ ] π¦Startupπ¦ [Image Import]: Lossless: Import 200 images generated at random resolutions with lossless compression (WebP)
[ ] π¦Startupπ¦ [Image Import]: Lossy: Import 200 images generated at random resolutions with lossy compression (WebP)
[ ] π¦Startupπ¦ [Image Import]: S3TC: Import 200 images generated at random resolutions as S3TC
[ ] π¦Startupπ¦ [Image Import]: ETC2: Import 200 images generated at random resolutions as S3TC
[ ] π¦Startupπ¦ [Image Import]: BC7: Import 200 images generated at random resolutions as BC7
[ ] π¦Startupπ¦ Image Import]: BC6H: Import 200 images generated at random resolutions as BC6H
[ ] π¦Startupπ¦ [Image Import]: ASTC: Import 200 images generated at random resolutions as ASTC
[ ] π¦Startupπ¦ GLTF Export : Export a large GLTF2 file (same as import I guess)
[ ] π¦Startupπ¦ [Audio Import]: OGG : Import 50 OGG Vorbis files
[ ] π¦Startupπ¦ [Audio Import]: MP3 Import: Import 50 MP3 files
[ ] π¦Startupπ¦ [Audio Import]: WAV Uncompressed: Import 50 WAV files
[ ] π¦Startupπ¦ [Audio Import]: WAV IMA-ADPCM: Import 50 WAV files
Scene nodes (base Node class)
[x] πͺAlgorithmπͺ [Adding 5000 children]: Unnamed: Adding 5000 random children nodes without name.
[x] πͺAlgorithmπͺ [Adding 5000 children]: Named: Adding 5000 random children nodes with the same name (let the conflict resolution happen).
[x] πͺAlgorithmπͺ Moving node children: Move 5000 children nodes between two random positions 5000 times.
[x] πͺAlgorithmπͺ [Delete children]: in order: Remove all 5000 children in order, first to last.
[x] πͺAlgorithmπͺ [Delete children]: in reverse order: Remove all 5000 children in order, last to first.
[x] πͺAlgorithmπͺ [Delete children]: in random order: Remove all 5000 children in random order.
[x] πͺAlgorithmπͺ Get node: Create a complex scene hierarchy of 1000 nodes with random nesting, Obtain paths of all nodes, test performance of get_node() for each one from root node
Animation
[x] π₯CPUπ₯ Animated Models Blended : Animate 100 Skeletal character model with complex AnimationBlendTree (Perhaps optimizing the searching way for TrackPath using other methods than string matching will improve the performance)
[x] π₯CPUπ₯ Animated Models State : Animate 1000 Skeletal character model with complex AnimationStateMachine
[x] π₯CPUπ₯ Tweens: Animate 100 properties with a Tween
[x] π₯CPUπ₯ Tween Methods: Animate 1000 Tweens using tween_method().
Navigation
GUI
Editor
[ ] π¦Startupπ¦ [Editor startup]: with no shader cache
[ ] π¦Startupπ¦ [Editor startup]: with shaders cached
[ ] π¦Startupπ¦ [Editor scan 5000 files on first open]: no cache
[ ] π¦Startupπ¦ [Editor scan 5000 files on first open]: with cache
[ ] π¦Startupπ¦ Inspector full tree update: Open an object with a large number of properties, variety of property editors; force
update_tree
.[ ] π¦Startupπ¦ Editor full theme update: Open the editor, change editor base/accent color or font size.
[ ] π¦Startupπ¦ Editor log update: Print 10000 messages (some duplicating, some unique) to the log, toggle on and off "Collapse similar".
Networking (no idea how to benchmark this, you are the expert)
[High level networking node]: sync: Sync 1000 nodes in the SceneTree
[High level networking node]: variable sync: Sync 1 node with 1000 variables in the SceneTree