KhronosGroup / glTF-Sample-Assets

To store all models and other assets related to glTF
255 stars 30 forks source link

Performance test models #105

Open javagl opened 4 months ago

javagl commented 4 months ago

This is related to other issues, but may warrant its own discussion.

It is also related to a recent discussion in https://github.com/KhronosGroup/glTF-Sample-Assets/pull/100 , which aims at having a model that challenges the runtime with many nodes. And I think that it is good to have such a model. But I think that it would be even better to not have a single model, but to have models that more or less systematically cover different dimensions along which the performance can be tested.

And there are many dimensions. Paraphrasing from a comment in the linked PR, there could be test models with ...

All these can be combined, yielding a many-dimensional space with billions of potential test models.

The question then is:

What should be tested?

I think that people with experience in engine/viewer development might have some ideas what the pain points (i.e. the interesting points) could be. And with a clearer idea about these interesting points, one could set up a collection of test models. (Not necessarily as actual assets - preferably, with a mechansim for generating them on demand).

As one overly specific example:

The following is a set of assets that each contain 4096 nodes, arranged in a grid. The only difference between these assets is the number of meshes. The number of meshes is [1, 8, 64, 512, 4096], meaning that in the first case, all nodes refer to the same mesh, and in the last case, each node refers to a different mesh:

output-numMeshes-2024-02-18.zip

The meshes are still equal: They have no textures, no materials, and each has only one mesh primitive for a plane with 10x10 points, and all these mesh primitives share the same accessor data (otherwise, all this wouldn't fit into 130KB)


An aside: The visual appearance is boring...

Khronos Performance Test Model 001

And even though "looking cool" is not the goal of this sort of model, that doesn't mean that such models can not look cool. For example, the same model with 4096 nodes and 4096 meshes, but this time with 32 different materials each with a different texture would look like this:

Khronos Performance Test Model 002

There are still many dimensions fixed here. For example, all textures have the size 1024x1024, and all meshes still have a single primitive plane with 10x10 points. Arbitrary combinations of that would look similar, but could pose other challenges for engines.

javagl commented 4 months ago

A trippy outtake, to open up the space further: These are 4 viewers, showing a model with ~10000 nodes that are all animated with a translation- and rotation component:

Khronos Node Animations

These may be varied along the number of nodes, the animated properties, the number of interpolation steps, the interpolation types, and much more...

(Only https://gltf-viewer.donmccurdy.com/ is somewhat smooth, but of course, that's not really a fair (or even just sensible) comparison for now)

javagl commented 4 months ago

In view of https://github.com/KhronosGroup/glTF-Sample-Assets/pull/100 : Here are ~10000 asteroids (100 meshes + textures), each one rotating and revolving around the center at different speeds:

Khronos Asteroids 0003

(Still, none of them are colliding 😎 )