Good demos for benchmarking purposes?

[michaellarabel]

Hi, I was just learning about NAP and saw this blog post [ https://blog.napframework.com/2020/moving-habitat-to-nap-0.4/ ] where the performance was talked about... Just wondering if there are any good public / freely available demos for use with NAP for the purpose of CPU/GPU comparison benchmarking? The few demos bundled with the NAP framework appear to be quite basic. Being new to NAP, just curious if there was any guidance on any publicly available assets that would be useful if wanting to feature NAP when conducting any GPU review testing, etc.

Thanks, Michael

[cklosters]

Hi @michaellarabel,

Potentially the heightmap or copystamp demo could be useful. The heightmap demo renders 2 high resolution meshes on top of each other and makes use of transparency. You can change the resolution of the grid, therefore increasing or decreasing the total vertex count. This demo is mostly GPU bound, because all meshes are cached and uploaded to the GPU on initialization.

You can change the rows / columns of the displacement grid in the heightmap.json file:

        {
            "Type": "nap::HeightMesh",
            "mID": "HeightMesh",
            "Usage": "Static",
            "CullMode": "None",
            "Size": {
                "x": 512.0,
                "y": 512.0
            },
            "Position": {
                "x": 0.0,
                "y": 0.0
            },
            "Rows": 1024,
            "Columns": 1024,
            "Heightmap": "HeightMapTexture",
            "Height": 75.0
        },

The copystamp demo could be good for benchmarking GPU / CPU performance. This demo stamps a mesh onto the vertex of another mesh, together with unique rotational, color, scale etc. values. Increasing the resolution of the stamp mesh increases both the CPU and GPU load. Changes the meshes that are stamped onto the vertices changes GPU load.

You can change the resolution of the stamp plane in the copystamp.json file:

        {
            "Type": "nap::PlaneMesh",
            "mID": "StampPlane",
            "Usage": "Static",
            "CullMode": "None",
            "Size": {
                "x": 110.0,
                "y": 120.0
            },
            "Position": {
                "x": 0.0,
                "y": 0.0
            },
            "Rows": 128,
            "Columns": 128
        },

You can also load a completely different mesh and use that as the vertex source:

                {
                    "Type": "nap::RenderableCopyMeshComponent",
                    "mID": "nap::RenderableCopyMeshComponent",
                    "Orient": false,
                    "Scale": 0.7250000238418579,
                    "RotationSpeed": 2.5,
                    "RandomScale": 0.800000011920929,
                    "RandomRotation": 0.949999988079071,
                    "MaterialInstance": {
                        "Material": "CopyMaterial",
                        "Uniforms": [],
                        "Samplers": [],
                        "BlendMode": "Opaque",
                        "DepthMode": "NotSet"
                    },
                    "ColorUniform": "meshColor",
                    "Camera": "CameraTransform",
                    "TargetMesh": "StampPlane",
                    "CopyMeshes": [
                        "CopyBox",
                        "CopySphere",
                        "CopyOctahedron"
                    ]
                }

Content changes are picked up by the demo automatically, making it easy to experiment / benchmark different configurations. Especially the copystamp demo is a lot faster when using Vulkan compared to OpenGL, because we minimized and therefore optimized pipeline switches. The blog post mentions the use of Habitat, but in terms of benchmarking GPU / CPU load that demo is equivalent to the heightmap demo. If you want to increase CPU load you could consider not caching the extreme states of the heightmap and interpolate everything on the CPU instead.

If you're interested in a specific demo you can always contact us at info@naivi.nl, maybe we can make something custom for your specific needs.

[cklosters]

Another potentially useful demo could be videomodulation, which involves CPU video playback, live mesh deformation based on that video and various render passes. You could write something that automatically switches videos or load in higher resolution videos, increasing the CPU workload. Video playback is threaded. Meshes are cached, all displacement is on the GPU. To increase the GPU load you can load a higher resolution mesh.

[cklosters]

Optional thing to consider for benchmarking: you can change the number of MSAA samples & sample based shading setting of every render target (window and backbuffer) individually. Simply look for the nap::RenderWindow or nap::RenderTarget in the json file and set accordingly:

        {
            "Type": "nap::RenderWindow",
            "mID": "Window0",
            "Borderless": false,
            "Resizable": true,
            "Visible": true,
            "SampleShading": true,
            "Title": "Window 1",
            "Width": 1280,
            "Height": 720,
            "Mode": "Immediate",
            "ClearColor": {
                "x": 0.066600002348423,
                "y": 0.07840000092983246,
                "z": 0.14900000393390656,
                "w": 1.0
            },
            "Samples": "Four"
        },

[michaellarabel]

Thanks, started looking at heightmap for starters. Does there happen to be any support for command line options to be able to automate such test easily? At least from quickly searching for documentation and trying --help and the like, hadn't seen any quick CLI controls... Such as for dumping the frame-rate and quitting the demo after a specified period of time? Apologies if it's available and mentioned somewhere in the docs, hadn't seen it from a quick look.

[cklosters]

We don't have customized CLI tools like that readily available. When we deploy NAP application we use a custom python script environment to automate launching, app synchronization etc. This environment is unfortunately not publicly available (yet).

You can however call, from within a nap application quit(), to shut it down after a certain amount of time. You can also add a CLI interface to pretty much any demo, all of our command line tools make use of it. Check commandline.h in licensegenerator for example. Don't forget to add TCLAP to the CMAKE file of the project (source only):

# Add TCLAP
set(TCLAP_FIND_QUIETLY TRUE)
find_package(tclap REQUIRED)
include_directories(${TCLAP_INCLUDE_DIRS})

You can use the nap::Logger to log any type of message to disk (including framerate). To do this, add:

nap::Logger::logToDirectory(utility::getExecutableDir() + "/logs", 'heightmap')

to the main() call of the application / demo.