search

nvpro-samples / vk_gltf_renderer

Rendering glTF scenes with ray tracer and raster (Vulkan)
Apache License 2.0
154 stars 13 forks source link
cubemap gltf gltf2 importance-sampling pbr raytracing tonemapping vulkan

readme

Vulkan glTF Scene Renderer

Pathtracer Raster

Overview

This application demonstrates a dual-mode renderer for glTF 2.0 scenes, implementing both ray tracing and rasterization pipelines. It showcases the utilization of shared Vulkan resources across rendering modes, including geometry, materials, and textures.

Key Features

Dependencies

Build Instructions

  1. Clone the repositories

    git clone https://github.com/nvpro-samples/nvpro_core.git
git clone https://github.com/nvpro-samples/vk_gltf_renderer.git

  2. Build the project

    cd vk_gltf_renderer
mkdir build
cd build
cmake ..
cmake --build . --config release

  3. Run the application

    .\bin_x64\Release\vk_gltf_renderer.exe

  4. Install [optional] : if you want to package the application

    cmake --install .

Draco Compression

To enable Draco mesh compression, you need to enable the option CMake. In the GUI interface, you will see the option USE_DRACO. If you are using the command line, you can add -DUSE_DRACO=ON to the cmake command. This will download the Draco library and it will be included in the project.

glTF Core features

What is currently not supported are animations and skins, multiple textures coordinates, morph targets, color at vertices, and some extensions.

GLTF Extensions

Here are the list of extensions that are supported by this application

Pathtracer

Implements a path tracer with global illumination.

The options are:

Raster

Utilizes shared Vulkan resources with the path tracer, including:

The options are:

Example with wireframe option turned on

Debug

There is also the ability to debug various out channels, such as:

metallic roughness normal base emissive

Environment

It is possible to modify the environment, either by choosing an integrated sun and sky, or by lighting the scene using Image Base Lighting. In the latter case, you need an image (.hdr). You can find examples of such images at the following address Poly Haven

Sun & Sky HDRi

Having HDRi (High Dynamic Range Imaging) to illuminate the scene greatly simplifies complex lighting environments. It also helps to integrate 3D objects into its environment.

This example loads HDR images, then creates an importance sampling acceleration structure used by the ray tracer and stores the PDF in the alpha channel of the RGBA32F image.

For real-time rendering, we use the created acceleration structure and create two cubemaps. One containing the diffuse irradiance and the other, storing the glossy reflection, where the different levels of glossiness are stored in separate mipmap levels.

The HDR can also be blured and rotated to better fit the scene.

An option to replace the background with a solid color is also available.

Tonemapper

We could not get good results without a tone mapper. This is done with a compute shader and different settings can be made.

Multiple tonemapper are supported:

Camera

The camera navigation follows the Softimage default behavior. This means, the camera is always looking at a point of interest and orbit around it.

Here are the default navigations:

The camera information can be fine tune by editing its values.

Note: copy will copy in text the camera in the clipboard, and pressing the paste button will parse the clipboard to set the camera.

Ex: {0.47115, 0.32620, 0.52345}, {-0.02504, -0.12452, 0.03690}, {0.00000, 1.00000, 0.00000}

Save and Restore Cameras

It is also possible to save and restore multiple cameras in the second tab. Press the + button to save a camera, the middle button to delete it. By pressing one of the saved cameras, its position, interests, orientation and FOV will be changed smoothly.

Note: If the glTF scene contains multiple cameras, they will be showing here.

Other modes

Other navigation modes also exist, like fly, where the w, a, s, d keys also moves the camera.

Schema of the Program

The nvvk::Application is a class that provides a framework for creating Vulkan applications. It encapsulates the Vulkan instance, device, and surface creation, as well as window management and event handling.

When using nvvk::Application, you can attach nvvkhl::IAppElement to it and each element will be called for the different state, allowing to customize the behavior of your application. The nvvkhl::IAppElement class provides default implementations for these functions, so you only need to override the ones you need.

Here is a brief overview of how nvvk::Application works:

Initialization:

When you create an instance of nvvk::Application, it sets up the Vulkan instance, device, and surface. It also creates a window and sets up event handling.

Attaching Elements

In main() we are attaching many elements, like:

But the main one that interest us, and which is the main of this application is GltfRendererElement. This is the one that will be controlling the scene and rendering.

Main Loop:

The nvvk::Application class provides a main loop that continuously processes events and updates the application state. Inside the main loop, it calls the following functions:

Scene Graph

The GLTF scene is loaded using tinygltf and then converted to a Vulkan version. The Vulkan version is a simplified version of the scene, where the geometry is stored in buffers, and the textures are uploaded to the GPU. The Vulkan version is used for both raster and ray tracing.

The scene is composed of nodes, where each node can have children and each node can have a mesh. The mesh is composed of primitives, where each primitive has a material. The material is composed of textures and parameters. However, none of this is directly used in the rendering, as we are using a simplified version of the scene.

Once the scene has been loaded, we proceed to parse it in order to collect the RenderNodes and RenderPrimitives. The RenderNode represents the flattened version of the tree of nodes, where the world transformation matrix and the material are stored. The RenderPrimitive, in contrast, represents the unique version of the primitive, where the index and vertex buffers are stored.

RenderNodes represent the elements to be rendered, while RenderPrimitives serve as references to the data utilized for rendering.

Animation

If there is animation in the scene, a new section will appear under the Scene section. It allows to play/pause, step and reset the animation, as well as changing its speed.

Multiple Scene

If there are multiple scenes, a new section will appear under the Scene section. It will show all the scenes and their name. Clicking on a scene name will switch to the scene.

Material Variant

If there are multiple material variant, a new section will appear under the Scene section. It will show all the material variant and their name. Clicking on a variant name will apply it on the models.

Scene Graph UI

It is possible to visualize the scene hierarchy, to select node, to modify their transformation and their material, to some level.

Here's a shorter version of the text, tailored for developers on GitHub:

Recompiling Shaders

For quick shader testing, use the Recompile Shaders button to hot-reload shaders. Requirements:

To force using the external shaders:

vk_gltf_reenderer.exe -forceExternalShaders

Note: Hot-reloading won't work without the shared libraries and shaders, but the app will still run.

Tools

The application comes with a few tools to help debug and visualize the scene.

Profiler

The profiler is a tool that allows to measure the time spent on the GPU. It is possible to measure the time spent on the different stages of the rendering, like the path tracing, the rasterization, the tonemapping, etc.

Logger

The logger is a tool that allows to see the log information. It is possible to filter the log information by selecting the level of the log.

Nvml

The Nvml is a tool that allows to see the status of the GPU. It is possible to see the temperature, the power, the memory usage, etc.

Tangent Space

There is a tangent space tool that allows to fix or to recreate the tangent space of the model. This is useful when the normal map is not looking right or there are errors with the tangents in the scene.

Utilities

gltf-material-modifier.py

Modify materials in a GLTF file and optionally reorient the scene from Z-up to Y-up.

usage: gltf-material-modifier.py [-h] [--metallic METALLIC] [--roughness ROUGHNESS] [--override] [--reorient]
                                 input_file output_file

positional arguments:

   input_file            Path to the input GLTF file.
   output_file           Path to save the modified GLTF file.

options:

  -h, --help             show this help message and exit
  --metallic METALLIC    Set the metallic factor (default: 0.1).
  --roughness ROUGHNESS  Set the roughness factor (default: 0.1).
  --override             Override existing material values if set.
  --reorient             Reorient the scene from Z-up to Y-up.