Draft: Proposed RFC Feature: Support for multiple Material-Types in Raytracing Hit Shaders

[kh-huawei]

Summary:

During raytracing the hit-shaders need access to all materials in the scene at once, which clashes with the current setup for the Forward+ - Pipeline, where a shader needs only access to a single Material Instance at any given time. The current solution for Raytracing is somewhat unfinished, in that it assumes that there exists only one material-type, and collects all Material-Instances in a single RayTracingMaterialSrg, and uses one single hit-shader that re-implements almost but not quite the BasePBR - material type.

What is the relevance of this feature?

The new Material-Pipeline and the Material Canvas allows users to easily create their own material types, but that has no influence on the material used during raytracing. This feature would remedy that, and enable user-created material types during raytracing. Incidentally, this would also make implementing a deferred pipeline easier.

Feature design description:

Similar to the existing RayTracingMaterialSrg - solution, but more general: Automatically create one MaterialSrg for each material type, that contains the settings of all material-instances of that type that are currently active in the scene. Each sub-mesh needs a Material Instance Index somehow to refer to the relevant instance. And to not have material shader-utils exist twice in the engine, the shader functions should be re-written so that they work with a MaterialInfo - struct and an array of textures, instead of accessing the MaterialSrg directly.

--- OR ---

Stick the MaterialInfo - struct for each instance into a BufferView so it ends up in a ByteArrayBuffer of the Bindless SRG, and the RayTracingMaterialSrg redirects only from the mesh to the right MaterialInfo - ByteArray Buffer. The shader for the material-type knows how to decode the contents of the ByteArrayBuffer to the appropriate MaterialInfo struct (from the Material Canvas Abstraction).

Technical design description:

First rough draft:

• Create one MaterialSRG for each material-Type, auto-generate MaterialInfo-struct -- OR --
• stick the material-info of all material instances into a buffer-view so it's accessible with the bindless array
• Modify existing materials so they don't access the MaterialSrg directly, but work with a MaterialInfo - struct and an array of textures
  • breaking changes to existing materials
  • how well supported is the BINDLESS_SRG by now?
• Add a Material Instance Index somewhere to the Mesh (ObjectSrg?)
  • the Instance Index should probably allow holes, otherwise we need to update all object-Srgs if one instance-id changes
• MeshFeatureProcessor maintains Material-SRGs
• Raster-Pass: Bind the right MaterialSrg in the Draw-Item based on the MaterialType, and use a Material instance Index in the shaders
  • can the new solution exist in parallel with one MaterialSrg for each material-instance, so the Raster-pass work essentially as the do now?
• Raytracing: Generate hit - shaders for each material-type, and add an entry in the raytracing shader table for each Material-Type. Each hit - shader binds the right MaterialSrg
• Deferred: One Fullscreen-triangle - pass for each (used) Material-Type in the scene that binds the MaterialSrg (not actually part of this)

What are the advantages of the feature?

Actually support user-provided materials during raytracing.

What are the disadvantages of the feature?

Breaking changes with existing (old) materials

How will this be implemented or integrated into the O3DE environment?

• Explain how a developer will integrate this into the codebase of O3DE and provide any specific library or technical stack requirements.

Are there any alternatives to this feature?

• Provide any other designs that have been considered. Explain what the impact might be of not doing this.
• If there is any prior art or approaches with other frameworks in the same domain, explain how they may have solved this problem or implemented this feature.

How will users learn this feature?

• Detail how it can be best presented and how it is used as an extension or a standalone tool used with O3DE.
• Explain if and how it may change how individuals would use the platform and if any documentation must be changed or reorganized.
• Explain how it would be taught to new and existing O3DE users.

Are there any open questions?

• What are some of the open questions and potential scenarios that should be considered?

[adiblev]

How would this cope with exposure of textures and handling them via the definition of a surface? was this taken into account?

[kh-huawei]

Like the RayTracingMaterialSrg does now: the Material-Info struct holds the indices of the Texture Views relevant for the material instance, and the textures are then accessed via the bindless Srg.

There are actually two indirections going on currently: The MaterialInfo holds one start-index into the m_materialTextureIndices, and that buffer holds the index of the bindless texture view (either in the bindless m_materialTextures array in the same srg, or the Bindless_Srg - index directly). Seems like an optimization so the materialInfo-struct only needs one texture-index, and/or so that the texture-views for one material-instance are sequential in memory, but i'm not sure how useful that actually is.