Proposed RFC Feature - Remixable Material Types

santorac commented 2 years ago

Summary:

Re-structure the material type file format to allow large portions of the file to be reused in multiple material types. To achieve this, we need to formalize the concept of a material property group, allow property groups to contain other property groups, and allow property groups to be defined in their own JSON files separate from the material type.

(Note that "Remixable Material Types" is not necessarily the official name of a particular feature set. It's a working-title to describe this body of work.)

What is the relevance of this feature?

O3DE ships with several material types including StandardPBR (1197 lines), StandardMultilayerPBR (3100 lines), EnhancedPBR (1679 lines), and Skin (1098 lines). These include roughly 1000 lines of JSON that are repeated 6 times. If we could factor out this duplication, it would reduce the total lines from 7092 to about 2500. More importantly, it will allow users to define new custom material types without further duplicating these files, and more easily receive fixes and improvements that are made to the core files.

Feature design description:

The end goal is to use the new JSON "$import" feature (see https://github.com/o3de/sig-core/issues/14 ) to factor-out the common parts of various .materialtype files. The common parts of these files are mostly related to the property layout, and that information is scattered accross several sections of the .materialtype file, so we first need to reorganize the file to co-locate related property definition data. Also, there are some related updates that would naturally coincide with this change, including nested property groups and flattening the .material file format. All of this is described in detail below, listed in roughly the order in which it needs to be implemented.

Note that all these changes can and should be made in a way that preserves backward compatibility with existing files.

Comprehensive Property Groups

Currently, the material property configuration is spread throughout the material type file, separated by configuration type. All property groups are in one section of the file, individual property definitions are in another section, and functors that process these properties are in another. This will make it difficult to factor out information about a subgroup of material properties.

Current .materialtype Format (click to expand)

``` { "propertyLayout": { "groups": [ { "id": "baseColor", "displayName": "Base Color", "description": "Properties for configuring the surface reflected color for dielectrics or reflectance values for metals." }, { "id": "metallic", "displayName": "Metallic", "description": "Properties for configuring whether the surface is metallic or not." }, ... ], "properties": { "baseColor": [ { "id": "color", "displayName": "Color", "description": "Color is displayed as sRGB but the values are stored as linear color.", "type": "Color", "defaultValue": [ 1.0, 1.0, 1.0 ], "connection": { "type": "ShaderInput", "id": "m_baseColor" } }, { "id": "factor", "displayName": "Factor", "description": "Strength factor for scaling the base color values. Zero (0.0) is black, white (1.0) is full color.", "type": "Float", "defaultValue": 1.0, "min": 0.0, "max": 1.0, "connection": { "type": "ShaderInput", "id": "m_baseColorFactor" } }, { "id": "textureMap", "displayName": "Texture Map", "description": "Base color texture map", "type": "Image", "connection": { "type": "ShaderInput", "id": "m_baseColorMap" } }, { "id": "useTexture", "displayName": "Use Texture", "description": "Whether to use the texture map.", "type": "Bool", "defaultValue": true }, ... ], "metallic": [ { "id": "factor", "displayName": "Factor", "description": "This value is linear, black is non-metal and white means raw metal.", "type": "Float", "defaultValue": 0.0, "min": 0.0, "max": 1.0, "connection": { "type": "ShaderInput", "id": "m_metallicFactor" } }, ... ], ... } }, "shaders": [ ... ], "functors": [ ... { "type": "UseTexture", "args": { "textureProperty": "baseColor.textureMap", "useTextureProperty": "baseColor.useTexture", "shaderOption": "o_baseColor_useTexture" } }, { "type": "UseTexture", "args": { "textureProperty": "metallic.textureMap", "useTextureProperty": "metallic.useTexture", "shaderOption": "o_metallic_useTexture" } }, ... ] } ```

Instead, we need to combine all configuration for related properties into one section. This will eventually allow them to be factored out together. In the example below, there is now one property group for all Base Color properties. It also includes the UseTexture functor, which can reference the properties assuming a local scope like just "textureMap" instead of "baseColor.textureMap".

.materialtype Format Rearranged Into Property Groups (click to expand)

``` { "propertyLayout": { "propertyGroups": [ { "id": "baseColor", "displayName": "Base Color", "description": "Properties for configuring the surface reflected color for dielectrics or reflectance values for metals." "properties": [ { "id": "color", "displayName": "Color", "description": "Color is displayed as sRGB but the values are stored as linear color.", "type": "Color", "defaultValue": [ 1.0, 1.0, 1.0 ], "connection": { "type": "ShaderInput", "id": "m_baseColor" } }, { "id": "factor", "displayName": "Factor", "description": "Strength factor for scaling the base color values. Zero (0.0) is black, white (1.0) is full color.", "type": "Float", "defaultValue": 1.0, "min": 0.0, "max": 1.0, "connection": { "type": "ShaderInput", "id": "m_baseColorFactor" } }, { "id": "textureMap", "displayName": "Texture Map", "description": "Base color texture map", "type": "Image", "connection": { "type": "ShaderInput", "id": "m_baseColorMap" } }, { "id": "useTexture", "displayName": "Use Texture", "description": "Whether to use the texture map.", "type": "Bool", "defaultValue": true }, ... ], "functors": [ { "type": "UseTexture", "args": { "textureProperty": "textureMap", "useTextureProperty": "useTexture", "shaderOption": "o_baseColor_useTexture" } } ] }, { "id": "metallic", "displayName": "Metallic", "description": "Properties for configuring whether the surface is metallic or not.", "properties": [ { "id": "factor", "displayName": "Factor", "description": "This value is linear, black is non-metal and white means raw metal.", "type": "Float", "defaultValue": 0.0, "min": 0.0, "max": 1.0, "connection": { "type": "ShaderInput", "id": "m_metallicFactor" } }, ... ], "functors": [ { "type": "UseTexture", "args": { "textureProperty": "textureMap", "useTextureProperty": "useTexture", "shaderOption": "o_metallic_useTexture" } } ] }, ] }, "shaders": [ ... ], "functors": [ ... ] } ```

Nested Property Groups

The StandardMultilayerPBR material type is essentially three layers of standard PBR properties that get blended together. Each of these layers are identical and should eventually be factored out. In the end, we want the same property group definition to be used once in StandardPBR and used three times for each of the layers in StandardMultilayerPBR. This is difficult with the current file format because it only has two levels in the property hierarchy: property groups and properties. So we name each group like "layer1_baseColor", "layer2_baseColor", etc. and each of these has the same standard base color properties. Similarly, each of these properties connects to a ShaderResourceGroup (SRG) field using a layer number prefix like "m_layer1_m_baseColorFactor", "m_layer2_m_baseColorFactor", etc.

Multilayer Material Type Using Current Format (click to expand)

``` { "propertyLayout": { "groups": [ ... { "id": "layer1_baseColor", "displayName": "Layer 1: Base Color", "description": "Properties for configuring the surface reflected color for dielectrics or reflectance values for metals." }, { "id": "layer2_baseColor", "displayName": "Layer 2: Base Color", "description": "Properties for configuring the surface reflected color for dielectrics or reflectance values for metals." }, { "id": "layer3_baseColor", "displayName": "Layer 3: Base Color", "description": "Properties for configuring the surface reflected color for dielectrics or reflectance values for metals." }, ... ], "properties": { "layer1_baseColor": [ { "id": "color", "displayName": "Color", "description": "Color is displayed as sRGB but the values are stored as linear color.", "type": "Color", "defaultValue": [ 1.0, 1.0, 1.0 ], "connection": { "type": "ShaderInput", "id": "m_layer1_m_baseColor" } }, { "id": "factor", "displayName": "Factor", "description": "Strength factor for scaling the base color values. Zero (0.0) is black, white (1.0) is full color.", "type": "Float", "defaultValue": 1.0, "min": 0.0, "max": 1.0, "connection": { "type": "ShaderInput", "id": "m_layer1_m_baseColorFactor" } }, ... ], "layer2_baseColor": [ { "id": "color", "displayName": "Color", "description": "Color is displayed as sRGB but the values are stored as linear color.", "type": "Color", "defaultValue": [ 1.0, 1.0, 1.0 ], "connection": { "type": "ShaderInput", "id": "m_layer2_m_baseColor" } }, { "id": "factor", "displayName": "Factor", "description": "Strength factor for scaling the base color values. Zero (0.0) is black, white (1.0) is full color.", "type": "Float", "defaultValue": 1.0, "min": 0.0, "max": 1.0, "connection": { "type": "ShaderInput", "id": "m_layer2_m_baseColorFactor" } }, ... ], ... } }, "shaders": [ ... ], "functors": [ ... { "type": "UseTexture", "args": { "textureProperty": "layer1_baseColor.textureMap", "useTextureProperty": "layer1_baseColor.useTexture", "shaderOption": "o_layer1_o_baseColor_useTexture" } }, { "type": "UseTexture", "args": { "textureProperty": "layer2_baseColor.textureMap", "useTextureProperty": "layer2_baseColor.useTexture", "shaderOption": "o_layer2_o_baseColor_useTexture" } }, ... ] } ```

In order to remove the "layerN_" prefixes from the property group names, we will add support for nested property groups. Each property group can contain any number of property subgroups. Thus layer1 will be a property group that contains property groups for baseColor, metallic, etc, and each of those will contain their relevant properties.

To remove the "layerN_" prefixes from the property connections, each property group can specify a prefix that will be automatically attached to the SRG names or shader option names within each property definition. So in the example below, the layer1.baseColor.factor property specifies a ShaderInput name "m_baseColorFactor"; because the layer has shaderInputPrefix "mlayer1", the property will be connected to the SRG field called "m_layer1_m_baseColorFactor".

Multilayer Material Type Using Nested Property Groups (click to expand)

``` { "propertyLayout": { "propertyGroups": [ { "id": "layer1", "shaderInputsPrefix": "m_layer1_", "shaderOptionsPrefix": "o_layer1_", "displayName": "Layer 1", "description": "Material properties for the first layer, to be blended with other layers.", "propertyGroups": [ { "id": "baseColor", "displayName": "Base Color", "description": "Properties for configuring the surface reflected color for dielectrics or reflectance values for metals.", "properties": [ { "id": "color", "displayName": "Color", "description": "Color is displayed as sRGB but the values are stored as linear color.", "type": "Color", "defaultValue": [ 1.0, 1.0, 1.0 ], "connection": { "type": "ShaderInput", "id": "m_baseColor" } }, { "id": "factor", "displayName": "Factor", "description": "Strength factor for scaling the base color values. Zero (0.0) is black, white (1.0) is full color.", "type": "Float", "defaultValue": 1.0, "min": 0.0, "max": 1.0, "connection": { "type": "ShaderInput", "id": "m_baseColorFactor" } }, { "id": "textureMap", "displayName": "Texture Map", "description": "Base color texture map", "type": "Image", "connection": { "type": "ShaderInput", "id": "m_baseColorMap" } }, { "id": "useTexture", "displayName": "Use Texture", "description": "Whether to use the texture map.", "type": "Bool", "defaultValue": true }, ], "functors": [ { "type": "UseTexture", "args": { "textureProperty": "textureMap", "useTextureProperty": "useTexture", "shaderOption": "m_useTexture" } } ] }, ... ] }, { "id": "layer2", "shaderInputsPrefix": "m_layer2", "shaderOptionsPrefix": "o_layer2", "displayName": "Layer 2", "description": "Material properties for the second layer, to be blended with other layers.", "propertyGroups": [ //////////////////////////////////////////////////////// // This section will be identical to the layer1 propertyGroups above ] }, ... ] }, "shaders": [ ... ], "functors": [ ... ] } ```

Flatten the .material file format

Currently, the .material file specifies property values in a two-level hierarchy of property groups and property values, with the group name and property name being specified separately. By flattening this into a single layer of full property names (i.e. "groupName.propertyName") we can support an arbitrarily deep nesting of property groups. (Note that this is easier than just expanding the nesting in the current format, because of the way the json serialization system works. It also makes it easier for developers to search .material files for specific properties by their full name).

The old format will still be supported for backward compatibility, but any new files created by the Material Editor will use the new format.

Current .material File Format (click to expand)

``` { "materialType": "Materials\\Types\\StandardPBR.materialtype", "properties": { "baseColor": { "textureMap": "Textures/Default/default_basecolor.tif" }, "normal": { "textureMap": "Textures/Default/default_normal.tif" }, "roughness": { "textureMap": "Textures/Default/default_roughness.tif" } } } ```

Flattened .material File Format (click to expand)

``` { "materialType": "Materials\\Types\\StandardPBR.materialtype", "properties": { "baseColor.textureMap": "Textures/Default/default_basecolor.tif", "normal.textureMap": "Textures/Default/default_normal.tif" "roughness.textureMap": "Textures/Default/default_roughness.tif" } } ```

Factor Out Core Material Types

Finally, we can update StandardPBR, EnhancedPBR, StandardMultilayerPBR, and Skin material types to share common property definitions. Some minor restructuring of properties might be needed to create exact alignment between the property groups. Then the common portions can be moved to separate json files using the $import feature of the json serialization system.

Some portions of this refactoring could be done in phases as the above features come online.

Technical design description:

Much of the work described here is already in progress on a branch. We have made significant updates to the MaterialTypeSourceData class, added a clean API for accessing property groups (all the data was publicly accessible before). The work in progress looks something like this...


class MaterialTypeSourceData
{
...
    PropertyGroup* AddPropertyGroup(AZStd::string_view propertyGroupId);
    PropertyDefinition* AddProperty(AZStd::string_view propertyId);

    const PropertyGroup* FindPropertyGroup(AZStd::string_view propertyGroupId) const;
    const PropertyDefinition* FindProperty(AZStd::string_view propertyId) const;

    bool EnumeratePropertyGroups(const EnumeratePropertyGroupsCallback& callback) const; // Run a callback function for each property group
    bool EnumerateProperties(const EnumeratePropertiesCallback& callback) const;     // Run a callback function for each property group

...
}

The structure of the PropertyGroup class naturally supports nesting by giving each PropertyGroup a list of PropertyGroups.

class PropertyGroup
{
...
    PropertyDefinition* AddProperty(AZStd::string_view name);
    PropertyGroup* AddPropertyGroup(AZStd::string_view name);

    const AZStd::string& GetName() const;
    const AZStd::string& GetDisplayName() const;
    const AZStd::string& GetDescription() const;
    const PropertyList& GetProperties() const;
    const AZStd::vector<AZStd::unique_ptr<PropertyGroup>>& GetPropertyGroups() const;

private:
    AZStd::vector<AZStd::unique_ptr<PropertyDefinition>> m_properties;
    AZStd::vector<AZStd::unique_ptr<PropertyGroup>> m_propertyGroups;
...
}

FAQ

What are the advantages of the feature?

Reduced duplication
Reduced maintenance overhead
Reduced opportunity for type-specific bugs
Ensures consistency for the users across multiple material types
You can fix or improve core material types in one place. Other material types can automatically benefit from from the improvements.

What are the disadvantages of the feature?

The implementation of MaterialTypeSourceData has to maintain old code paths for backward compatibility.
Sharing property layouts could make it a bit more diffucult make certain customizations; you might need to inline some shared data before modifying it.

How will this be implemented or integrated into the O3DE environment?

These changes will be transparent to most users. Anyone who has made custom .materialtypes might want to update them to use the new format or import common property groups. This isn't require though, as the current format will continue to be supported.

Are there any alternatives to this feature?

Instead of using the JSON $import feature, we could formalize the Property Group even more, with a bespoke reference in the .materialtype to a specialized PropertyGroup asset. This may be desirable in the future if we someday bundle shader code together with a property group.

How will users learn this feature?

After completing this work, we will provide updated documentation that shows how to author .materialtype files. We can also provide a document that shows how to convert existing .materialtype files to the new format.

Are there any open questions?

How soon do we really need this?
Are there many customers who are already making custom material types?

Out Of Scope:

The following related features could be pursued after implementing the "Remixable Material Types" concept, but they are considered out of scope for this RFC and would need their own respective RFCs.

Property Group Version Auto Updates

We currently have planned a feature that will automatically update .material files when a .materialtype file changes. For example, if a property "baseColor.map" is renamed to "baseColor.texture", the .materialtype can provide a procedure for updating any older .material file that still uses the old name, thus ensuring backward compatibility. It would be useful to do this on the property-group level as well, so that any materialtype using this property-group will inherit the upgrade procedure for that property group.

This is quite a bit more complex than supporting version updates for the .materialtype. At the material type level, the main downstream data is the .material file, which is relatively easy to update. But since the .materialtype consumes the property group data, any renames at that the property group level will have to be applied to functors within the materialtype. These functors are often implemented by lua scripts which reference properties by name. Applying renames or other transformations inside a lua script would be prohibitively complex. We will have to change our lua script API to not directly reference properties, but rather expose data slots, and those slots could be connected to properties from inside the .materialtype file. Then it would be straight-forward to provide mechanisms for editing those connections into and out of the the lua functor scripts.

This will likely be out of scope for the initial version of remixable material types. For now, users can write conversion procedures at the top .materialtype level, and will have to manually update those procedures if any of the imported property groups are changed.

Material Multiple Inheritance

In the existing system, materials can inherit the property values from any other material that shares the same material type. So in the case of StandardMultilayerPBR, these materials can only inherit from other StandardMultilayerPBR materials. But this material type functions as three layers of StandardPBR that get blended together. It would be reasonable to allow StandardMultilayerPBR materials to inherit three different StandardPBR material files, as the parents for each of its three layers. This would allow game teams to maintain a large library of StandardPBR materials, and also combine those into multilayer materials, and any changes made to the StandardPBR materials would be automatically applied to any multilayer materials where they are used.

This feature is somewhat complex, probably deserves its own RFP, and is therefore out of scope for the initial version of remixable material types. It will be important for artist workflows that use multilayer materials extensively, and should be considered a high priority follow-up especially if there are game teams with this need.

wintermute-motherbrain commented 2 years ago

Approved by sig/graphics-audio on 2021/10/20

santorac commented 2 years ago

@o3de/sig-graphics-audio One minor update here, I'd like to call these "property groups" instead of using a new term "property sets", to be more consistent with the existing system's use of the term "group". After discussing with @gadams3 I think this will help make sure the new code is consistent with existing code, and avoid disrupting people who are already used to the term "group" in this context.

santorac commented 2 years ago

@o3de/sig-graphics-audio One minor update here, I'd like to call these "property groups" instead of using a new term "property sets", to be more consistent with the existing system's use of the term "group". After discussing with @gadams3 I think this will help make sure the new code is consistent with existing code, and avoid disrupting people who are already used to the term "group" in this context.

I went ahead and updated the RFC content to use "property groups" instead of "property sets". I also changed "optionsPrefix" to "shaderOptionsPrefix" and changed "srgFieldPrefix" to "shaderInputsPrefix". This brings the document in line with the actual implementation I'm working on.

o3de / sig-graphics-audio