Custom shader parameters

[nithinp7]

Already a good bit of discussion on this here: https://community.cesium.com/t/providing-additional-shader-parameter-to-the-material-instance/13602

Custom materials are already possible through the material slot in the tileset details panel. After https://github.com/CesiumGS/cesium-unreal/pull/427 (@argallegos will modularize the default materials into reusable material functions) it will be even easier to make custom materials that fulfill all the needed functionality for tilesets (gltf material properties, raster overlays, opacity masks, water masks, etc.). But to fully take advantage of custom materials, there needs to be a way for users to set custom shader parameters programmatically.

[getinthedamnbox]

Would Material Parameter Collections help with this?

If you have an MPC in the Content folder, it'll automatically get instantiated when your level starts, and you can find its instance by name like this:

// in .h file
UPROPERTY(EditAnywhere, BlueprintReadWrite)
FName NameOfYourMPC;

// in .cpp file
_yourMPCInstance = GetWorld()->GetParameterCollectionInstance(NameOfYourMPC);

Then you can set scalar and vector parameters:

_clippingPlaneParamsInstance->SetScalarParameterValue(NameOfYourScalarParameter, 1);
_clippingPlaneParamsInstance->SetVectorParameterValue(NameOfYourVectorParameter, planeRepresentation);

And access the values in any material:

You can then wire the result into your GltfMaterialWithOverlays variant with break/make nodes (or put it wherever else you need):

You can see an example in this video.

[nithinp7]

@getinthedamnbox Oh wow that's interesting I hadn't used MPCs before! For any uniforms that will be constant across all materials users would totally be able to use MPCs. I think there may still be an additional need for users to be able to set custom shader parameters based on the gltf model being parsed in specific cases. Basically an interface where users can run arbitrary code to parse the gltf model and write material parameters.

Hey @Nodrev, continuing the conversation from the forum, does the above suggestion of using material parameter collections cover your use case?

[xuelongmu]

The other way to do it besides MPC (which is great for reusing variables across several material instances) is to use dynamic material instances. It looks like already doing this in the applyTexture method in CesiumGltfComponent (calls SetTextureParameterValue).

In terms of the overall flow, it looks like the Material property in Cesium3DTileset is passed into the CesiumGltfComponent in the prepareInMainThread method on Cesium3DTileset. Then, the CesiumGltfComponent sets its base material to the passed in material, and when new models are loaded in, the CesiumGltfComponent updates its material with the new textures from the models.

More info on editing dynamic material instance properties at runtime: https://robincouwenberg.com/change-material-properties-at-runtime-ue4-c-mini-tutorial/

@nithinp7 Building on your "additional need", I think we would probably need a way to propagate a user-defined material parameter down to the gltf components.

[Nodrev]

Hey @Nodrev, continuing the conversation from the forum, does the above suggestion of using material parameter collections cover your use case?

I do not think it does, my use case is more the need to set programmaticaly specific values of shader parameter per Cesium Tile (CesiumGltf mesh). For example, adding a normal map texture to be blended with the tile normal map, and be able to select the normal map to use per tile (using any discriminent, could be geo-coordinates, tile zoom level, etc...)

[nithinp7]

For reference, this is how the raster overlays are tacked onto the CesiumGltf mesh after it is already created (CesiumGltfComponent::UpdateRasterOverlays):

https://github.com/CesiumGS/cesium-unreal/blob/9c445821b1a39fbbd2deb2ab8d72918bac6e8dfc/Source/CesiumRuntime/Private/CesiumGltfComponent.cpp#L1754-L1759

One extra step will be needed when doing this from outside CesiumGltfComponent, you'll need a reference to the Cesium3DTileset and you can iterate through its components to look for the CesiumGltfComponent you want, then you can follow the code snippet above. Once the dynamic material instance is retrieved, any desired shader parameters can be directly set on it.

To further leverage per-tile information (i.e., information that came in the gltf), we may need a plugin-side change to save the CesiumGltf::Model locally in CesiumGltfComponent after the mesh is initially created. This would allow it to later inform the setting of custom shader parameters.

[Nodrev]

One extra step will be needed when doing this from outside CesiumGltfComponent, you'll need a reference to the Cesium3DTileset and you can iterate through its components to look for the CesiumGltfComponent you want, then you can follow the code snippet above. Once the dynamic material instance is retrieved, any desired shader parameters can be directly set on it.

Yes, storing a pointer to Cesium3DTileset and parsing children CesiumGltfComponent each tick was the second solution I have in head, looking if the shader parameters were already setted for a specific tile. For this my idea was to store references to CesiumGltfComponent using WeakPtr (to not mess up with the GC). It would work but I think it's not as good as the possibility to override UnrealResourcePreparer.

To further leverage per-tile information (i.e., information that came in the gltf), we may need a plugin-side change to save the CesiumGltf::Model locally in CesiumGltfComponent after the mesh is initially created. This would allow it to later inform the setting of custom shader parameters.

I can probably get the information I need by modifying my terrain tiles filenames (and thus, this will be available in UE4 as the name of the CesiumGltfComponent). So this part is not a mandatory need to me, but it could indeed be great if we could retrieve the original Gltf data outside the plugin.

[kring]

Another request for something like this on the forum: https://community.cesium.com/t/using-dynamic-materials-on-cesium-tilesets/14783

[kring]

Another request from the forum: https://community.cesium.com/t/c-how-to-use-the-private-class-in-the-latest-released-version/15149

[leng2490]

@getinthedamnbox Can I see the video about this solution

[getinthedamnbox]

@leng2490 It looks like the user made the video I shared above private, so I unfortunately don't have a way of accessing it anymore. If you search for "Material Parameter Collection," though, you should find additional tutorials online (e.g., this one).