CesiumGS / cesium-unreal

Bringing the 3D geospatial ecosystem to Unreal Engine
https://cesium.com/platform/cesium-for-unreal/
Apache License 2.0
950 stars 296 forks source link

Use (non-Material) Blueprints to pass metadata to Materials #829

Open kring opened 2 years ago

kring commented 2 years ago

After #698, Cesium for Unreal has the ability to pass feature metadata into Materials as textures 🎉. While the "Auto Fill" button on the CesiumEncodedMetadataComponent only discovers numeric properties, even string properties can be coerced to numbers (i.e.: parsed) and then used for shading.

However, it's not currently possible to do more complicated transformations of properties to prepare them for use in Materials. For example, Cesium OSM Buildings has a cesium#color property with string values like "rgb(100, 65, 170)". There's no way to parse this string and write an actual color into the generated texture. I hacked up a way to do this in https://github.com/CesiumGS/cesium-unreal/tree/metadata-colors, but I don't think we should ship it in this form.

In CesiumJS, 3D Tiles Styling allows the use of JavaScript-like expressions to style. Implementing this in Unreal at runtime is unlikely, but we could offer an in-editor option to turn a style description into a Material.

Perhaps the more Unreal-y way to do it, though, is to let the user write a Blueprint that takes feature metadata as input and returns values to be stored in a texture, which can then be accessed from a Material. Conceptually this is really straightforward, and a proof of concept of this would probably be quick to implement. But it raises some questions:

kring commented 2 years ago

Related to #771.

j9liu commented 1 year ago

We considered potentially incorporating this into the metadata overhaul, but we probably won't implement it due to the time constraints and performance concerns. But to quote @kring,

Even just the ability to write and register encoding functions in C++, and then select which one of them to use in the CesiumEncodedMetadataComponent UI, would be huge.

Inspired by this idea, I'd like to rework the encoded metadata component UI as so:

This way, it more accurately reflects the data that is available, even if it's incompatible with the GPU. And if the user chooses a coercion method that isn't compatible with that type, maybe we can make use of templates to indicate that it can't be coerced?

That's what I aim to implement for now. This probably won't close the issue, should this implementation make it in, but it's hopefully a good start.