Dynamic properties in pipeline configuration

[spiralhalo]

Definition

Dynamic properties are properties of pipeline (sub-)configuration that may be derived from the state of the user's pipeline options. For example, previously the size of an image config is always a constant, but now it may be derived from an int option.

Usage

Since this PR, there are 3 ways to define a property (examples are in json5 format):

1. Constant value

size: 128

2. Use option value

size: {
  default: 128,
  useOption: "texture_size"
}

Notes:

• The data type of the option must match the expected type of the property, otherwise the pipeline will fail to compile.
• Int and float are distinct option types. However, since float properties can receive int constant value, float properties can receive int option type as well.

3. Use mapping

size: {
  default: 128,
  useOptionMap: {
    option: "texture_size_selection",
    map: [
      {
        condition: "large",
        value: 256
      },
      {
        condition: "medium",
        value: 128
      },
      {
        condition: "small",
        value: 64
      }
    ]
}

Notes:

• Same rules as point number 2 applies.
• useOption is prioritized.
• Condition can be any type, so it's technically possible to map float values, but is impractical.

Supported uses

• Any config properties with the type String (name references and GL enum constant), Boolean, Int, and Float are supported, including within an array or object.
• UPDATE: Attachment objects, that is the value of depthAttachment and the contents of colorAttachments arrays are supported exceptionally.

Exceptions and Limitations

• Name definitions aren't supported (name references are supported).
• Properties of options aren't supported; No recursion by design.
• Dynamic properties within dynamic properties aren't supported either.
• UPDATE: Arrays are not supported (contents of arrays are supported).
  • Note: This is potentially a subject for future improvement, especially for ease-of-use reason. See this comment.

If there is any property that is otherwise unsupported, it is a bug and please report it.

Backward compatibility

Canvas will stay backward compatible with older pipelines. However, newer pipelines utilizing the dynamic properties won't be compatible with older Canvas. This is for simplicity reason, and not unreasonable as the current standard for "shaderpacks" (i.e. Iris/OF) is to provide support older version manually. Explicit support for 1.18.2 is already planned and prepared for.

Implementation detail

This system works by first loading option configuration and stored option values before anything else in the pipeline, then applying those stored option values while reading the dynamic properties. No extra memory or execution footprint persists after the pipeline is fully loaded and running.

[spiralhalo]

Can't think of anything else to add. Will merge ASAP unless someone gave further reviews, after doing some more rounds of testing.

Tests:

• [x] image property mapping
• [x] arbitrary top level mapping
• [x] samplers mapping
• [x] framebuffer attachment mapping
• [x] enum -> named dependency = requires toLowerCase? nope
• [x] enum -> glEnum = requires toUpperCase? -> maybe, but should it be enforced?

[spiralhalo]

I noticed while testing that the option map can be quite verbose, so I decided to shorten some keywords in order to speed up work on pipelines:

• useOption -> option
• useOptionMap -> optionMap
• condition -> from
• value -> to

Additionally, option map format is simpler using the json key as option name. This is consistent with option config entry definition, but different enough to differentiate at a glance (and option config uses options and never singular option).

Note that the usage of json element key as the option key value also allows multiple options to be used in the map (because why not), in case anyone needs that for some reason.

Preview of this change:

Before

// single option
{
    useOption: "sky_color"
}

// option map
{
    default: "minecraft:textures/environment/moon_phases.png",
    useOptionMap: {
        option: "vanilla_moon",
        map: [
            {
                condition: false,
                value: "lumi:textures/misc/moon_phases.png"
            }
        ]
    }
},

After

// single option
{
    option: "sky_color"
}

// option map
{
    default: "minecraft:textures/environment/moon_phases.png",
    optionMap: {
        vanilla_moon: [
            { from: false, to: "lumi:textures/misc/moon_phases.png" }
        ]
    }
},

[spiralhalo]

Done testing. Made a documentation for this iteration: https://github.com/vram-guild/canvas/wiki/Pipeline-Dynamic-Properties

[spiralhalo]

UPDATE: Reason why array support is good.

Example of good dynamic array formatting (currently unsupported):

cascadeRadius: {
    default: [48, 16, 4],
    optionMap: {
        shadow_resolution: [
            { from: "256", to: [64, 32, 8] },
            { from: "512", to: [64, 32, 8] },
            { from: "1024", to: [64, 16, 8] },
            { from: "2048", to: [48, 16, 4] },
            { from: "4096", to: [48, 16, 4] },
        ]
    }
}

Actual currently supported formatting to achieve the same function as above:

cascadeRadius: [
    {
        default: 48,
        optionMap: {
            shadow_resolution: [
                { from: "256", to: 64},
                { from: "512", to: 64},
                { from: "1024", to: 64},
                { from: "2048", to: 48},
                { from: "4096", to: 48},
            ]
        }
    },
    {
        default: 16,
        optionMap: {
            shadow_resolution: [
                { from: "256", to: 32 },
                { from: "512", to: 32 },
                { from: "1024", to: 16 },
                { from: "2048", to: 16 },
                { from: "4096", to: 16 },
            ]
        }
    },
    {
        default: 4,
        optionMap: {
            shadow_resolution: [
                { from: "256", to: 8 },
                { from: "512", to: 8 },
                { from: "1024", to: 8 },
                { from: "2048", to: 4 },
                { from: "4096", to: 4 },
            ]
        }
    }
]

Based on example above, supporting dynamic arrays can increase formatting brevity on top of ease of reading and editing (which impacts the ease of development).

Further notes:

• IMPORTANT: if array support is enabled, then it must be asked whether dynamic property reading within the array contents (nested dynamic property) should be supported, or if it falls under the "no recursion by design" principle.
• The example above might simply be special case for cascadeRadius, but it might benefit unforeseen other use case (potentially, with sampler sets, for example). The benefit of generalized use and API consistency should be weighed against maintenance cost (i.e. when adding new configurable pipeline properties). The current trend is that new properties are seldom added and therefore maintenance should be of little concern.