Improve v3.0 Shader Handling

[ilexp]

Summary

There are some rough edges on how v3.0 handles shader compilation and reflection. They should be fixed.

Analysis

• Unused shader fields are optimized away by the driver, so they won't get a field location and will be filtered out by the NativeShaderProgram.Compile call on the Duality side.
  • This means that users can define shader properties, but can't set material values for them unless they're used, or worse, previously set values might be removed if a shader field goes unused in a WiP version when working on a shader.
  • Change the implementation so that fields that are defined will always show up, regardless of whether they have a field location assigned or not.
  • Make sure the backend can handle shader fields with an invalid location properly. Consider providing a second code path with valid fields only for internal use in the backend.
• Some intel drivers crash with an access violation when encountering #pragma directives they don't know. This is a driver bug, but a workaround would be nice.
  • Could be achieved by removing or commenting out Duality pragmas before passing them on to the backend.
  • However, since the backend currently does the shader field parsing, this is currently not viable.
  • Consider moving all shader field parsing to the core,, maybe in conjunction with ShaderSourceBuilder, so the backend will only get the processed, reduced version of the shader code in the first place.

[ilexp]

Progress

• Started working on this in the feature/3.0/shader-handling-improvements branch.
• Implemented shader field parsing as part of ShaderSourceBuilder.

ToDo

• Write more specific shader field parsing tests.
  • Array fields
  • Field types
• Adjust ShaderSourceBuilder to comment out all Duality specific pragma directives on build to fix the intel driver crash issue.
  • Refactor shader reflection code to use metadata retrieved by ShaderSourceBuilder instead.
  • Remove shader field parsing from the graphics backend.
• No longer filter out shader fields based on location bindings, keep them for editing and reflection purposes instead.
  • Make sure the backend can handle shader fields with an invalid location properly.
  • Consider providing a second code path with valid fields only for internal use in the backend.

[ilexp]

Progress

• Added type and array declaration unit tests for shader field parsing.
• Removed all shader field parsing from the graphics backend.
• Shader reflection now uses ShaderSourceBuilder parsing results instead.
• Now distinguishing between declared fields (Shader, DrawTechnique) and bound / located fields (NativeShaderProgram).
• Rendering is currently broken / blank, probably due to a copy paste error, or forgetting to adjust a spot to match the new parsing code.

ToDo

• Fix rendering with the new shader field code.
• Make sure BatchInfoPropertyEditor is still fine with the new code.
  • Also re-check editing shaders and observing property editor updates to match the new field declarations.
• Adjust ShaderSourceBuilder to comment out all Duality specific pragma directives on build to fix the intel driver crash issue.
• No longer filter out shader fields based on location bindings, keep them for editing and reflection purposes instead.
  • Make sure the backend can handle shader fields with an invalid location properly.
  • Consider providing a second code path with valid fields only for internal use in the backend.

[ilexp]

Progress

• Fixed the rendering / shader issue, which was caused by a bug in ShaderSourceBuilder ignoring an additional line when commenting out version directives, leading to ignoring the builtin view matrix field.
• Added a unit test to verify expected behavior.
• Shader fields are now split into declared (core) and reflected (backend), keeping BatchInfo editing consistent regardless of shader optimizations, but also avoiding extra work in the backend.

ToDo

• Make sure BatchInfoPropertyEditor is still fine with the new code.
  • Also re-check editing shaders and observing property editor updates to match the new field declarations.
• Adjust ShaderSourceBuilder to comment out all Duality specific pragma directives on build to fix the intel driver crash issue.

[ilexp]

Done and merged. Closing this.