search

mellinoe / ShaderGen

Proof-of-concept library for generating HLSL, GLSL, and Metal shader code from C#,
MIT License
497 stars 56 forks source link

[BUG] Using bool field type in buffer causes stack overflow #70

Closed thargy closed 6 years ago

thargy commented 6 years ago

When using the bool field type in a struct that is itself used in a buffer, ShaderGen crashes due to a stack overflow. This can be traced to TypeSizeCache.GetCSharpSize not having a known size for System.Boolean, to fix that issue requires reference to the conversation in #14, as the size is different depending on the backend in question.

This bug then causes it to try to determine the size manually, there are only two fields in a bool, and both are static strings (FalseString and TrueString). As they are statics they should be skipped as they don't contribute to the size.

However, this bug causes TypeSizeCahce.GetCSharpAlignment to be called for the symbol System.String. This two is not a known type, and also has a single field, which is also a static string (Empty), so TypeSizeCahce.GetCSharpAlignment is called again for the symbol System.String. And we get recursion.

The fixes therefore are:

  1. Add a size of 4 bytes for bool, which should be correct for everything but Metal (see #14). May need to add support to specify a size of 1 for Metal.
  2. Skip consideration of static fields in structs when calculating Size.
  3. Change recursive methods to use stack based recursion and detect repeats.
  4. Consider adding support for System.String by treating as a byte[] or similar.
thargy commented 6 years ago

I've completed a fix for this an tested that the new analyser returns the same info as the old algorithm.