csyonghe / Spire

Other
174 stars 22 forks source link

HLSL codegen support for Hull/Domain shaders #48

Closed tangent-vector closed 7 years ago

tangent-vector commented 7 years ago

The relatively easy part of this change was emitting the overall skeleton of a DS/HS main(). A bit of churn went into handling indexing of input/output, but the recent change to make "projection" first-class in the IR greatly simplified that work.

This checkin changes the approach used for assigning semantics a bit. The good news is that the new approach appears to be far more robust (it handles fields with struct or array types properly). There are two caveats worth mentioning:

  1. In a few cases (specifically VS input and FS output) we codegen a struct with manually-specified semantics, and then "clobber" those by giving a semantic to an entire input/output with that type. That generates a warning from the HLSL compiler, which we suppress with a #pragma.

  2. I'm not clear on how far back the approach I take is supported in the HLSL compiler. I used a different (and much more complicated) workaround for the same basic issues in Spark, and I'd like to think I investigated all the simpler options first.

There's definitely still room for improvement in generating nice and debuggable declarations. Ideally, we might prefer to generate code where rather than large struct types for input/output, we simply generate a bunch of in and out declarations in the parameter list of main(). This should work for many stages, and will lead to better code for the debugger, but it doesn't work for the Hull Shader (at least last time I checked) due to limitaitons in the implementation of the HLSL compiler, and it already wouldn't work for per-corner input/output and per-control-point inputs. This is best left to future work.

Another issue that this code brings to the forefront is the matter of matrix conventions. HLSL matrices are semantically row-major (both in terms of indexing and matrix-vector multiplication), but use column-major storage by default. In contrast, GLSL matrices are semantically column-major (again, for both indexing and matrix-vector multiplication), and use column-major storage by default.

We can't change the indexing convention of either target, so some sleight-of-hand is always required where what HLSL thinks is a row, GLSL thinks is a column. Getting consistency up the rest of the stack then requires a few tweaks:

For the purposes of this changelist, I'm applied both of the above tweaks on the HLSL backend, so that it uses the opposite layout convention and the opposite order of operands for multiplication. I would aruge that this is backwards, though, and the GLSL back-end is the one that should change its conventions. :)