VSH: Inverse square root (rsq) hardcoded swizzle

[AlpyneDreams]

In the D3D8 docs for rsq vertex shader instruction there is a contradiction: the code snippet shown uses the w-component of the source register. However it says "If source has no subscripts, the x-component is used."

It also says x-component is used in D3D9 docs, but I have no reason to believe this is correct in either case. D3D9 docs also state that the source register "requires explicit use of replicate swizzle,that is, exactly one of the .x, .y, .z, .w swizzle components (or the .r, .g, .b, .a equivalents) must be specified." Despite this, for D3D9, DXSO uses an arbitrary, not replicate, swizzle. So it's assumed whatever the shader compiler outputs is correct.

In the spheremap example, a vs 1.0 shader, the shader gives us D3DVS_NOSWIZZLE, which is actually the identity swizzle (.xyzw - 0xE4) and treated as such by d9vk. But this produces incorrect results. Correct results are produced if this is remapped to use the w-component before it reaches DXSO.

In Indiana Jones (#35), which uses vs 1.1: we get both .xyzw (0xE4) and other swizzles including .zwxy (0x4E), .xxxx (0x00), .yyyy (0x55), and .zzzz (0xAA). Remapping the arbitrary swizzles to .wwww produces no noticeable difference, so I assume only the output w is used anyway. Remapping all swizzles to .wwww breaks things, so other replicate swizzles are clearly possible.

.zwxy (0x4E) is weirdest one as it appears to be 0xE4 (xyzw) but with backwards byte order. Either that or it is an intentional arbitrary swizzle (after all, the D3D8 docs don't explicitly forbid it and neither does DXSO).

Questions:

• Should xyzw (or other arbitrary swizzles) be remapped to wwww only in vs1.0, or in vs1.1 too?
• Should xyzw (or other arbitrary swizzles) ever be remapped to wwww in any D3D9 shaders?
• Is there ever a case where xxxx is actually used or are the docs lying?

[AlpyneDreams]

Following code may be helpful to test this:

if (((token & D3DVS_NOSWIZZLE) == D3DVS_NOSWIZZLE)) {
  token &= ~D3DVS_SWIZZLE_MASK;
  token |= (D3DVS_X_W | D3DVS_Y_W | D3DVS_Z_W | D3DVS_W_W);
  Logger::info("RSQ swizzle xyzw");
} else {
  Logger::info(str::format("RSQ swizzle ", std::hex, VSD_SHIFT_MASK(token, D3DVS_SWIZZLE_)));
}

[AlpyneDreams]

For now I will only remap xyzw to wwww, but there may be issues in the future if certain cases are missed. Additionally, this issue will serve as a place to put other instructions with hardcoded swizzles if there are any others.

[AlpyneDreams]

I believe the software vertex VM always uses .wwww, but I'm not sure, and again doing that would break Indiana Jones.