vec3.transformMat4Upper3x3 doesn't take w

[magcius]

It's common to want both a point transform, and a vector transform. My own utility library has vec3TransformMat4w0 and vec3TransformMat4w1 to handle these cases, respectively.

Also, I would consider making the function take m, v ordering for consistency with m * v syntax in WGSL.

[greggman]

I'm confused. w, whether 0 or 1 would not be used in a 3x3 matrix vector3 multiply. Here's 4x4 vec4

  dst[0] = m[0] * x + m[4] * y + m[ 8] * z + m[12] * w;
  dst[1] = m[1] * x + m[5] * y + m[ 9] * z + m[13] * w;
  dst[2] = m[2] * x + m[6] * y + m[10] * z + m[14] * w;
  dst[3] = m[3] * x + m[7] * y + m[11] * z + m[15] * w;

if w is zero than the last column goes away. If w is one well, on 3x3 matrix that last column doesn't exist so it's all zero.

[magcius]

I guess I'd prefer something in between transformMat4 and transformMat4Upper3x3 -- respects translation but assumes the matrix is affine and the last row is 0,0,0,1

[greggman]

  w = 1
  dst[0] = m[0] * x + m[4] * y + m[ 8] * z + 0 * w;
  dst[1] = m[1] * x + m[5] * y + m[ 9] * z + 0 * w;
  dst[2] = m[2] * x + m[6] * y + m[10] * z + 0 * w;
  dst[3] = w;

This is what it seems like you said, last row (12,13,14,15) is 0, 0, 0, 1. Which is what it's already doing

Can you write the code that you want to happen?

[magcius]

Sorry, I meant column. Too used to D3D conventions.

  dst[0] = m[0] * x + m[4] * y + m[ 8] * z + m[12];
  dst[1] = m[1] * x + m[5] * y + m[ 9] * z + m[13];
  dst[2] = m[2] * x + m[6] * y + m[10] * z + m[14];

[greggman]

There is no m[12], m[13], m[14] in a 3x3 matrix and vec3.transformMat4 is effectively already exactly that

  w = 1
  dst[0] = (m[0] * x + m[4] * y + m[ 8] * z + m[12]) / w;
  dst[1] = (m[1] * x + m[5] * y + m[ 9] * z + m[13]) / w;
  dst[2] = (m[2] * x + m[6] * y + m[10] * z + m[14]) / w;