tdc22
commented
1 year ago So, after some testing the proposed solution of the previous comment was totally wrong. This is because in the current iteration of the frame interpolation mechanism only the final matrices are actually interpolated. While there would be some numerical algorithms to re-seperate the rotation out from the other transformations done in the 3x3 top-left submatrix, this would be inappropriate at that point.
The true solution to this issue is to actually implement the second intended frame interpolation method: full interpolation. As stated above, the current approach only interpolated the final transformation matrices which leads to the artifacts and inaccuracies demonstrated in the previous comment. The solution would be to build a tree of all matrix operations. This is already done for many operations but most likely you would need to extend this to any matrix operation so you can interpolate the parameters of the operation between the frames to get an accurate interpolation. Some examples of functions that are still missing in the interpolation tree are:
- guPerspective
- guLookAt
- guLookAtF (in z_view.c, func_800AAA9C)
The idea would be to interpolate the parameters of each function between the frames, instead of just the resulting transformation matrices as it is done currently. This might result in worse performance, but more accurate (at least mathematically) interpolation and thus it would be nice to keep a toggle to enable switching back to the more inaccurate but faster interpolation method.
While I'm a big fan of the frame interpolation system, it has a central flaw: currently it simply Lerps between two transformation matrices. That is mathematically incorrect, since those transformation matrices might contain rotation information. Here's a good visualization of the issue, that also hints at the proper solution: https://www.mdpi.com/sensors/sensors-15-19302/article_deploy/html/images/sensors-15-19302-g004-1024.png
To test this, I created a branch where I modified the signs to rotate by 90° in each frame, the sign in front of link's house is in a good place to examine the issue. Branch: https://github.com/tdc22/Shipwright/tree/interpolation-rotation-inaccuracy Commit: https://github.com/tdc22/Shipwright/commit/c4b462d396de141a0f3214a3a92234165911c809 Just compile this, go in front of links house with interpolation turned off. You should see that sign spinning by 90° per frame. Now turn up Interpolation all the way and check the results: the sign is deforming (scaling) while rotating.
Video of the result: https://www.youtube.com/watch?v=R5mZs9G5JvM&t=22
The solution to this is to isolate rotations from the matrix, convert them to a quaternion and apply a Slerp interpolation, instead of a simple Lerp. The remaining parts of the matrix can still be interpolated linearily (I think, I'm not sure about sheering right now). Then recombine the (rotation) quaternion and the rest of the matrix back into a single transformation matrix.