Serious scaling issues when objects are parented

[QuantumCoderQC]

Description

I encountered a major scaling issue when I was trying to understand transforms in Armory. In the test scene I have four cubes. Two of them have a parent child relationship, and the other two dont. The green cubes are scaled by 0.5 along the X axis. The red cubes are simply rotated along z axis by 45 degrees. The only diffference between the two sets of cubes is parenting. Note that scaling and rotations were not "applied" in Blender.

The image above shows parented cubes on the right and un-parented ones on the left.

When viewed from orthographic projection, here is the comparison between Blender and Armory

From the image, the parented object has wrong scaling. While the unparented one looks correct.

However, when I checked the local transform of the objects in blender and Armory, they were a bit different:

arm/MyTrait.hx:18: Cube.002 :
arm/MyTrait.hx:19: Local Transform =
arm/MyTrait.hx:84: Loc:
arm/MyTrait.hx:85: (0, 0, 3, 1)
arm/MyTrait.hx:86: Rot
arm/MyTrait.hx:87: 0, 0, -0.36288262477061145, 0.9318348569567418
arm/MyTrait.hx:88: Scl
arm/MyTrait.hx:89: (1.5683433228513, 1.5683434814930588, 1, 1)

The differences were that:

1. The local rotation quaternion (object.transform.rot) in Armory was not normalized.
2. The local scale (object.transform.scale) in Armory was different form that of Blender

To Reproduce Use the provided blend file and run the project.

Expected behavior The scaling of objects must be proper, even if object scaling is not applied beforehand.

System Blender: 293 Armory: v2021.5 ($Id: 99497c80048d5b0614144ab4b87fb2303e416be2 $) OS: Windows10

Test File ScaleRotBugTestFile.zip

[koji-k]

This issue is same cause maybe...? https://github.com/armory3d/armory/issues/2208

[ItsCubeTime]

I wouldnt be surprised if this is the root of why physics wont work like expected when using scales other than 1 as well.

[QuantumCoderQC]

This issue seems to be a bit deeper than I anticipated. The problem is more mathematical than code-related. Even Blender has this issue in some cases. Here is a thread that I opened in Blender stack exchange to ask about it, but could not get a clear answer.

The problem is about decomposing and composing the local transform matrix. Armory uses this every time an object is spawned or a transform is modified. In some cases, the Decompose --> Re-Compose does not always return the same transform matrix. This causes objects to have wrong transforms.

This issue is same cause maybe...?

As for the animation, it uses delta transforms, so it does not decompose/ compose matrix each frame I believe. So might not be the same problem.

I wouldn't be surprised if this is the root of why physics won't work like expected when using scales other than 1 as well.

Physics uses world matrices to set/get transforms and not local matrices, So Physics should not be affected by this AFAIK.

[ItsCubeTime]

I wouldn't be surprised if this is the root of why physics won't work like expected when using scales other than 1 as well.

Physics uses world matrices to set/get transforms and not local matrices, So Physics should not be affected by this AFAIK.

Sure, but I mean when you parent an object to a object with physics enabled, if that child has a scale other than 1 this results in the rigid body mesh seemingly moving towards/away from that object (and pulling that child with it ofc) (I believe, not 100% sure if it really moves towards/awway from that object). Although there might be more issues related to physics contributing to this since you cant even have the simulated object itself in a scale other than 1, unless theres some method/function somewhere being reused in several locations causing similar issues all over the software.

[QuantumCoderQC]

Okay, so after digging a bit deeper into the problem, I found the actual issue.

When it occurs: This issue occurs only when:

1. Mat4.decompose() is used on a local transform matrix of the object
2. The said object must have a parent with non-uniform scale

It is worth mentioning that decomposing works fine on world matrices.

The problem: When the above two conditions are satisfied, the child object's local matrix is no longer made of pure Translation, Rotation and Scale. It has an additional component Shear due to the non uniform scaling of parent object. Hence when Mat4.decompose() is called, the returned Translation, Rotation and Scale are improper.

Similar cases: I found a very similar issue with three.js. They finally decided to simply tell users that such cases were "not supported".

Probable solutions: From what I understand, this could go in 3 different directions:

1. Follow what three.js did and simply tell that such object pairs are not allowed or supported. Maybe add an export warning.
2. Improve the decompose() and compose() methods to somehow include shear. (Not sure if this would work)
3. Change the whole transform logic to use matrices only. No decompose/ compose of local matrices. (Blender probably does this)

[MoritzBrueckner]

2. Improve the decompose() and compose() methods to somehow include shear. (Not sure if this would work)

I'm not sure if I really understand the issue, but wouldn't it be possible to multiply the transformation matrix of an object with the inverse of the parent's matrix (recursively) before decomposition to remove the shear? If you have matrix = ParentMatrix * T * R * S (like in the linked three.js issue), wouldn't be ParentMatrixInv * ParentMatrix * T * R * S be equal to T * R * S?

3. Change the whole transform logic to use matrices only. No decompose/ compose of local matrices. (Blender probably does this)

Independent of whether we completely use this approach, could it help to make the calculations more robust even in case that there still is a decompose function for user code?

[QuantumCoderQC]

Thanks for asking, @MoritzBrueckner.

If you have matrix = ParentMatrix T R S (like in the linked three.js issue), wouldn't be ParentMatrixInv ParentMatrix T R S be equal to T R * S?

Yes, the above logic holds good. The matrices will be equal. However, the R matrix will no longer be orthogonal(pure rotation) if the parent matrix is scaled non-uniformly. This is because the R matrix now contains shear information along with the rotation information. The consequence of R matrix not being orthogonal is that it cannot be converted into a pure rotation quaternion*. Hence why the decompose() method does not work. This also means that the local transform matrix that has shear component to it cannot be represented by pure translation, rotation and scale matrices**.

Independent of whether we completely use this approach, could it help to make the calculations more robust even in case that there still is a decompose function for user code?

Yes, the users can still use the decompose() method, but has to be careful where they use. That is, they should not use it only in cases where the said object has a parent which is scaled non-uniformly.

*More details on converting rotation matrix to quaternion here

**A paper about decomposing a matrix into its translation, rotation, shear and scale matrix components: paper

Please feel free to ask any question if still exists

[QuantumCoderQC]

Some more info:

The reason for a sheared matrix in the first place is because Armory exports only local transform matrix from Blender. export code. And this matrix has shear information.

If Armory was able to handle such a matrix with shear, the child object would appear exactly as in the Blender's viewport.(image for reference) But since there is a decompose() and compose() involved when initially spawning the object, the shear is lost in the process. This results in the object being not as viewed in Blender's viewport.

Note that this cannot be solved by simply exporting location, rotation, and scale matrices from Blender. This is because when multiplied ParentMatrix * T * R * S would still result in a matrix without shear, and the object would still not appear as in Blender viewport

[MoritzBrueckner]

Thanks a lot for the detailed answers!

What I meant above is, why can't we change the exporter to export a non-sheared matrix? In your Blender stack exchange question, BatFINGER linked to another question where they state that ob.matrix_local = ob.matrix_parent_inverse @ ob.matrix_basis (with @ being the usual matrix multiplication in Blender).

This way (even if matrix_basis would not be accessible, like in the exported game with the current exporter), it would be possible to retrieve the basis matrix from the local matrix and the parent matrix. Then the rotation could be decomposed from the basis matrix and later be combined with the rotation of matrices higher in the hierarchy (this might be laborious though). Or does even the basis matrix contain shear (if yes, how)? Please forgive me, my linear algebra skills are a little rusty^^

But you don't have to explain that to me if you don't have time for this and I don't want to clutter this issue with unnecessary discussion, you know what's going on and that's what is important. If you want we can continue to talk about this on Discord, but it's not that important. Thanks again!

[QuantumCoderQC]

I have no issues answering questions, someone in the future might refer to this issue.

Then the rotation could be decomposed from the basis matrix and later be combined with the rotation of matrices higher in the hierarchy (this might be laborious though).

Right, this is how Blender currently handles transforms. The matrix_basis is not and will not be a sheared matrix. Hence it can be decomposed and composed without any issues. This decomposed form of basis matrix is what one sees in the Blender's Transform tab in the 3D viewport.

The method you mentioned does not include decomposing or composing of the local matrix. Just matrix multiplications to arrive at the local matrix. Hence, this is a probable solution (I already mentioned this in my previous post "Probable solution No. 3").

Note that following the above solution still results in a sheared local matrix, This is because we multiply ob.matrix_parent_inverse @ ob.matrix_basis and ob.matrix_parent_inverse introduces shear. But we are in the green as long as we don't decompose or compose this local matrix. And, Armory would render the object just like in Blender.

What I meant above is, why can't we change the exporter to export a non-sheared matrix?

A sheared matrix in itself is not the main issue here. Just the decomposing and re-composing of such a sheared matrix is. So, even if we export a sheared local matrix, as long as we don't decompose and compose this matrix, Armory would render the object just like in Blender.

Translation, rotation and scaling matrices can be further multiplied to this sheared local matrix to achieve local transformations without issues.

I will make a more detailed post about how Blender handles these transforms on the Armory Forums later. That way, we could keep this thread a bit clean.