Open orenwang opened 4 weeks ago
Thank you for your feedback and for checking out the project! :)
The rotmats
output as identity matrices is expected behavior in your described situation. Given that the keyframe values are all zero, it implies that there are no additional rotations applied to the bones.
As I understand for BVH files:
local_rotations
being zero means no rotation from the initial (rest) pose.rotmats
being identity matrices means that the bones remain in their rest pose without any further rotation.The BVH implementation is based on the following document: https://research.cs.wisc.edu/graphics/Courses/cs-838-1999/Jeff/BVH
Another way to think about it, rotmats
are applied first and then translations (pos
).
However, I may be missing something, if you could provide the outcome of the expected behavior.
If you have any further questions or need more clarifications, feel free to ask!
Yeah I should have provided the behavior that I would expect.
The
rotmats
being identity matrices means that the bones remain in their rest pose without any further rotation
But lets look at the values of pos
above:
tensor([[[0., 0., 0.],
[0., 0., 1.]]])
They are non-zero. Actually result of pos
is exactly what I would expect because even bones are in their rest pose, their world positions may not be zero. Same could be said for their rotations, right? From a rest pose, computed pos
being "world" but rotmats
being rather "local" is where I am confused.
To further make my point, in Blender we can get a world matrix of the second bone by running:
import bpy
arm_ob = bpy.data.objects["test"]
print(arm_ob.pose.bones[1].matrix)
<Matrix 4x4 (0.5000, -0.7071, 0.5000, 0.0000)
(0.7071, 0.0000, -0.7071, 0.0000)
(0.5000, 0.7071, 0.5000, 1.0000)
(0.0000, 0.0000, 0.0000, 1.0000)>
The position part matches the behavior of PyMotion. But the rotation part does not. That's how I understand world rotations too because even bones are in their rest pose without any further rotations, their world rotations may not be zero.
Thank you for the detailed explanation! Now I understand the problem better. In my understanding, it is a problem of conventions, which I did not realize before because right now, we only support the BVH file format.
The BVH file does not explicitly declare the initial orientation of each bone, this should be automatically computed. For example, assuming a primary axis having the direction of the parent-child vector, and then assuming some second axis. From this, an initial rotation is computed, which I believe is what Blender is doing under the hood. The problem is that all this assumes a lot, since the base orientation is not explicitly declared in the BVH files.
For now, PyMotion does not assume any of this, and the base orientation is equal to the identity orientation. It all depends on what we consider our initial bone orientation. I understand this may yield counterintuitive results, such as that some bones may have the parent-child vector aligned with, e.g., the Y-world axis, others with the X-world axis, or any vector in general.
I've been reading about it, and I think it may be desirable to align the parent-child direction of all bones with a single world axis, such as the Y-axis. This will help with the interpretation of rotmats
and any 3D object alignment. However, this would change the base joint pose representation from a 3D vector (offsets) to a length value and a 3D rotation. But the behaviour would be similar to Blender.
I will write all this down in my roadmap and think of a flexible way to integrate it with the project for all use cases. Hopefully I can work on it soon! :) I will leave this issue open and come back to it once I find a solution.
I leave this here for future reference: https://staffwww.dcs.shef.ac.uk/people/S.Maddock/publications/Motion%20Capture%20File%20Formats%20Explained.pdf
Thank you for pointing out this issue!
First of all, amazing project! Thank you for sharing this valuable work.
I am a little confused, though, about "Compute world positions and rotations from a BVH file" part in the README.
Given a simple bvh
test.bvh
with an all-zero keyframe but non-zero offsets:Following code
gives
rotmats
of valuewhich indicates "world rotations" are all zero. So if I'm understanding it correctly
rotmats
is pre-offsets (a.k.a. without applying the rest pose). Is this expected behavior?