Some obfuscation in the instance function "to_numpy" of class"BVH_file" in retargeting/datasets/bvh_parser.py

[ANYMS-A]

Hi, I was kind of confused about a code snippet in the instance function "to_numpy" of class"BVH_file" in retargeting/datasets/bvh_parser.py.

from line #180 to line #184 if edge: >>>>index = [] >>>> for e in self.edges: >>>>>>>> index.append(e[0]) >>>>rotations = rotations[:, index, :] rotations = rotations.reshape(rotations.shape[0], -1)

According to my understanding so far, after doing: rotations = self.anim.rotations[:, self.corps, :] in line #174 the variable "rotation" will hold the rotation info of the simplified skeleton. Then why we still need to do the operation from line #180 to line #184?

B.T.W could you please explain the "MOTION" part of the .bvh file for me? I searched some explainations from the website, but I didn't get a satisfied one. I guessed each line of the "MOTION" part presents the rotation information of the skeleton's joints in one frame. But how to understand the order of these numbers？

Many thanks!

[PeizhuoLi]

The "MOTION" part works like this: in the "HIERARCHY" part, there are "CHANNELS" segment, in which there are identifiers like "Xposition", "Yrotation". In the "MOTION" part, each line represents one frame and the numbers follow the order of identifiers in HIERARCHY part, e.g. the first 6 numbers represents the position and rotation (Euler angles) for the root joint, respectively. If it's still not clear, I would recommend you to read utils/BVH.py to get more specific description.

After understanding how bvh works, you will find that for a joint with more than one child joint, the child joints are rigid: a same rotation is always applied to all of them, because the rotation of a joint in bvh file is applied to its children instead of itself. But we use "armature" in our paper, which requires each armature (edge) to have different rotations, so we reorganize the representation of rotations, as line #180 - #184 do, to make the rotations of each joint applied to itself.

[ANYMS-A]

Thanks for the patient explaination! But I still had a question, for example, a .bvh file "HIERARCHY" defined as blow:

HIERARCHY ROOT Hips { >>>>OFFSET 0.000000 35.082581 0.000000 >>>>CHANNELS 6 Xposition Yposition Zposition Xrotation Yrotation Zrotation >>>>JOINT Spine >>>>{ >>>>>>>>OFFSET 0.000000 6.876099 0.125700 >>>>>>>>CHANNELS 3 Xrotation Yrotation Zrotation >>>>>>>>JOINT Spine1 >>>>>>>>{...} >>>>>>>> ... >>>>}

>>>>JOINT LeftUpLeg >>>>{ >>>>>>>>OFFSET 10.205400 -3.823500 0.155100 >>>>>>>>CHANNELS 3 Xrotation Yrotation Zrotation >>>>>>>>JOINT LeftLeg >>>>>>>>{...} >>>>} }

Now the 1st \~ 6th number in "MOTION" part represent the position and rotation (Euler angles) for the root joint "Hips", and 7th\~9th numbers represent the rotation of the joint "Spine", then how about the 10th\~12th number? Do them belongs to the joint "Spine1" or the joint "LeftUpLeg"? Many thanks!

[PeizhuoLi]

They belong to Spine1. More specifically, the numbers follow the order that the identifiers appear in plain text.

[ANYMS-A]

They belong to Spine1. More specifically, the numbers follow the order that the identifiers appear in plain text.

Thanks! That's exactly the point that I'm doubting of!

[ANYMS-A]

The "MOTION" part works like this: in the "HIERARCHY" part, there are "CHANNELS" segment, in which there are identifiers like "Xposition", "Yrotation". In the "MOTION" part, each line represents one frame and the numbers follow the order of identifiers in HIERARCHY part, e.g. the first 6 numbers represents the position and rotation (Euler angles) for the root joint, respectively. If it's still not clear, I would recommend you to read utils/BVH.py to get more specific description.

After understanding how bvh works, you will find that for a joint with more than one child joint, the child joints are rigid: a same rotation is always applied to all of them, because the rotation of a joint in bvh file is applied to its children instead of itself. But we use "armature" in our paper, which requires each armature (edge) to have different rotations, so we reorganize the representation of rotations, as line #180 - #184 do, to make the rotations of each joint applied to itself.

If so, does it mean the root joint will never have a rotation applied to itself?

[PeizhuoLi]

I think I made some mistake or inaccuracy before. From the perspective of bvh file definition, each joint is a local coordinate system and connects to its child joints, which are also coordinate systems, with a fixed offset in its local coordinate system. And the rotation of a joint is applied to its coordinate system. So from this perspective, it's applied to itself. Since the offset is fixed, the child joints of one joint is "rigid". It also means a rotation on a joint doesn't affect its position.

But from the perspective of edge (armature), it's kind of subtle. In our paper we represent an edge with the joint corresponding to its end point. (As I mentioned before, we want to make the child joints of a joint not to be rigid, so each edge should be assigned to different rotations.) This means that the rotation corresponding to a joint, is not applied to its corresponding edge (the edge connects it and its parent).

[ANYMS-A]

So from this perspective, it's applied to itself. Since the offset is fixed, the child joints of on

Ah, got it! So the rotation part of "global root position & rotation" feature vector which mentioned in the paper(in Fig.4, the blue one) just means the rotation applied on the root joint's local coordinate system?

[PeizhuoLi]

Yes. However, in the implementation it is copied three times, by line #180 ~ $184. We keep a rotation for each edge.

[ANYMS-A]

Yes. However, in the implementation it is copied three times, by line #180 ~ $184. We keep a rotation for each edge.

Thank you so much!

[LuletterSoul]

@PeizhuoLi Hi, Thanks for your great contribution of this repo ! I am also confused about the code snippet in #180 ~ #184. As far as I know, the e[0] is the parent joint index of each joint, if we only use parent indexes to select rotations, would the complete rotations be preserved? How can we recover original rotations from edges? For example, the effectors (hand, head, and etc.) are not the parents of any joints, where are the original rotations of effectors placing, if we just use the parent indexes to reorder rotation topology?

I also don't understand that why ignore root orientation in build_edge_topology, because it just iterates ranging from 1 to joint_num. So the original root orientation is not adopted over the whole dataset ?

[PeizhuoLi]

Hi, regarding the edge representation, a simple solution could be attach one placeholder child joint to those end-effectors. Alternatively, you may use the joint representation and make some slight change to the pooling process.

The root orientation is not ignored, it actually is duplicated for three times since the joint joints has three children.

[mjdxyr]

I think not only root rotation is repeated ,but also Spine2's rotation is repeated.