search

wyysf-98 / GenMM

Apache License 2.0
259 stars 21 forks source link

Implemented codes about Skeleton-aware motion patches #12

Closed yongw00 closed 9 months ago

yongw00 commented 9 months ago

Hi, I'd like to see how the whole skeleton can split into sub-sets. However, I cannot find any parts about splitting the skeleton depending on any graph algorithms. The argument "Skeleton-aware" is even not used in the codes. How can I find the related parts? Thanks.

wyysf-98 commented 9 months ago

Hi, as it is mentioned in the paper, there is no graph algorithms performed in the skeleton aware component as it is hard to properly split into parts automatically for arbitrary skeleton. So we manually split it into parts in the paper. Readme also mentioned that we did not release the related code yet, as it is non-trival for a clean code. (I may upload a new branch for it) A simple way you can try is to send a splited joints id group and use for-loop in https://github.com/wyysf-98/GenMM/blob/main/GenMM.py#L152 to compute the loss for each joints group. Then combine them to the final results.

yongw00 commented 9 months ago

@wyysf-98 o-ok, understood. Thank you for the quick reply!

PeterWangyi commented 7 months ago

@wyysf-98 Maybe any planned updates scheduled? Looking forward to the release of skeleton-aware component！

wyysf-98 commented 7 months ago

Hi, I am not sure about the specific time. The skeleton-aware component is skeleton-specific and we need to manually split joints into groups. I think the current code is sufficient to implement this function and I may finish a brief toy version in the next month. (maybe)

minggini commented 7 months ago

Wouldn't it take longer when looping over the splited joints? I was wondering how many body parts did you divided when measuring the inference time, and if this "looping" is how you guys implemented. Because when I split a body into 22 parts, and loop through the body parts as you mentioned, it seemed to take quite a long time.

wyysf-98 commented 7 months ago

Wouldn't it take longer when looping over the splited joints? I was wondering how many body parts did you divided when measuring the inference time, and if this "looping" is how you guys implemented. Because when I split a body into 22 parts, and loop through the body parts as you mentioned, it seemed to take quite a long time.

Sure, if you trivially split body parts into groups with different numbers of joints, the time of using for-loop will increase linearly, which is 22 times in your case. If the number of joints of each group is the same, you can batchfy the data. Or you can pad zeros at the end of the tensor to make all groups with the same size. Moreover, you may try nested tensors to speed up if you want. https://pytorch.org/docs/stable/nested.html

PeterWangyi commented 7 months ago

@wyysf-98 Thank you for your reply! So in my understanding, the current code does not have the skeleton-aware component module. So now I input a piece of my own bvh file. Whether it is generated using python command or using blender add on, the whole body is used to generate the action?

If so, it means there is no restriction on the footsteps. Some sliding issues were observed in my case (because there was some foot movement in the source action, adding noise made the movement look weird).

Anyway, I will try the looping method mentioned above to punish the foot loss. I don’t know if it will be effective. Maybe it’s polite to ask if you have any good suggestions?