search

shubham-goel / 4D-Humans

4DHumans: Reconstructing and Tracking Humans with Transformers
https://shubham-goel.github.io/4dhumans/
MIT License
1.24k stars 118 forks source link

A question about the difference between PHALP' with HMR2.0 and 4DHumans #29

Open jdobbang opened 1 year ago

jdobbang commented 1 year ago

Hi! I'm really appreciate for you great work!! I followed your work about 3d feature based multi-person tracking model since T3DP model proposed.

I have a question during reading your paper.

What is the difference between PHALP' with HMR 2.0 and 4DHumans? I thought these two are same but at Table3 and 4, these seemed to show different performance in tracking and I'm not sure there are two seperate versions in the code.

shubham-goel commented 1 year ago

Hi @jdobbang, thank you for your interest in our work.

PHALP'+HMR2 simply replaces PHALP's poses (that come from HMAR) with HMR2 but continues to use the old appearance and future no pose prediction.

4DHumans uses a new pose prediction network (Figure 3) and appearance head, leading to better results.

Hope this helps!

jdobbang commented 1 year ago

Thank you for your answer!

Based on your answer, I have few more questions.

  1. Then, PHALP’ predicts SMPL pose parameter poses using verification of HMMR ? I am asking this because before, PHALP predicts 2048 dimension feature pose with HMMR. And I am curious that you trained HMMR for predict pose like the shape of the output of HMR 2.0
  2. I cannot find the explanation about the new appearance head of H4D, like what is its input and the difference between the old appearance head.
  3. I’m wonder the versions of codes represents PHALP’ or H4D?

Thank you

kidkych commented 1 year ago

Hi @jdobbang, thank you for your interest in our work.

PHALP'+HMR2 simply replaces PHALP's poses (that come from HMAR) with HMR2 but continues to use the old appearance and future pose prediction.

4DHumans uses a new pose prediction network (Figure 3) and appearance head, leading to better results.

Hope this helps!

I'd like to echo question 3. from jdobbang. Based on your quoted reply and having a look at track.py it seems like you've only released the code for PHALP` + HMR2. Is 4DHumans (specifically with the new pose predictor) available yet?

spicer-c commented 1 year ago

Is there an answer to this question? Thanks!

Decide02 commented 2 months ago

Hi, 4DHumans uses Pose Transformer V2 for pose prediction, which is different from PHALP. However, I've tried running both the latest PHALP and 4DHumans repositories with AVA datasets, and I don't see any difference in the results. Do you have any examples where there is enough of a performance difference to separate the two repositories? Thank you!