how to use if use yolo ?

naver / multi-hmr

Pytorch demo code and models for Multi-HMR

Other

183 stars 18 forks source link

how to use if use yolo ? #35

Open luoww1992 opened 1 week ago

luoww1992 commented 1 week ago

i get human bbox by use yolo8 from image，then i see the func forward code in model.py. i find you use the project: shubham-goel/4D-Humans to get base smpl pose.

q1, in model.py -- class Model() -- func def detection(), what the idx is? the index of label ?

q2. the self.x_attention_head=HPH(...) is used to get head and body base pose by 4D-Humans ?

q3, if q2 is yes, can i use yolo to replace some code ?

q4, if i use the original code from class SMPLTransformerDecoderHead(nn.Module): in 4D-Humans, can i get the same result ? or a litter different ?

q0 -- other : a plan abou onnx or tensorrt for a higher fps ?

fabienbaradel commented 1 week ago

Hi, the goal of this project it to do both detection and regression using a single network. So we do not build on top of yolo detector.

q1: idx corresponds to the coarse primary keypoint locations
q2: it is an extension of head propose by 4D-Humans
q3/q4: you cannot use 4D-Humans code to get the results. There are two disctints methods.
q0: no plan to release onnx/tensorrt at the moment. Best,

luoww1992 commented 1 week ago

@fabienbaradel about q1, the idx in code is a tuple with 4 tensor: like ( tensor([0], device='cuda:0'), tensor([17], device='cuda:0'), tensor([33], device='cuda:0'), tensor([0], device='cuda:0') ) you say the meaning is the primary keypoint locations, while it doesn't like (x, y). the detail about idx... thank you.

fabienbaradel commented 1 week ago

It corresponds to the token location and not (x,y) location in pixel space :)

coolkidmacho commented 1 week ago

Is there a method to get the X and Y pixel locations of a joint?

fabienbaradel commented 1 week ago

@coolkidmacho this output gives you X/Y pixel locations of SMPL-X joints.

luoww1992 commented 1 week ago

@fabienbaradel What is the orientation of the coordinate system ？ the rotvec in humans is euler ?

fabienbaradel commented 6 days ago

It is a left-hand coordinate system:

X is going from left to right
Y is going from bottom to top
Z is going forward

luoww1992 commented 6 days ago

@fabienbaradel i use a picture to test, but the result is not good. and i use nosmpl to see , the show is wrong, my data: np.save("53.npy", human["rotvec"].cpu().numpy())

Ques.zip