search

mks0601 / 3DMPPE_ROOTNET_RELEASE

Official PyTorch implementation of "Camera Distance-aware Top-down Approach for 3D Multi-person Pose Estimation from a Single RGB Image", ICCV 2019
MIT License
477 stars 65 forks source link

Demo results on custom image/video #17

Closed Naveennavik closed 4 years ago

Naveennavik commented 4 years ago

Hi,

Your work seems to be great and I was wondering if you could include a demo.py code that takes a single image or video as input and provides the 3d visual output.

mks0601 commented 4 years ago

Hi, thanks for having interest in our research! Yes I heard this request often, but unfortunately, it is not easy to write a single script because the PoseNet and RootNet are separated models :( Also, to be honest, I'm busy nowadays :( I'll make some demo script sometime when I have time!

mks0601 commented 4 years ago

I think you can refer this code? https://github.com/mks0601/3DMPPE_POSENET_RELEASE/issues/36

DA-fromindia commented 4 years ago

@mks0601 Great work and kudos to You!!

It would be a great help If you create a demo.py file in the future. So that people can pass Image/Video and get 3D visualization! I think, it will become more straightforward!

@Naveennavik agree!! Thanks!

mks0601 commented 4 years ago

Hi guys, I made a demo code. Check this out. https://github.com/mks0601/3DMPPE_ROOTNET_RELEASE/blob/master/demo/demo.py

DA-fromindia commented 4 years ago

@mks0601 Thanks alot for this can you tell me which model did you use?

can you please share this model -"AssertionError: Cannot find model at ./snapshot_10.pth.tar"

and also , it will help other people if you add "QUICK DEMO" in ReadME file.

mks0601 commented 4 years ago

Added!

DA-fromindia commented 4 years ago

@mks0601 Thanks alot when i use this "Human3.6M+MPII_24/Protocol2/snapshot_24.pth.tar" and "Human3.6M+MPII_24/Protocol1/snapshot_24.pth.tar"

i get this

RuntimeError: Error(s) in loading state_dict for DataParallel: Missing key(s) in state_dict: "module.root.deconv_layers.0.weight", "module.root.deconv_layers.1.weight", "module.root.deconv_layers.1.bias", "module.root.deconv_layers.1.running_mean", "module.root.deconv_layers.1.running_var", "module.root.deconv_layers.3.weight", "module.root.deconv_layers.4.weight", "module.root.deconv_layers.4.bias", "module.root.deconv_layers.4.running_mean", "module.root.deconv_layers.4.running_var", "module.root.deconv_layers.6.weight", "module.root.deconv_layers.7.weight", "module.root.deconv_layers.7.bias", "module.root.deconv_layers.7.running_mean", "module.root.deconv_layers.7.running_var", "module.root.xy_layer.weight", "module.root.xy_layer.bias", "module.root.depth_layer.weight", "module.root.depth_layer.bias". Unexpected key(s) in state_dict: "module.head.deconv_layers.0.weight", "module.head.deconv_layers.1.weight", "module.head.deconv_layers.1.bias", "module.head.deconv_layers.1.running_mean", "module.head.deconv_layers.1.running_var", "module.head.deconv_layers.1.num_batches_tracked", "module.head.deconv_layers.3.weight", "module.head.deconv_layers.4.weight", "module.head.deconv_layers.4.bias", "module.head.deconv_layers.4.running_mean", "module.head.deconv_layers.4.running_var", "module.head.deconv_layers.4.num_batches_tracked", "module.head.deconv_layers.6.weight", "module.head.deconv_layers.7.weight", "module.head.deconv_layers.7.bias", "module.head.deconv_layers.7.running_mean", "module.head.deconv_layers.7.running_var", "module.head.deconv_layers.7.num_batches_tracked", "module.head.final_layer.weight", "module.head.final_layer.bias".

can you please help me to solve this

mks0601 commented 4 years ago

It seems you are trying to load PoseNet, not RootNet

DA-fromindia commented 4 years ago

It seems you are trying to load PoseNet, not RootNet

https://drive.google.com/drive/folders/1V7fuKtzNovLte2jSF2fw0gyBiEZQFQ_k

i download from here is it right??

if it is not right, please share a link of it

DA-fromindia commented 4 years ago

@mks0601 MuCo-3DHP+MSCOCO/snapshot_18.pth.tar THIS WORKS

but it did not return 3D _pose!

and this is a 2D pose i get from that image

input

output_root_2d _

mks0601 commented 4 years ago

This is RootNet repo. Please visit PoseNet repo.

mks0601 commented 4 years ago

https://github.com/mks0601/3DMPPE_POSENET_RELEASE

DA-fromindia commented 4 years ago

@mks0601 Thanks for this when i run it 2D pose is well, 3D pose is not good i think output_pose_2d

I used both models one by one it is not giving a proper output 3D_pose

please have a look and help me in this

mks0601 commented 4 years ago

Have you rotated the 3d pose in the window?

DA-fromindia commented 4 years ago

@mks0601 i am getting this error

pose_3d[:,2] = (pose_3d[:,2] / cfg.depth_dim * 2 - 1) * (cfg.bbox_3d_shape[0]/2) + root_depth

TypeError: unsupported operand type(s) for +: 'float' and 'NoneType'

in derailed

No handles with labels found to put in legend. Traceback (most recent call last): File "C:/Users/Admin/Downloads/3DMPPE_POSENET_RELEASE-master/demo/demo.py", line 124, in <module> pose_3d[:,2] = (pose_3d[:,2] / cfg.depth_dim * 2 - 1) * (cfg.bbox_3d_shape[0]/2) + root_depth TypeError: unsupported operand type(s) for +: 'float' and 'NoneType'

mks0601 commented 4 years ago

Please do not report all errors here and take a look at the code.

# camera back-projection
focal = (None, None) # focal length of x-axis, y-axis. please provide this. if do not know, set normalized one (1500, 1500).
princpt = (None, None) # princical point of x-axis, y-aixs. please provide this. if do not know, set (original_width/2, original_height/2).

You should provide intrinsics here. It is applicable to PoseNet as well.

DA-fromindia commented 4 years ago

@mks0601 i am working with this "https://github.com/mks0601/3DMPPE_POSENET_RELEASE" (posenet)

when i set focal = (1500, 1500) princpt = (500, 500)

it still gives a error and same 3D rendering

pose_3d[:,2] = (pose_3d[:,2] / cfg.depth_dim * 2 - 1) * (cfg.bbox_3d_shape[0]/2) + root_depth

TypeError: unsupported operand type(s) for +: 'float' and 'NoneType'

should i pass anything root_depth = None here??

mks0601 commented 4 years ago

L106 says: # if you do not know root_depth, focal, and princpt, visit https://github.com/mks0601/3DMPPE_ROOTNET_RELEASE/tree/master/demo. Please use output of RootNet.

DA-fromindia commented 4 years ago

@mks0601 https://github.com/mks0601/3DMPPE_ROOTNET_RELEASE

models -: https://drive.google.com/drive/folders/1xqBRlye_YGhO8oU0iwisNfLXm23_VQPh this is, right?

mks0601 commented 4 years ago

# if you do not know root_depth, focal, and princpt, visit https://github.com/mks0601/3DMPPE_ROOTNET_RELEASE/tree/master/demo <- here is url. And you can see quick demo of README.

DA-fromindia commented 4 years ago

@mks0601 https://github.com/mks0601/3DMPPE_ROOTNET_RELEASE/blob/master/demo/demo.py here on line 95-96 you set focal = (None, None) princpt = (None, None) if i run it , it gives me this error Traceback (most recent call last): File "C:/Users/Admin/3DMPPE_ROOTNET_RELEASE-master/demo/demo.py", line 116, in <module> root_3d = pixel2cam(root_3d[None,:], focal, princpt) File "C:\Users\Admin\3DMPPE_ROOTNET_RELEASE-master\main\..\common\utils\pose_utils.py", line 14, in pixel2cam x = (pixel_coord[:, 0] - c[0]) / f[0] * pixel_coord[:, 2] TypeError: unsupported operand type(s) for -: 'float' and 'NoneType'

Now

if i comment it, means it is now pass as above (line 70 and 73) and run it works perfectly with no error

2D output is same as this -: https://github.com/mks0601/3DMPPE_ROOTNET_RELEASE/issues/17#issuecomment-668523628

and it is not creating 3D pose window!! (can you please download this and run in your system?, i used MuCo-3DHP+MSCOCO/snapshot_18.pth.tar )

I tried to figure out, but i still do not know where did i go wrong, i followed all steps that you suggested