search

zczcwh / PoseFormer

The project is an official implementation of our paper "3D Human Pose Estimation with Spatial and Temporal Transformers".
497 stars 74 forks source link

AttributeError: 'NoneType' object has no attribute 'astype' of Wild video #27

Open Whisht opened 2 years ago

Whisht commented 2 years ago

I follow the instruction

With the same pre-processed video as well as same hyperparameters, VideoPose3D runs normally, but PoseFormer has this bug:

Traceback (most recent call last):
  File "/data/xielongfei/code/github/PoseFormer-main/run_poseformer.py", line 589, in <module>
    prediction = evaluate(gen, return_predictions=True)
  File "/data/xielongfei/code/github/PoseFormer-main/run_poseformer.py", line 511, in evaluate
    inputs_3d = torch.from_numpy(batch.astype('float32'))
AttributeError: 'NoneType' object has no attribute 'astype'.

It seems the dataloading procedure has some problem.

FinallyKiKi commented 2 years ago

I also have this qustion. BTW, I change the code somewhere and get this result: 微信图片_20220325101205

ClimberXIE commented 2 years ago

I also have this question! And I change the code but get bad results. image

ghost commented 2 years ago

@FinallyKiKi @ClimberXIE This Net input format is human3.6M's 17 points You need to change the COCO format 17 points from the detectron2 of VideoPose3d to human3.6M's 17 points As for the code, click here for reference: https://github.com/fabro66/GAST-Net-3DPoseEstimation/blob/master/tools/mpii_coco_h36m.py and then

# first load your videopose 2d coco points
loaded = np.load('PoseFormer/data/data_2d_custom_myvideos.npz', allow_pickle=True)
data = loaded['positions_2d'].item()
for i in data:
    # convert them to human3.6M format
    data[i]['custom'][0]=coco_h36m(data[i]['custom'][0])
np.savez_compressed('data_2d_custom_myvideos', positions_2d=data, metadata=loaded['metadata'])
ClimberXIE commented 2 years ago

Thank you for your reply !I have solved the problem now.

------------------ 原始邮件 ------------------ 发件人: "zczcwh/PoseFormer" @.>; 发送时间: 2022年3月31日(星期四) 中午12:00 @.>; @.**@.>; 主题: Re: [zczcwh/PoseFormer] AttributeError: 'NoneType' object has no attribute 'astype' of Wild video (Issue #27)

@Whisht @FinallyKiKi @ClimberXIE This Net input format is human3.6M's 17 points You need to change the COCO format 17 points from the detectron2 of VideoPose3d to human3.6M's 17 points As for the code, click here for reference: https://github.com/fabro66/GAST-Net-3DPoseEstimation/blob/master/tools/mpii_coco_h36m.py and then

first load your videopose 2d coco points loaded = np.load('PoseFormer/data/data_2d_custom_myvideos.npz', allow_pickle=True) data = loaded['positions_2d'].item() for i in data: # convert them to human3.6M format data[i]['custom'][0]=coco_h36m(data[i]['custom'][0]) np.savez_compressed('data_2d_custom_myvideos', positions_2d=data, metadata=loaded['metadata'])

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

ghost commented 2 years ago

@ClimberXIE can you share your wild video inference result if you are free? In my videos the results with gt81f.bin are not good as the example author gives. I want to know reason that is the input convert or the net weight caused it🤔

my email address is momiji_ao@outlook.com

Thank you!:)

update: (my result)

  1. the convert visualization image
  2. video output image
wshjtuk commented 2 years ago

Thank you for your reply !I have solved the problem now. … ------------------ 原始邮件 ------------------ 发件人: "zczcwh/PoseFormer" @.>; 发送时间: 2022年3月31日(星期四) 中午12:00 @.>; @.**@.>; 主题: Re: [zczcwh/PoseFormer] AttributeError: 'NoneType' object has no attribute 'astype' of Wild video (Issue #27) @Whisht @FinallyKiKi @ClimberXIE This Net input format is human3.6M's 17 points You need to change the COCO format 17 points from the detectron2 of VideoPose3d to human3.6M's 17 points As for the code, click here for reference: https://github.com/fabro66/GAST-Net-3DPoseEstimation/blob/master/tools/mpii_coco_h36m.py and then # first load your videopose 2d coco points loaded = np.load('PoseFormer/data/data_2d_custom_myvideos.npz', allow_pickle=True) data = loaded['positions_2d'].item() for i in data: # convert them to human3.6M format data[i]['custom'][0]=coco_h36m(data[i]['custom'][0]) np.savez_compressed('data_2d_custom_myvideos', positions_2d=data, metadata=loaded['metadata']) — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

Thank you for your reply !I have solved the problem now. … ------------------ 原始邮件 ------------------ 发件人: "zczcwh/PoseFormer" @.>; 发送时间: 2022年3月31日(星期四) 中午12:00 @.>; @.**@.>; 主题: Re: [zczcwh/PoseFormer] AttributeError: 'NoneType' object has no attribute 'astype' of Wild video (Issue #27) @Whisht @FinallyKiKi @ClimberXIE This Net input format is human3.6M's 17 points You need to change the COCO format 17 points from the detectron2 of VideoPose3d to human3.6M's 17 points As for the code, click here for reference: https://github.com/fabro66/GAST-Net-3DPoseEstimation/blob/master/tools/mpii_coco_h36m.py and then # first load your videopose 2d coco points loaded = np.load('PoseFormer/data/data_2d_custom_myvideos.npz', allow_pickle=True) data = loaded['positions_2d'].item() for i in data: # convert them to human3.6M format data[i]['custom'][0]=coco_h36m(data[i]['custom'][0]) np.savez_compressed('data_2d_custom_myvideos', positions_2d=data, metadata=loaded['metadata']) — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

Hello, I met the same problem but still fill to restore the bug by using coco_h36m transform. I use the script after Step4 in the VideoPose3D tutorial but still exist the same bug. Are there any other tricky bugs? Thanks.

wshjtuk commented 2 years ago

@ClimberXIE can you share your wild video inference result if you are free? In my videos the results with gt81f.bin are not good as the example author gives. I want to know reason that is the input convert or the net weight caused it🤔

my email address is momiji_ao@outlook.com

Thank you!:)

update: (my result)

  1. the convert visualization image
  2. video output image

Hello, I met the same problem but still fill to restore the bug by using coco_h36m transform. I use the script after Step4 in the VideoPose3D tutorial but still exist the same bug. Are there any other tricky bugs? I have also Emailed you, looking forward to your reply! Thanks.

ghost commented 2 years ago

@Whisht Yes, there is another problem with the code. Please refer to the email for details (In a nutshell, remove inputs_3d from the original code when you use it to visualizations). You need to change the code somewhere. This code is for the problem of FinallyKiKi and ClimberXIE.

TomCatOnline commented 2 years ago

@Whisht Yes, there is another problem with the code. Please refer to the email for details (In a nutshell, remove inputs_3d from the original code when you use it to visualizations). You need to change the code somewhere. This code is for the problem of FinallyKiKi and ClimberXIE.

thanks for you solution but can you specify where and what code exactly should i make changes to make visualizations run? thanks again!

safsin commented 2 years ago

I'm trying to infer 3d human pose on custom videos using provided pretrained model (gt81f.bin). I've done the following preprocessing -

  1. Input 2d keypoints are in H3.6m format
  2. Input 2d keypoints are normalized keeping width in the range [-1, 1] and maintaining aspect ratio.
  3. Output 3d keypoints are in H3.6m format (https://github.com/facebookresearch/VideoPose3D/issues/36), i.e. same as Videopose3d output.

Still, I end up with randomly located keypoints. Anything I might be missing??

Also, @AO2233 I'd be grateful if you can please share the email contents (that you shared with @Whisht ) here.

TomCatOnline commented 2 years ago

@safsin Have you recieved their email? It would be kind if you could tell me how you did your preprocessing step 1、2、3, please. Looking forward to your reply, thank you very much!

safsin commented 2 years ago

@TomCatOnline I haven't received any emails yet. These are the preprocessing steps I followed (which doesn't seem to work for me) in detail -

  1. The input 2d keypoints should be in H3.6m format, unlike the COCO format most 2d human pose detectors' output in.
  2. Normalize the input 2d keypoints in the range [-1, 1].

You can also try using https://github.com/paTRICK-swk/P-STMO. It has much better examples and codes for in-the-wild human pose3d detection. Better look into it.

TomCatOnline commented 2 years ago

@TomCatOnline I haven't received any emails yet. These are the preprocessing steps I followed (which doesn't seem to work for me) in detail -

  1. The input 2d keypoints should be in H3.6m format, unlike the COCO format most 2d human pose detectors' output in.
  2. Normalize the input 2d keypoints in the range [-1, 1].

You can also try using https://github.com/paTRICK-swk/P-STMO. It has much better examples and codes for in-the-wild human pose3d detection. Better look into it.

I was traped by detectron2 output coco format 2d keypoints but don't know how to convert to h36m format, will look into it, thanks for your quick reply!

ghost commented 2 years ago

I'm trying to infer 3d human pose on custom videos using provided pretrained model (gt81f.bin). I've done the following preprocessing -

  1. Input 2d keypoints are in H3.6m format
  2. Input 2d keypoints are normalized keeping width in the range [-1, 1] and maintaining aspect ratio.
  3. Output 3d keypoints are in H3.6m format (Mapping of model output to 3d key point  facebookresearch/VideoPose3D#36), i.e. same as Videopose3d output.

Still, I end up with randomly located keypoints. Anything I might be missing??

Also, @AO2233 I'd be grateful if you can please share the email contents (that you shared with @Whisht ) here.

@TomCatOnline @safsin

  1. I guess that the pre-trained weight (gt81f.bin) given by the author can not reproduce the results of the wild video inference part in his paper. Others have had the same results which is not so good in the wild video inference. for example, this one. You may need to retrain the model by yourself?
  2. Here is 2 issues in the original code. One is that you need to convert the point, the other one is AttributeError: 'NoneType' object has no attribute 'astype' of Wild video.
  3. MY STEP 3.1 convert the point to human36M code. 3.2 remove the input_3d of evaluation in run_poseformer.py to solve the error AttributeError: 'NoneType' object has no attribute 'astype' of Wild video? I can't remember exactly. A part of the code was unfortunately missing.😢 Because after discussing, I changed to other methods. Maybe you can ask vicentowang for codes, his results look the same as mine.
  4. about your question You mentioned that your code end up with randomly located keypoints. So you may have solved 3.2? Then I think you should just check your conversion code(COCO->Human3.6M)? A wrong conversion may lead a random points. Maybe you can use 3.1 to convert the point, and a convert visualization is here to make sure that the conversion code works well.