search

bmartacho / UniPose

We propose UniPose, a unified framework for human pose estimation, based on our “Waterfall” Atrous Spatial Pooling architecture, that achieves state-of-art-results on several pose estimation metrics. Current pose estimation methods utilizing standard CNN architectures heavily rely on statistical postprocessing or predefined anchor poses for joint localization. UniPose incorporates contextual seg- mentation and joint localization to estimate the human pose in a single stage, with high accuracy, without relying on statistical postprocessing methods. The Waterfall module in UniPose leverages the efficiency of progressive filter- ing in the cascade architecture, while maintaining multi- scale fields-of-view comparable to spatial pyramid config- urations. Additionally, our method is extended to UniPose- LSTM for multi-frame processing and achieves state-of-the- art results for temporal pose estimation in Video. Our re- sults on multiple datasets demonstrate that UniPose, with a ResNet backbone and Waterfall module, is a robust and efficient architecture for pose estimation obtaining state-of- the-art results in single person pose detection for both sin- gle images and videos.
Other
211 stars 44 forks source link

Inference error #14

Closed spoonerj closed 3 years ago

spoonerj commented 3 years ago

Dear Authors,

Thank you very much for your research. It is a very interesting research area! I have recently adapted your code to very simply test on my own images, however, I think I have made some errors as my predictions are all wrong.

I have used the test function in unipose.py and uncommented line 165 of utlis\utils.py in the draw_paint function so that it could find the source image.

I have used the MPII args and downloaded the MPII pretrained weights for the inference.

This is the source image I used, as you can see, it is a very simple human pose: test4

and this is the output I get from the human pose estimation:

0

Can you please advise if you think there are any steps I might have done wrong? I am happy to share my code if you want, but it is mainly adapted only to look for the correct files and that's it

I look forward to hearing from you

Many thanks

GuaiYiHu commented 3 years ago

Same issue as yours...

YuQi9797 commented 3 years ago

How you deal with the invisible points in the process of the experiment?

nicolasugrinovic commented 3 years ago

I have the same issue, output joints are not correct. This happens with MPII and LSP, both. I get worse results with the MPII weights. Here is a sample image from MPII dataset, the prediction is completely wrong and the heatmaps show nothing. image image

For LSP weights, the prediction is a little better but still wrong (the heatmaps look better than the previous). image

image

bmartacho commented 3 years ago

Thanks for bringing the issue with the weights to our attention.

We uploaded two updated training weights that are better for generalization since the previous weights were use for testing and might have issues for outside use.

The weights can be found at the following link: https://drive.google.com/drive/folders/1dPc7AayY2Pi3gjUURgozkuvlab5Vr-9n?usp=sharing

For the MPII dataset, the pre-trained weights are in the file UniPose_MPII.pth. The best model for generalization outside the dataset in real images is the model trained on the COCO dataset (UniPose_COCO.pth) since it uses a more diverse and larger dataset.

bmartacho commented 3 years ago

How you deal with the invisible points in the process of the experiment?

"Invisible" or occluded keypoints are still tentatively estimated by the network with the location where the method believes there is a higher likelihood of it occurring.

For the testing on the datasets used in this paper, occluded joints are detected but not counted for accuracy, as per pre-defined metrics by the dataset challenges.

In case you would like to not have the occluded joints in your visualization, one possible approach would be to set a minimum confidence threshold for the detected joints. I.e. only show joints with detection confidence >= 0.5.

EveningLin commented 1 year ago

I use the tar file, because the pth file will report errors, and the joint points are basically all wrong. What should I do to get better joint points based on MPI 1 I