search

PoseAIChallenger / mxnet_pose_for_AI_challenger

Implementation of "Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields" for AI challenger keypoint competition
35 stars 9 forks source link

How about the performance？ #6

Closed Xiangyu-CAS closed 7 years ago

Xiangyu-CAS commented 7 years ago

Hi, I aslo particiate AI Challenger, using the original Caffe implementation of CMU-pose. However the result s are not desirable, the submission obtain a scroe of 41%. I am wondering what is the performance of your implementation. If you did not get a desirable results either, maybe this kind of method is not fit for this challenger. After all, the evaluation criterion differs from COCO and MPI, which heavily depends on the amount of persons.

dragonfly90 commented 7 years ago

Sorry I didn't submit it. I remember that @qqsh0214 said the validation is about 30% using the mxnet code, but I wonder if we have some bugs in augmentation. Correct me if I am wrong @qqsh0214 ?

qqsh0214 commented 7 years ago

@Xiangyu-CAS We didn't get a desirable results either and the validation is about 30%. We should have some bugs in augmentation. But I don't find where it is.

Xiangyu-CAS commented 7 years ago

Actually, image augmentations, such as scale , rotation and flip, are removed to simplify the training process in my implementation. I am curious about how much these affect final performance.

By the way, your implementation seems to be the only one which really works, others like pytorch and keras implementations are not completed yet. I really aprreciate your work : )

qqsh0214 commented 7 years ago

@Xiangyu-CAS Thank you! My result on validation is also trained without augmentations. Now that AI dataset is large enough, I guess augmentations may not improve much.

laoxihongshi commented 7 years ago

@Xiangyu-CAS Can you tell me what you have done to improve the CMU-pose? Have you changed any other way? thx!:)

Xiangyu-CAS commented 7 years ago

@luohuan2uestc 41% is the performance of baseline, without data augementation, we did not do anything to improve it.

laoxihongshi commented 7 years ago

@Xiangyu-CAS thx.So when used data augementation, you get 52%?

Xiangyu-CAS commented 7 years ago

@luohuan2uestc when data augmentation was used, we got 44% on test set.

laoxihongshi commented 7 years ago

@Xiangyu-CAS you are nice guy! thanks for your help! what's the number of epoch you have trained? so you get 52% mAP by other method?

Xiangyu-CAS commented 7 years ago

@luohuan2uestc Did I mentioned 52% mAP somewhere? 44% is the best performance on testset, on validation set the result is 51.6%.