search

yangsenius / TransPose

PyTorch Implementation for "TransPose: Keypoint localization via Transformer", ICCV 2021.
https://github.com/yangsenius/TransPose/releases/download/paper/transpose.pdf
MIT License
353 stars 56 forks source link

Transfer to other Datasets #15

Closed EckoTan0804 closed 3 years ago

EckoTan0804 commented 3 years ago

Hi,

Can TransPose be transferred to other datasets, e.g. MPII and PoseTrack? If yes, how should I perform the fine-tuning?

Many thanks in advance!

yangsenius commented 3 years ago

Yes, please see this version for reference.

EckoTan0804 commented 3 years ago

Thanks for the response! I have read this update version and seen that pretrained TransPose models can be successfully transferred to MPII dataset.

I am trying to transfer TransPose to PoseTrack dataset and I have a few questions about the details:

  1. Should I prepare PoseTrack dataset and annotations just like preparing COCO described in README? Should human detection results of PoseTrack be prepared in advance?

  2. In PoseTrack, there're only 15 keypoints annotation (left_ear and right_ear are excluded compared to COCO). To transfer TransPose to PoseTrack, I try to replace the final layer of the pre-trained TransPose model with a uniform-initialized dx15 linear layer and freeze other parts of the model. Is this the correct way?

  3. How should I correctly conduct the fine-tuning? (I plan to train only the re-initialized last layer for a few epochs, then fine-tune the entire model for a few epochs. )

yangsenius commented 3 years ago
  1. I suggest following the dataset preparation of mmpose for Posetrack.
  2. Yes. You also can use a smaller learning rate for the pre-trained layers.
  3. Your plan seems to be effective. And setting different learning rates for pre-trained layers and the final layer might work well.
EckoTan0804 commented 3 years ago

Thanks for your suggestion!