You can check our project page at here.
(To access smoothly, you need to hold down the Ctrl key and click with the mouse.)
We used 3DCrowdNet as a baseline, and whenever an issue occurred, the author of 3DCrowdnet responded kindly. I am always grateful to him
π SEFD achieves the SOTA on 3DPW
π We improved MPJPE to 77.37 PA-MPJPE to 49.39mm and MPVPE to 92.60mm using a ResNet 50 backbone!
(Our model has no difference between baseline 3DCrowdNet model parameter and MACs.)
This repo is the official PyTorch implementation of [SEFD : Learning to Learning to Distill Complex Pose and Occlusion].
We recommend you to use an Anaconda virtual environment. Install PyTorch >=1.6.0 and Python >= 3.7.3.
To implement RCF edge detection, mmcv here must be installed.
You need to install kornia here to implement canny edge detection.
(This makes it possible to implement on GPU and canny edge is implemented many times faster than Opencv.)
All this toolkit is on requirements.sh
here.
Then, run sh requirements.sh
. You should slightly change torchgeometry
kernel code following here.
${ROOT}/demo/
. ${ROOT}/demo/input
(just unzip the demo_input.zip).${ROOT}/demo/output
directory.J_regressor_extra.npy
from here and place under ${ROOT}/data/
. python demo.py --gpu 0 --model_path SEFD.pth.tar
or python demo.py --gpu 0
. You can change the input image with --img_idx {img number}
. ${ROOT}/demo/
.python demo.py --gpu 0 --model_path baseline.pth.tar
. You can change the input image with --img_idx {img number}
. ${ROOT}/demo/
. if you run the baseline, change ${ROOT}/main/config.py
distillation_module=True => False
distillation_loss="KD_loss" => ""
dilstillation_edge="Canny" => ""
distillation_pretrained=True => False
SMPL_overlap=True => False
SMPL_edge=True => False
first_input=True => False
nothing=False => True
The demo images and 2D poses are from CrowdPose and HigherHRNet respectively.
π Refer to the supplementary material for diverse qualitative results
Refer to here.
Frist finish the directory setting. then, refer to here to train and test SEFD.
refer to here to train and test SEFD with various edges.
refer to here to train and test without feature distillation.
${ROOT}/main/config.py
.refer to here to train and test SEFD with various losses.
refer to here to train and test SEFD with various feature connections.