Full pipeline of AlphaPose

[dekathomas]

Does anyone here know about the full pipeline of AlphaPose? What architecture it use? Thanks in advance.

[dekathomas]

Did AlphaPose use yolo for it's human detector? If true, where is the place to load yolo model in training file? I have search to the all the files and I could not find the code to load yolo for training.

[Serdnad]

Yes, AlphaPose uses yolo. Where to download the weights from and where to save them in the project are both documented in the install instructions. That file should cover everything you need to get started training.

[dekathomas]

Hello thanks for replying @Serdnad. But sorry, maybe you got it wrong with my question, my question is about the training process, not the inference process. What I know the training process doesn't load yolo, but the inference process does load yolo.

Maybe you know about the training process? Until AlphaPose could output the keypoints?

[Serdnad]

I'm not intimately familiar with the project, but here are the logs for a training that I just began.

+ python ./scripts/train.py --exp-id simple-test --cfg configs/coco/resnet/256x192_res50_lr1e-3_1x-simple.yaml
******************************
Namespace(board=True, cfg='configs/coco/resnet/256x192_res50_lr1e-3_1x-simple.yaml', debug=False, detector='yolo', device=device(type='cuda', index=0), dist_backend='nccl', dist_url='tcp://192.168.1.214:23345', exp_id='simple-test', gpus=[0], launcher='none', map=True, nThreads=60, rank=-1, snapshot=2, sync=False, work_dir='./exp/simple-test-256x192_res50_lr1e-3_1x-simple.yaml/', world_size=4)
******************************
{'DATASET': {'TRAIN': {'TYPE': 'Mscoco', 'ROOT': './data/coco/', 'IMG_PREFIX': 'train2017', 'ANN': 'annotations/person_keypoints_train2017.json', 'AUG': {'FLIP': True, 'ROT_FACTOR': 40, 'SCALE_FACTOR': 0.3, 'NUM_JOINTS_HALF_BODY': 8, 'PROB_HALF_BODY': -1}}, 'VAL': {'TYPE': 'Mscoco', 'ROOT': './data/coco/', 'IMG_PREFIX': 'val2017', 'ANN': 'annotations/person_keypoints_val2017.json'}, 'TEST': {'TYPE': 'Mscoco_det', 'ROOT': './data/coco/', 'IMG_PREFIX': 'val2017', 'DET_FILE': './exp/json/test_det_yolo.json', 'ANN': 'annotations/person_keypoints_val2017.json'}}, 'DATA_PRESET': {'TYPE': 'simple', 'SIGMA': 2, 'NUM_JOINTS': 17, 'IMAGE_SIZE': [256, 192], 'HEATMAP_SIZE': [64, 48]}, 'MODEL': {'TYPE': 'SimplePose', 'PRETRAINED': '', 'TRY_LOAD': '', 'NUM_DECONV_FILTERS': [256, 256, 256], 'NUM_LAYERS': 50}, 'LOSS': {'TYPE': 'MSELoss'}, 'DETECTOR': {'NAME': 'yolo', 'CONFIG': 'detector/yolo/cfg/yolov3-spp.cfg', 'WEIGHTS': 'detector/yolo/data/yolov3-spp.weights', 'NMS_THRES': 0.6, 'CONFIDENCE': 0.1}, 'TRAIN': {'WORLD_SIZE': 4, 'BATCH_SIZE': 32, 'BEGIN_EPOCH': 0, 'END_EPOCH': 200, 'OPTIMIZER': 'adam', 'LR': 0.001, 'LR_FACTOR': 0.1, 'LR_STEP': [90, 120], 'DPG_MILESTONE': 140, 'DPG_STEP': [20, 50]}, 'FILE_NAME': '256x192_res50_lr1e-3_1x-simple.yaml'}
******************************
Create new model
=> init weights
loading annotations into memory...
Done (t=5.48s)
creating index...
index created!

... (omitted for brevity...)

Will create detection results to ./exp/json/test_det_yolo.json
loading annotations into memory...
Done (t=0.18s)
creating index...
index created!
Loading YOLO model..

Based on the last line, it seems to me like the training process does indeed load yolo as well. What makes you think otherwise?

[dekathomas]

Waw, thanks for the fast response, hmmm maybe it's the part of the training validation? In the validation it's indeed using yolo. When I try to run the train.py and remove the detector yolo in the config file, it doesn't produce the error when you comment the validation process.

Because of that I think AlphaPose doesn't use yolo for it's training? The config file I used was the same as you use 256x192_res50_lr1e-3_1x-simple.yaml

[dekathomas]

Maybe you can enlighten use? @Fang-Haoshu @Jeff-sjtu

[Fang-Haoshu]

Hi! Sorry for the late reply. AlphaPose use the human bounding boxes in the COCO annotation file, not the yolo detection for training. But we will add box augmentation to the ground-truth bounding box as data augmentation. The yolo detector used during training is to evaluate the AlphaPose's pose estimation result for in-the-wild images.