3D Shot Posture Dataset

The Offical repo for the CVsports at CVPR paper "AutoSoccerPose: Automated 3D posture Analysis of Soccer Shot Movements". This dataset consists of the 3d and 2d postures of professional football players under shot situations.

Content of the dataset:

In 3dsp/train
- 20 cropped image x 200 shot
- Tracklet, 2d and 3d keypoints
In 3dsp/test
- 20 cropped image x 10 shots
- Tracklet

Clustering:

We provide 2 clustering methods, with pca+kmean or deep learning+kmeam. For the result of the clustering, refer to our paper.

AutoSoccerPose: Automated 3D Shooter Posture Extraction:

Extracting the shooter 2D&3D Posture from broadcast video (from SoccerNet).

Data source

Result

Data collection method:

The broadcast videos were collected from SoccerNet, and with annotation on actions, the videos were clipped (0.5 before and after the annotated ms, 25 frames total). The tracklet for the clips was then generated using a fine-tuned YOLO v8 and BoT-Sort. Furthermore, the shooter's traklet id was manually selected, and the cropped image was generated using the Bbox. The first 20 images' 2D postures (determined empirically) were manually annotated and lifted to 3D using the MotionAGFormer. This repository includes the modified code for BoT-Sort and MotionAGFormer.

The guide for downloading the dataset, the dataset structure explanation, and performing additional annotation and visualization are available below.

Getting Started

Download dataset

Step 1. Clone this repo

git clone https://github.com/calvinyeungck/3D-Shot-Posture-Dataset.git

Step 2. Locate the repo

cd path/to/this/repo/

Step 3. Unzip the file

unzip 3dsp.zip

The following are not required for retrieving the dataset.

Annotation and Visualization functions

Step 4. Install required package

pip install -r requirements.txt

Step 5. Download the required models parameters

Download the parameters for the following models and place them accordingly.

MotionAGFormer_2d at 3dsp_utils/MotionAGFormer/checkpoint/
MotionAGFormer_3d at 3dsp_utils/MotionAGFormer/demo/lib/checkpoint/
YOLOv8 at 3dsp_utils/bot_sort/yolov8_player/best.pt

Step 6.1. Automated 3D Shooter Posture Extraction, results in /3dsp_utils/output.

cd path/to/this/repo/
cd 3dsp_utils
python demo.py -t <path/to/target/clips>

Step 6.2. Perform additional annotation on SoccerNet

cd path/to/this/repo/
cd 3dsp_utils
python dataset_annotation.py

Step 7. Visualize the 2d and 3d posture

cd path/to/this/repo/
cd 3dsp_utils
python vis_post.py

Step 8. Pre-process the 3d posture

cd path/to/this/repo/
cd 3dsp_utils
python pre_process.py

Dataset Structure

The following shows the overall structure of the dataset and the format of each file.

xxxxx  --- img --- 001.jpg
            |   |- 002.jpg
            |   |- ...
            |   |- 020.jpg
            |   
            posture --- 001.json (H3WB format)
            |        |- 002.json
            |        |- ...
            |        |- 020.json
            |
            gt --- gt.txt (MOT20 format)
            |
            info --- info.ini

H3WB format

Removed the sample id and added the name of the joint in the original format
Use tlwh (xywh) instead of tlbr (xyxy) in the original format (following SoccerNet)
2d posture has the xy coordinate for the cropped image

3d posture has the xyz coordinate from the output of MotionAGFormer

xxx.json   --- 'image_path'
        |
        -- 'shooter_tracklet_id' --- 'id'
        |
        -- 'bbox' --- 't'
        |          |- 'l'
        |          |- 'w'
        |          |- 'h'
        |
        -- 'keypoint_2d' --- joint id --- 'name'
        |                             |- 'x'
        |                             |- 'y'
        |
        -- 'keypoint_3d' --- joint id --- 'name'
                                        |- 'x'
                                        |- 'y'
                                        |- 'z'

MOT20 format

frame ID, track ID, top left coordinate of the bounding box, top y coordinate, width, height, confidence score for the detection (always 1. for the ground truth) and the remaining values are set to -1 as they are not used

info.ini

info from SoccerNet

[info]
id = folder id 
previous_id = id previously given to the clips
shooter_tracklet_id = the tracklet if of the shooter
gameTime = time of the game
label = shot on target or shot off target
annotated_position = the ms where the action is annotated
start_position = the ms where the first image is captured
end_position = the ms where the last image is captured
position_step = the ms between each image
team = home or away
visibility = visibility of the player
game = info of the game
half = first or second half
time = time of the video in seconds

Reference

Please consider citing our work if you find it helpful to yours:

@inproceedings{yeung2024autosoccerpose,
  title={AutoSoccerPose: Automated 3D posture Analysis of Soccer Shot Movements},
  author={Yeung, Calvin and Ide, Kenjiro and Fujii, Keisuke},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={3214--3224},
  year={2024}
}

calvinyeungck / 3D-Shot-Posture-Dataset

readme