search

facebookresearch / SlowFast

PySlowFast: video understanding codebase from FAIR for reproducing state-of-the-art video models.
Apache License 2.0
6.52k stars 1.2k forks source link

Performance on minority classes #335

Open vasnakh opened 3 years ago

vasnakh commented 3 years ago

I ran the inference on a video where there was shooting but it seems that the trained models don't really have good performance to predict shoot which is one the classes and instead picks "carry/hold (an object)". Not sure whether I am doing something wrong or that is expected due to data imbalance etc. (also tried on train data but it gets classified as "carry/hold (an object)"). Would appreciate any comments/help. The models I tried were pre-trained model for AVA dataset and for the config I have:

DEMO: ENABLE: True FPS: 30 OUTPUT_FILE: ./demo_ms_32.mp4 LABEL_FILE_PATH: ./ava_classids.json # Add local label file path here. INPUT_VIDEO: demo/movie_shoot.mkv DETECTRON2_CFG: "COCO-Detection/faster_rcnn_R_50_FPN_3x.yaml" DETECTRON2_WEIGHTS: detectron2://COCO-Detection/faster_rcnn_R_50_FPN_3x/137849458/model_final_280758.pkl COMMON_CLASS_NAMES: []

lininglouis commented 3 years ago

the AP of shoot is 0.16 only. It doesnt do well in shooting. Also there is not much data in the shoot category in AVA dataset.

you can evaluate the model in the validation dataset, it will show the ap for each class, and map for all classes. It should match with the paper's mAP. someone post their training log at this.

irvingzhang0512 commented 3 years ago

AVA suffers from class imbalance. There aren't a lot shoot samples in ava. Training custom classes may help. But for now, PySlowFast doesn't support training ava custom classes. An alternative way to train custom classes is filter training samples.