search

Dawn-LX / OpenVoc-VidVRD

Official code for the ICLR2023 paper Compositional Prompt Tuning with Motion Cues for Open-vocabulary Video Relation Detection
41 stars 1 forks source link

Different traj_info_dir during train & eval stage #4

Open QiueY514 opened 1 year ago

QiueY514 commented 1 year ago

Hi, I find that during object tracklet classification stage, traj_info_dir in config file (such as experiment/TrajCls_VidVRD/NoBgEmb/cfg_.py) is data0/VidVRD-II/tracklets_results/VidVRD_segment30_tracking_results_th-15-5. When comes to relation classification stage, traj_info_dir in config file (such as experiment/RelationCls_VidVRD/RePro/stage2/cfg_.py) is data0/VidVRD-II/tracklets_results/VidVRD_segment30_tracking_results. Why filtered trajs are only used for TrajCls module?

Dawn-LX commented 1 year ago

Hi, I find that during object tracklet classification stage, traj_info_dir in config file (such as experiment/TrajCls_VidVRD/NoBgEmb/cfg_.py) is data0/VidVRD-II/tracklets_results/VidVRD_segment30_tracking_results_th-15-5. When comes to relation classification stage, traj_info_dir in config file (such as experiment/RelationCls_VidVRD/RePro/stage2/cfg_.py) is data0/VidVRD-II/tracklets_results/VidVRD_segment30_tracking_results. Why filtered trajs are only used for TrajCls module?

Hi, sorry for this confusion. In fact we also use the filtered traj for training RelationCls Module. in class VidVRDUnifiedDataset(object):, we assign relation cls's label based on the filtered traj. refer to https://github.com/Dawn-LX/OpenVoc-VidVRD/blob/3cb19017f76c614f9c9a494efb19f075cf51a69e/dataloaders/dataset_vidvrd_v2.py#L767

also refer to https://github.com/Dawn-LX/OpenVoc-VidVRD/blob/3cb19017f76c614f9c9a494efb19f075cf51a69e/dataloaders/dataset_vidvrd_v2.py#L889

QiueY514 commented 1 year ago

Hi, I find that during object tracklet classification stage, traj_info_dir in config file (such as experiment/TrajCls_VidVRD/NoBgEmb/cfg_.py) is data0/VidVRD-II/tracklets_results/VidVRD_segment30_tracking_results_th-15-5. When comes to relation classification stage, traj_info_dir in config file (such as experiment/RelationCls_VidVRD/RePro/stage2/cfg_.py) is data0/VidVRD-II/tracklets_results/VidVRD_segment30_tracking_results. Why filtered trajs are only used for TrajCls module?

Hi, sorry for this confusion. In fact we also use the filtered traj for training RelationCls Module. in class VidVRDUnifiedDataset(object):, we assign relation cls's label based on the filtered traj. refer to

https://github.com/Dawn-LX/OpenVoc-VidVRD/blob/3cb19017f76c614f9c9a494efb19f075cf51a69e/dataloaders/dataset_vidvrd_v2.py#L767

also refer to

https://github.com/Dawn-LX/OpenVoc-VidVRD/blob/3cb19017f76c614f9c9a494efb19f075cf51a69e/dataloaders/dataset_vidvrd_v2.py#L889

Thank you for your explanation! So why not directly use the filtered traj_info as the same as in TrajCls Module (I mean set traj_info_dir in experiment/RelationCls_VidVRD/RePro/stage2/cfg_.py as data0/VidVRD-II/tracklets_results/VidVRD_segment30_tracking_results_th-15-5, and can save the time of filtering trajs during building datasets)? Or the filtered trajs are the same as the trajs in file VidVRD_segment30_tracking_results_th-15-5?

Dawn-LX commented 1 year ago

Hi, I find that during object tracklet classification stage, traj_info_dir in config file (such as experiment/TrajCls_VidVRD/NoBgEmb/cfg_.py) is data0/VidVRD-II/tracklets_results/VidVRD_segment30_tracking_results_th-15-5. When comes to relation classification stage, traj_info_dir in config file (such as experiment/RelationCls_VidVRD/RePro/stage2/cfg_.py) is data0/VidVRD-II/tracklets_results/VidVRD_segment30_tracking_results. Why filtered trajs are only used for TrajCls module?

Hi, sorry for this confusion. In fact we also use the filtered traj for training RelationCls Module. in class VidVRDUnifiedDataset(object):, we assign relation cls's label based on the filtered traj. refer to https://github.com/Dawn-LX/OpenVoc-VidVRD/blob/3cb19017f76c614f9c9a494efb19f075cf51a69e/dataloaders/dataset_vidvrd_v2.py#L767 also refer to https://github.com/Dawn-LX/OpenVoc-VidVRD/blob/3cb19017f76c614f9c9a494efb19f075cf51a69e/dataloaders/dataset_vidvrd_v2.py#L889

Thank you for your explanation! So why not directly use the filtered traj_info as the same as in TrajCls Module (I mean set traj_info_dir in experiment/RelationCls_VidVRD/RePro/stage2/cfg_.py as data0/VidVRD-II/tracklets_results/VidVRD_segment30_tracking_results_th-15-5, and can save the time of filtering trajs during building datasets)? Or the filtered trajs are the same as the trajs in file VidVRD_segment30_tracking_results_th-15-5?

  1. They are indeed the same.
  2. actually, we directly use the cached assigned labels when training RelationCls module.
QiueY514 commented 1 year ago

Hi, I find that during object tracklet classification stage, traj_info_dir in config file (such as experiment/TrajCls_VidVRD/NoBgEmb/cfg_.py) is data0/VidVRD-II/tracklets_results/VidVRD_segment30_tracking_results_th-15-5. When comes to relation classification stage, traj_info_dir in config file (such as experiment/RelationCls_VidVRD/RePro/stage2/cfg_.py) is data0/VidVRD-II/tracklets_results/VidVRD_segment30_tracking_results. Why filtered trajs are only used for TrajCls module?

Hi, sorry for this confusion. In fact we also use the filtered traj for training RelationCls Module. in class VidVRDUnifiedDataset(object):, we assign relation cls's label based on the filtered traj. refer to https://github.com/Dawn-LX/OpenVoc-VidVRD/blob/3cb19017f76c614f9c9a494efb19f075cf51a69e/dataloaders/dataset_vidvrd_v2.py#L767 also refer to https://github.com/Dawn-LX/OpenVoc-VidVRD/blob/3cb19017f76c614f9c9a494efb19f075cf51a69e/dataloaders/dataset_vidvrd_v2.py#L889

Thank you for your explanation! So why not directly use the filtered traj_info as the same as in TrajCls Module (I mean set traj_info_dir in experiment/RelationCls_VidVRD/RePro/stage2/cfg_.py as data0/VidVRD-II/tracklets_results/VidVRD_segment30_tracking_results_th-15-5, and can save the time of filtering trajs during building datasets)? Or the filtered trajs are the same as the trajs in file VidVRD_segment30_tracking_results_th-15-5?

  1. They are indeed the same.
  2. actually, we directly use the cached assigned labels when training RelationCls module.

Thanks for your patient help!

Mortyzhou-Shef-BIT commented 1 year ago

Thank you for your help, @Dawn-LX If I want to use the unseen video and generate the subject-relationship-object triplet. Can I use the data preparing script from VidSGG-TrajDataPrepare and then evaluate it by eval_script from OpenVoc-VidVRD.

Which variable in this eval script can provide information about this triplet? Is it triplet_5tuple or triplets_topk? Is triplet_scores the probability obtained? Thank u so much