search

ZijiaLewisLu / ICCV21-TASL

Official Repo for ICCV 2021 Paper "Weakly-Supervised Action Segmentation and Alignment via Transcript-Aware Union-of-Subspaces Learning".
7 stars 1 forks source link

when i run the code, why the performance of IOU and IOU-NObg metrics are always same? #4

Closed AcodeC closed 1 year ago

AcodeC commented 1 year ago

Action Alignment. Time 0.015138 m {'IoD': 0.4563928549401837, 'IoD_noBG': 0.4563928549401837, 'IoU': 0.2967080450303721, 'IoU_noBG': 0.2967080450303721, 'MoF': 0.5890393126789899} Action Segmentation Action Segmentation. Time 0.065104 m {'IoD': 0.26168015088350116, 'IoD_noBG': 0.26168015088350116, 'IoU': 0.20964444196304519, 'IoU_noBG': 0.20964444196304519, 'MoF': 0.515869446876997}

ZijiaLewisLu commented 1 year ago

You can refer to code at line below for the computation of metrics with and without background. https://github.com/ZijiaLewisLu/ICCV21-TASL/blob/6da3eeae59d90cc19b22d1cb4fc70e51c4d784cb/src/inference.py#L69

Basically, 0 is assumed to be the index of background class, which is true for the three datasets TASL is tested on. If your dataset does not contain action whose index is 0, the two metrics will be the same.

AcodeC commented 1 year ago

But my dataset does contain action whose index is 0, just follow your readme to conduct this experiment.

ZijiaLewisLu commented 1 year ago

Oh, I read the code again and there might be an error. The labels and predictions are lists of action names (str) instead of action indices (int). Can you try changing the line to a != "SIL" and evaluate again?

AcodeC commented 1 year ago

I try it.

AcodeC commented 1 year ago

another question is that when you inference, you use the Generate transcript group for Hierarchical Segmentation for your result in paper? not just as the github bash breakfast.sh default?

ZijiaLewisLu commented 1 year ago

In Table 1 of the paper, for Breakfast and Hollywood, I did not use the hierarchical segmentation. It was only used for CrossTask with C=20. This is explained in the "Implementation Details" section.