Closed AcodeC closed 1 year ago
You can refer to code at line below for the computation of metrics with and without background. https://github.com/ZijiaLewisLu/ICCV21-TASL/blob/6da3eeae59d90cc19b22d1cb4fc70e51c4d784cb/src/inference.py#L69
Basically, 0 is assumed to be the index of background class, which is true for the three datasets TASL is tested on. If your dataset does not contain action whose index is 0, the two metrics will be the same.
But my dataset does contain action whose index is 0, just follow your readme to conduct this experiment.
Oh, I read the code again and there might be an error. The labels and predictions are lists of action names (str) instead of action indices (int). Can you try changing the line to a != "SIL"
and evaluate again?
I try it.
another question is that when you inference, you use the Generate transcript group for Hierarchical Segmentation for your result in paper? not just as the github bash breakfast.sh default?
In Table 1 of the paper, for Breakfast and Hollywood, I did not use the hierarchical segmentation. It was only used for CrossTask with C=20. This is explained in the "Implementation Details" section.
Action Alignment. Time 0.015138 m {'IoD': 0.4563928549401837, 'IoD_noBG': 0.4563928549401837, 'IoU': 0.2967080450303721, 'IoU_noBG': 0.2967080450303721, 'MoF': 0.5890393126789899} Action Segmentation Action Segmentation. Time 0.065104 m {'IoD': 0.26168015088350116, 'IoD_noBG': 0.26168015088350116, 'IoU': 0.20964444196304519, 'IoU_noBG': 0.20964444196304519, 'MoF': 0.515869446876997}