labels encoding multihot.

hi, the labels (over thumos14 at least) in labels_multihot here do not seem to be multihot. is it normal? they are class frequencies formatted in something that looks like hot encoding. can you please check?

i run separately the piece of code that create labels_multihot over thumos14 files:

In [81]: labels_multihot[:10]
Out[81]: 
[array([0., 0., 3., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0.]),
 array([0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0.]),
 array([ 0.,  0., 15.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,
         0.,  0.,  0.,  0.,  0.,  0.,  0.]),
 array([0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0.]),
 array([0., 0., 9., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0.]),
 array([ 0.,  0., 11.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,
         0.,  0.,  0.,  0.,  0.,  0.,  0.]),
 array([ 0.,  0., 13.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,
         0.,  0.,  0.,  0.,  0.,  0.,  0.]),
 array([0., 0., 4., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0.]),
 array([ 0.,  0., 15.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,
         0.,  0.,  0.,  0.,  0.,  0.,  0.]),
 array([0., 0., 9., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0.])]

each number is the frequency of the corresponding class.

it seems they stay in this form unchanged even when being used in the loss. https://github.com/XiaojunTang22/ICCV2023-DDGNet/blob/0fe0b1421b432c6931c59c81683d6597d95beced/wsad_dataset.py#L130 https://github.com/XiaojunTang22/ICCV2023-DDGNet/blob/0fe0b1421b432c6931c59c81683d6597d95beced/train.py#L15 https://github.com/XiaojunTang22/ICCV2023-DDGNet/blob/0fe0b1421b432c6931c59c81683d6597d95beced/train.py#L20 https://github.com/XiaojunTang22/ICCV2023-DDGNet/blob/0fe0b1421b432c6931c59c81683d6597d95beced/train.py#L23

inside the loss, they seem to be used directly without change, except adding an extra class for the background as here: https://github.com/XiaojunTang22/ICCV2023-DDGNet/blob/0fe0b1421b432c6931c59c81683d6597d95beced/model.py#L785 as if they are binary.

can you please double-check at your end? thanks

XiaojunTang22 / ICCV2023-DDGNet

labels encoding multihot. #3