Question about bilinear fusion and loss caculation in the code

[SCZwangxiao]

In the code, the bilinear fusion and loss function is different from that in the paper. I'm not questioning the reproducibility because I've successfully reproduced the results. I'm just wondering how much difference these modifications will make?

• Bilinear fusion is replaced by concatenation:

(CMIN/models_/gcn_final.py line 148-154)

# interactive
x1 = self.v2s(frames, x, node_mask)
frames1, x1 = self.cross_gate(frames, x1)
x = torch.cat([frames1, x1], -1)
# x = self.bilinear(frames1, x1, F.relu)
x = self.rnn(x, frames_len, self.max_num_frames)
x = F.dropout(x, self.dropout, self.training)

• In the loss function, only the prediction corresponding to the highest IoU score will be considered in loss function, while in the paper predictions whose IoU within a threshold will all be considered. (CMIN/models_/gcn_final.py line 167-171)

  if not self.training:
  indices = torch.argmax(predict_flatten, -1)
  else:
  indices = torch.argmax(label, -1)
  predict_box = proposals[indices]  # [nb, 2]

[Starboy-at-earth]

Dear SCZwangxiao:

Could you please tell how to process the splitted C3D features of Activity Caption dataset? It is weired five parts and I cannot figure out how to process such files with suffixs of .part-XX.

[SCZwangxiao]

Dear SCZwangxiao:

Could you please tell how to process the splitted C3D features of Activity Caption dataset? It is weired five parts and I cannot figure out how to process such files with suffixs of .part-XX.

Enter the dataset directory of *.part-XX files and type command:

cat activitynet_v1-3.part-* > temp.zip && unzip temp.zip

[Starboy-at-earth]

Thank you very much！！！