search

zhihou7 / BatchFormer

CVPR2022, BatchFormer: Learning to Explore Sample Relationships for Robust Representation Learning, https://arxiv.org/abs/2203.01522
246 stars 20 forks source link

the code with czsl #19

Open Jianghao-Li opened 1 year ago

Jianghao-Li commented 1 year ago

Sorry to bother, i don't know how you insert batchformer to czsl tasks, could you please tell me more details? Thanks

zhihou7 commented 1 year ago

Hi @liccoco, I mainly change the code in two parts:

  1. batchformer: https://github.com/zhihou7/BatchFormer/blob/6f89e22d089e6efd5b0f84477cc56974dac8dc20/czsl/utils/config_model.py#L36, https://github.com/zhihou7/BatchFormer/blob/6f89e22d089e6efd5b0f84477cc56974dac8dc20/czsl/utils/config_model.py#L81
  2. repeat corresponding labels https://github.com/zhihou7/BatchFormer/blob/6f89e22d089e6efd5b0f84477cc56974dac8dc20/czsl/models/graph_method.py#L73

Besides, I choose a smaller lr (--lrb) for batchformer module because the network is quite small for cgqa.

Feel free to ask if you have any questions.

Regards, Zhi Hou

Jianghao-Li commented 1 year ago

Thanks for your reply. I found the code on github is different from the code in your paper, such as batchv1. The code in your paper image i didn't find the module appear in your real code, could you please explain the meaning of y in your paper? Thanks

zhihou7 commented 1 year ago

Hi @licoco, Thanks for your comment. That is actually the same as the description in the paper. It is just because the code of czsl utilizes a semantic graph to infer the zero-shot classes. y is the label, that is the same as the pairs in czsl. I do not implement an individual function for batchformer in czsl. But the current implementation is the same as the description, please refer to the two links that I provide in last comment. It is not necessary to change the code base of czsl to implement batchformer.

Regards,