search

thuml / Transfer-Learning-Library

Transfer Learning Library for Domain Adaptation, Task Adaptation, and Domain Generalization
http://transfer.thuml.ai
MIT License
3.39k stars 553 forks source link

Question regarding DebiasMatch implementation #192

Closed quito418 closed 1 year ago

quito418 commented 1 year ago

Hi,

I am looking into the DebiasMatch code and found one thing confusing.

In the DebiasMatch code from this repo, it seems the same qhat is used for casual inference in weakly unlabeled data and adaptive marginal loss in strongly unlabeled data. https://github.com/thuml/Transfer-Learning-Library/blob/efe6e33caf5a6ea67f6e72135a2ac0ebf847742d/examples/semi_supervised_learning/image_classification/debiasmatch.py#L187

In the code from authors, it seems qhat before update is used for casual inference, and updated qhat is used for adaptive marginal loss.

Would it have difference in the result?

Thanks!

thucbx99 commented 1 year ago

We tried both of these update rules and found that the performance was similar when using pretrained models. And I think this is because the momentum coefficient is 0.999. Therefore, the impact of the current batch will not be great. But I don't know whether it behaves similarly under the standard SSL setting (no pretrained models).

quito418 commented 1 year ago

Thank you for your reply!