search

google-research / uda

Unsupervised Data Augmentation (UDA)
https://arxiv.org/abs/1904.12848
Apache License 2.0
2.17k stars 313 forks source link

tuning uda_confidence_thresh #100

Closed stellaywu closed 2 years ago

stellaywu commented 3 years ago

Thanks for the implementation! I am trying to implement the code on a new dataset and see unsupervised loss is most of the time zero. Should I just lower the uda_confidence_thresh? What's the best strategy?

Thanks!

spolavar commented 2 years ago

Hey Stella, Did you have significant improvement on your data set with UDA training over fully supervised counter part? Did you run on GPUs and how did you deal with memory issues? After re-implementing the UDA training, due to memory limitations I realize only a small number of unlabeled examples are seen by the training loop. The performance also is not greater than its fully-supervised counterpart. Just wanted to know your experience on this! I appreciate any thoughts that you may share.

Thank you, Sridevi

stellaywu commented 2 years ago

Hi @spolavar , I was able to train the UDA on my data with GPU and with distributed CPU. I re-implemented UDA in Pytorch and used a benchmark data. I didn't run into memory issue as data is loaded in batches with dataloader. It was able to train DBPedia data with 10 labeled examples per class and achieved 98% accuracy, similar to the paper. But DBPedia is a simple dataset, I'm not sure how the result will be for other datasets. Hope that helps.