search

azshue / TPT

Test-time Prompt Tuning (TPT) for zero-shot generalization in vision-language models (NeurIPS 2022))
https://azshue.github.io/TPT/
MIT License
136 stars 16 forks source link

What is the meaning of "Test-Time training"? #8

Closed jingzhengli closed 1 year ago

jingzhengli commented 1 year ago

Hi, recently I have trained a model by using very small amount of unlabeled test data (16-shot). Could I call this training manner "Test-Time training"? Thanks

jingzhengli commented 1 year ago

I have another question. During test-test training, why train the model by augmenting only one sample to multiple samples (one origin test sample in a batch), is it possible to augment multiple test samples at once (many origin test sample in a batch)?

azshue commented 1 year ago

Hi, recently I have trained a model by using very small amount of unlabeled test data (16-shot). Could I call this training manner "Test-Time training"? Thanks

Hi, I think it depends on how you update your model at test time, and whether your model has been trained on some training data before test time. Please refer to Sun et al. for more details about their definition of "test-time training".

azshue commented 1 year ago

I have another question. During test-test training, why train the model by augmenting only one sample to multiple samples (one origin test sample in a batch), is it possible to augment multiple test samples at once (many origin test sample in a batch)?

Yes, you can adapt our method to do batch test-time prompt tuning. However, in our paper, we consider a more practical and challenging setting, where at deployment (i.e., test-time), a model may not always have access to a batch of test samples, so we manage to do test-time adaptation on a single test sample.

jingzhengli commented 1 year ago

Hi, recently I have trained a model by using very small amount of unlabeled test data (16-shot). Could I call this training manner "Test-Time training"? Thanks

Hi, I think it depends on how you update your model at test time, and whether your model has been trained on some training data before test time. Please refer to Sun et al. for more details about their definition of "test-time training".

Thanks for you insightful response. My confusion is resolved.