Open hx1024 opened 5 years ago
ft_tsa_only
: only finetune TSA module
We first train a model w/o TSA. Then we add the TSA module and only finetune TSA for some iterations. After that, we finetune the whole model.
I wanted to make sure I understood is correctly. Is this how the complete training goes:
If that is the case; can you please answer the following questions:
Hi, by looking at the provided log files; it feels you are first training for 600K iterations woTSA and then using this pre-trained model, further training for another 600K iterations with TSA module in place. Is this correct?
Why fine-tuning based on TSA required equal amount of training?
In the training .yml file,what does the "ft_tsa_only" mean? when I doesn't use it, prompted WARNING: Offset mean is XX, larger than 100. and the loss is larger than use the "ft_tsa_only" to train