Closed zphang closed 1 year ago
tl;dr: Add a second phase of pre-training to explicitly train models to be good at prompt-tuning and few-shot learning.
Full proposal: https://docs.google.com/document/d/1ER4tCfXO_Qn6GBWLi70sjrz-6kaMj8HCrTmFFnLf088/
Short version:
@zphang To what extent would you say that this was scooped by FLAN and T0?
tl;dr: Add a second phase of pre-training to explicitly train models to be good at prompt-tuning and few-shot learning.
Full proposal: https://docs.google.com/document/d/1ER4tCfXO_Qn6GBWLi70sjrz-6kaMj8HCrTmFFnLf088/
Short version: