I have questions about details of training.The batch size of 3B model reported in the paper is 2 million tokens. When finetuning, I should concat different protein sequences into one seq like this: 'bos' protein1token1id protein1token2id ...... 'eos' 'bos' protein2token1id ... 'eos'. The total length is 2M. Am I right?
Hi, congratulations on these fantastic models!
I have questions about details of training.The batch size of 3B model reported in the paper is 2 million tokens. When finetuning, I should concat different protein sequences into one seq like this: 'bos' protein1token1id protein1token2id ...... 'eos' 'bos' protein2token1id ... 'eos'. The total length is 2M. Am I right?
Looking forward to your reply! thanks in advance