Closed yongzx closed 2 years ago
I have checked @vnikouliNLE's tokenization strategy and confirmed that the indices of added vocab tokens are after those of the original vocab.
I used register hook to update subelements of the embedding weights.
Resolved by #20
I have checked @vnikouliNLE's tokenization strategy and confirmed that the indices of added vocab tokens are after those of the original vocab.
I used register hook to update subelements of the embedding weights.