Closed AlexeySorokin closed 3 years ago
Try <|endoftext|> token
@ollmer I have tried already for the small model, there is no such token in the vocabulary. If I take its index (50256), it corresponds to the word бросать.
for some models we don't provide special eos token.
Is it possible to generate text using an empty prompt or calculate the probability of the first token? There is no
bos_token_id
defined, therefore I do not see a natural way to do it. See https://colab.research.google.com/drive/1JvOSnKU4Mn1VtHoYXQ3rDXLRbOWwtRds#scrollTo=BjHAdxfyYw13 for an example.If I add
<s>
as the first token, the probability distribution for the first word is weird (see below), which means it does not solve the problem.The output is