search

deepseek-ai / DeepSeek-Coder

DeepSeek Coder: Let the Code Write Itself
https://coder.deepseek.com/
MIT License
6.01k stars 433 forks source link

eos_token_id for v1.5 model #118

Closed G07cha closed 4 months ago

G07cha commented 4 months ago

A rookie question, I was following README's guide on setting up deepseek-coder-instruct to complete the code and wanted to try it with deepseek-coder-7b-instruct-v1.5, but there eos_token_id is set to 100015 unlike 32021 for regular deepseek-coder-7b-instruct model. Can eos_token_id: 32014 still used for v1.5 model?

guoday commented 4 months ago

The eos_token_id of deepseek-coder-7b-instruct v1.5 is 100015, because the vocabulary is different from v1.

G07cha commented 4 months ago

@guoday do you know what value eos_token_id of v1.5 model should be set to when support for code completion is needed?

guoday commented 4 months ago

Are you referring to deepseek-coder-base-v1.5? The eos_token_id is 100001. However, when you use the coder for code completion, you shouldn't be able to stop generation using the eos_token_id. Typically, in code completion, you need to specify when to stop generation yourself, such as generating one line, generating a block, or generating 64 tokens.

G07cha commented 4 months ago

Ah clear, couldn't make the connection between eos_token_id from base model and instruct one, thanks!