search

deepseek-ai / DeepSeek-Coder

DeepSeek Coder: Let the Code Write Itself
https://coder.deepseek.com/
MIT License
6.01k stars 433 forks source link

Support for gpt fast. #72

Closed briandw closed 6 months ago

briandw commented 6 months ago

I'm trying to get DeepSeek to work with gpt-fast. https://github.com/pytorch-labs/gpt-fast/issues/35

I have it converted and running but the output is gibberish. Something is going wrong in conversion. Anyone interested in helping out?

briandw commented 6 months ago

Looks like the only difference is in the scaled rotary embeddings. If you replace the tokenizer and fix the sampling and use this PR https://github.com/pytorch-labs/gpt-fast/pull/44 and the correct model params. gpt fast works!