Open mprudra opened 6 months ago
Hi, thanks for reporting the issue. Would you please upgrade to 0.9.0 to see if the problem still persist?
Would require significant efforts, will keep this as last resort. Do you have any idea about what could be the cause of this error? Is this issue known to some previous versions?
Happens for me too, on 0.9.1, when running with delwiv/codefuse-deepseek-33B
model. Doesn't happen with TabbyML/DeepseekCoder-6.7B
model.
Happens for me too, on 0.9.1, when running with
delwiv/codefuse-deepseek-33B
model. Doesn't happen withTabbyML/DeepseekCoder-6.7B
model.
Could you also share the log output and your system info?
Seems related: https://github.com/ggerganov/llama.cpp/issues/3959 https://github.com/ggerganov/llama.cpp/issues/4206
@mprudra could you share the model you were using when encountering the issue?
Happens for me too, on 0.9.1, when running with delwiv/codefuse-deepseek-33B model. Doesn't happen with TabbyML/DeepseekCoder-6.7B model.
...
Seems related: ggerganov/llama.cpp#3959 ggerganov/llama.cpp#4206
@mprudra could you share the model you were using when encountering the issue?
I'm also using our fined-tuned version of DeepSeekCoder-33b.
Correction: I had noticed it with 6.7B model.
Is it the case that Deepseek-Coder models aren't yet supported? (Deepseek coder merge, ggerganov/llama.cpp#5464)[https://github.com/TabbyML/tabby/issues/1666]
https://github.com/ggerganov/llama.cpp/issues/5981 is the latest issue opened to support deepseek in llama.cpp
Describe the bug I'm noticing below error with our Tabby deployment, looks like a memory error. Don't have any additional logs, since we've modified the logs to mask input, output information, this was needed for production deployment. Process exit code was 1.
Information about your version 0.5.5
Information about your GPU