search

aixcoder-plugin / aiXcoder-7B

official repository of aiXcoder-7B Code Large Language Model
Apache License 2.0
2.15k stars 342 forks source link

Can you support running it through ollama (https://github.com/ollama/ollama)? #24

Open ICLXL opened 2 months ago

beeant0512 commented 1 month ago

convert aixcoder-7b-base to gguf format using ollama modelfile to create ollama model eg:

FROM ggml-model-f16.gguf

TEMPLATE """{{ .Prompt }}"""

PARAMETER temperature 0.7
PARAMETER top_p 0.8
PARAMETER top_k 20
PARAMETER stop "[INST]"
PARAMETER stop "[/INST]"
PARAMETER stop "<<SYS>>"
PARAMETER stop "<</SYS>>"
PARAMETER stop "<s>"
PARAMETER stop "</s>"

@ICLXL

zjh-nuc-AIOT commented 5 days ago

@beeant0512 我按照你这modefile部署了一下结果模型总是在胡言乱语

zjh-nuc-AIOT commented 5 days ago

我去hugg下载的.gguf,然后在ollama部署

zjh-nuc-AIOT commented 5 days ago

有合适的.gguf以及modefile分享吗