search

Isotr0py / SakuraLLM-Notebooks

Notebooks to run SakuraLLM on colab/kaggle
49 stars 4 forks source link

v0.10pre模型无法使用 #6

Closed jyxjyx1234 closed 4 months ago

jyxjyx1234 commented 5 months ago

我准备在翻译中使用术语表，想要调用v0.10版的模型。将notebook中以下地方进行了修改 MODEL = "sakura-13b-qwen2beta-v0.10pre0-Q6_K"

hf_hub_download(repo_id="SakuraLLM/Sakura-14B-Qwen2beta-v0.10pre0-GGUF", filename=f"{MODEL}.gguf", local_dir="models/")

--model_version 0.10 \ 但是最后报错显示 2024-04-14 18:08:51 ec111550ff32 utils.consts[124] ERROR Invalid model version: v0.10pre0, please check the arguments Traceback (most recent call last): File "/kaggle/working/Sakura-13B-Galgame/server.py", line 109, in <module> state.get_model().check_model_by_magic() File "/kaggle/working/Sakura-13B-Galgame/utils/model.py", line 188, in check_model_by_magic (prompt, ground_truth, output) = self.test_loaded() File "/kaggle/working/Sakura-13B-Galgame/utils/model.py", line 311, in test_loaded generation_config = testcase.generation_config AttributeError: 'NoneType' object has no attribute 'generation_config' 请问我该如何进行修改？

Isotr0py commented 4 months ago

0.10pre版目前sakura那边还没有做python版api的适配，先等正式版出来吧。

jyxjyx1234 commented 4 months ago

好的，我在用galtransl的时候发现prompt的处理好像是在galtransl中就实现了，所以我把报错的py文件中跟版本相关的都跳过了，目前使用正常