search

ztxz16 / fastllm

纯c++的全平台llm加速库，支持python调用，chatglm-6B级模型单卡可达10000+token / s，支持glm, llama, moss基座，手机端流畅运行
Apache License 2.0
3.28k stars 333 forks source link

添加代码后运行报错，这是为什么 #304

Open renllll opened 1 year ago

renllll commented 1 year ago

Traceback (most recent call last):
File "web_demo3.py", line 9, in <module>
model = llm.from_hf(model, tokenizer, dtype = "float16") # dtype支持 "float16", "int8", "int4"
File "/home/remotesense/anaconda3/envs/glm/lib/python3.8/site-packages/fastllm_pytools-0.0.1-py3.8.egg/fastllm_pytools/llm.py", line 94, in from_hf
return hf_model.create(model, tokenizer, dtype = dtype); File "/home/remotesense/anaconda3/envs/glm/lib/python3.8/site-packages/fastllm_pytools-0.0.1-py3.8.egg/fastllm_pytools/hf_model.py", line 84, in create if len(active_adapter) != 0: TypeError: object of type 'method' has no len()

renllll commented 1 year ago

我的model.active_adapter是一个方法，而84行附近却是调用数值，这该如何解决

renllll commented 1 year ago

active_adapter = "" if hasattr(model, "active_adapter"): try: active_adapter = model.active_adapter() except ValueError: print("No adapter loaded.") 我将源代码修改成以上后，报以下错误 No adapter loaded. convert ( 200 / 200 ) Warmup... status = 7 1 16 128 Error: cublas error. terminate called after throwing an instance of 'char const*' Aborted (core dumped)

ImmNaruto commented 1 year ago

一样的情况，相同代码，centos下无问题，在WSL2中的ubuntu编译后出现该问题

renllll commented 1 year ago

一样的情况，相同代码，centos下无问题，在WSL2中的ubuntu编译后出现该问题

这是编译时的问题，还是代码本身问题，后面报错是显存爆了，我在源代码中更改了下

ChanChiChoi commented 1 year ago

尝试tramsformer版本改低点试试呢，我们这边4.33.0也遇到这问题，改成4.31.0就好了

renllll commented 1 year ago

尝试tramsformer版本改低点试试呢，我们这边4.33.0也遇到这问题，改成4.31.0就好了

No adapter loaded. convert ( 200 / 200 ) Warmup... status = 7 1 16 128 Error: cublas error. terminate called after throwing an instance of 'char const*' Aborted (core dumped) 但是这个报错没法解决

ARES3366 commented 1 year ago

User：你好 Error: cublas error. Aborted (core dumped)

直接报错，这是因为啥呢，cuda117环境

failable commented 12 months ago

同样错误

Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:12<00:00,  1.82s/it]
Traceback (most recent call last):
  File "/Users/Maximin/Downloads/fastllm/build/tools/test.py", line 14, in <module>
    model = llm.from_hf(
  File "/Users/Maximin/Downloads/fastllm/build/tools/fastllm_pytools/llm.py", line 161, in from_hf
    return hf_model.create(model, tokenizer, dtype=dtype)
  File "/Users/Maximin/Downloads/fastllm/build/tools/fastllm_pytools/hf_model.py", line 84, in create
    if len(active_adapter) != 0:
TypeError: object of type 'method' has no len()
Illegal instruction: 4
128Ghe980 commented 11 months ago

同样报错，transformer==4.33.2