search

lyogavin / airllm

AirLLM 70B inference with single 4GB GPU
Apache License 2.0
5.09k stars 408 forks source link

跑不通chatglm3，请大佬指教。 #130

Open ZiQiangXie opened 6 months ago

ZiQiangXie commented 6 months ago

按照文档介绍的，安装后跑chatglm3-6b-base，输出是一串问号，跑chat模型输出是一堆空白，打印的日志和命令行中间有几行空白。不知道什么情况。

ZiQiangXie commented 6 months ago

base模型的输出： [gMASK]sop What is the capital of United States?????????????????????

ZiQiangXie commented 6 months ago

from airllm import AutoModel

MAX_LENGTH = 128 model = AutoModel.from_pretrained("../ChatGLM3/ZhipuAI/chatglm3-6b-base")

input_text = [ 'What is the capital of United States?',

'I like',

    #'介绍一下你自己，如名字，年龄等'
]

input_tokens = model.tokenizer(input_text, return_tensors="pt", return_attention_mask=False, truncation=True, max_length=MAX_LENGTH, padding=True )

generation_output = model.generate( input_tokens['input_ids'].cuda(), max_new_tokens=20, use_cache=True, return_dict_in_generate=True)

output = model.tokenizer.decode(generation_output.sequences[0]) print(output)