search

THUDM / VisualGLM-6B

Chinese and English multimodal conversational language model | 多模态中英双语对话语言模型
Apache License 2.0
4.08k stars 415 forks source link

微调模型不能在使用web_demo.py运行 #127

Open trjoker opened 1 year ago

trjoker commented 1 year ago

微调脚本运行成功了，可以使用cli_demo.py的方式运行。 未进行修改前的web_demo.py可以正常运行，但是修改inter_util.py中的模型路径后，再运行则报错。 image

image user@lsp-ws:/ark-contexts/data/VisualGLM-6B/VisualGLM-6B$ python3 web_demo.py --share [2023-06-13 09:06:01,093] [INFO] [real_accelerator.py:110:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2023-06-13 09:06:02,170] [INFO] building VisualGLMModel model ... [2023-06-13 09:06:02,183] [INFO] [RANK 0] > initializing model parallel with size 1 [2023-06-13 09:06:02,184] [INFO] [RANK 0] You are using model-only mode. For torch.distributed users or loading model parallel models, set environment variables RANK, WORLD_SIZE and LOCAL_RANK. /usr/local/lib/python3.10/dist-packages/torch/nn/init.py:405: UserWarning: Initializing zero-element tensors is a no-op warnings.warn("Initializing zero-element tensors is a no-op") [2023-06-13 09:06:13,340] [INFO] [RANK 0] > number of parameters on model parallel rank 0: 7810582016 [2023-06-13 09:06:14,480] [INFO] [RANK 0] global rank 0 is loading checkpoint ./checkpoints/finetune-visualglm-6b-06-13-07-30/300/mp_rank_00_model_states.pt [2023-06-13 09:06:23,284] [INFO] [RANK 0] Will continue but found unexpected_keys! Check whether you are loading correct checkpoints: ['transformer.layers.0.attention.query_key_value.original.weight', 'transformer.layers.0.attention.query_key_value.original.bias', 'transformer.layers.0.attention.query_key_value.matrix_A.0', 'transformer.layers.0.attention.query_key_value.matrix_A.1', 'transformer.layers.0.attention.query_key_value.matrix_A.2', 'transformer.layers.0.attention.query_key_value.matrix_B.0', 'transformer.layers.0.attention.query_key_value.matrix_B.1', 'transformer.layers.0.attention.query_key_value.matrix_B.2', 'transformer.layers.0.attention.dense.original.weight', 'transformer.layers.0.attention.dense.original.bias', 'transformer.layers.0.attention.dense.matrix_A.0', 'transformer.layers.0.attention.dense.matrix_B.0', 'transformer.layers.14.attention.query_key_value.original.weight', 'transformer.layers.14.attention.query_key_value.original.bias', 'transformer.layers.14.attention.query_key_value.matrix_A.0', 'transformer.layers.14.attention.query_key_value.matrix_A.1', 'transformer.layers.14.attention.query_key_value.matrix_A.2', 'transformer.layers.14.attention.query_key_value.matrix_B.0', 'transformer.layers.14.attention.query_key_value.matrix_B.1', 'transformer.layers.14.attention.query_key_value.matrix_B.2', 'transformer.layers.14.attention.dense.original.weight', 'transformer.layers.14.attention.dense.original.bias', 'transformer.layers.14.attention.dense.matrix_A.0', 'transformer.layers.14.attention.dense.matrix_B.0']. Traceback (most recent call last): File "/ark-contexts/data/VisualGLM-6B/VisualGLM-6B/web_demo.py", line 129, in <module> main(args) File "/ark-contexts/data/VisualGLM-6B/VisualGLM-6B/web_demo.py", line 83, in main model, tokenizer = get_infer_setting(gpu_device=0, quant=args.quant) File "/ark-contexts/data/VisualGLM-6B/VisualGLM-6B/model/infer_util.py", line 23, in get_infer_setting model, args = VisualGLMModel.from_pretrained('./checkpoints/finetune-visualglm-6b-06-13-07-30/', args) File "/home/user/.local/lib/python3.10/site-packages/sat/model/base_model.py", line 216, in from_pretrained load_checkpoint(model, args, load_path=model_path, prefix=prefix) File "/home/user/.local/lib/python3.10/site-packages/sat/training/model_io.py", line 217, in load_checkpoint raise ValueError(f'Missing keys for inference: {missing_keys}.\nIf you still want to inference anyway, pass --force_inference to args.') ValueError: Missing keys for inference: ['transformer.layers.0.attention.query_key_value.weight', 'transformer.layers.0.attention.query_key_value.bias', 'transformer.layers.0.attention.dense.weight', 'transformer.layers.0.attention.dense.bias', 'transformer.layers.14.attention.query_key_value.weight', 'transformer.layers.14.attention.query_key_value.bias', 'transformer.layers.14.attention.dense.weight', 'transformer.layers.14.attention.dense.bias']. If you still want to inference anyway, pass --force_inference to args.

likeboo commented 1 year ago

同问，微调模型加载不了。

trjoker commented 1 year ago

看了一下代码，是模型加载的方式不同，可以参考下cli_demo中的模型加载方式（FineTuneVisualGLMModel被调用了，init方方法中做了一些逻辑），修改web_demo&api_demo。

Jananzzzz commented 1 year ago

同问，微调模型加载不了。

模型加载方式变了，将 model, args = VisualGLMModel.from_pretrained('visualglm-6b', args) 改为： model, model_args = AutoModel.from_pretrained('your_finetuned_model_path', args) 记得导入 Automodel 包: from sat.model import AutoModel

Sowhat007 commented 1 year ago

谢谢楼上的方法，我这里尝试用本地下载的VisualGLM模型也遇到小问题，会提示没有model_config.json，我看了huggingface上确实没有这个文件https://huggingface.co/THUDM/visualglm-6b/tree/main

JerryPW commented 1 year ago

我在微调后用cli_demo.py运行模型也出现了和楼主差不多的问题：

截屏2023-06-30 11 41 33

而且在无法进行正常的对话，会出现nan in logits, set nan_default_token to proceed in BaseStrategy.forward.的错误提示，请问有什么解决办法么？

yazheng0307 commented 1 year ago

看了一下代码，是模型加载的方式不同，可以参考下cli_demo中的模型加载方式（FineTuneVisualGLMModel被调用了，init方方法中做了一些逻辑），修改web_demo&api_demo。

可以具体说以下吗，改了还是一样的错误

xiulianzw commented 1 year ago

用这个就行了，
from finetune_visualglm import FineTuneVisualGLMModel model, args = FineTuneVisualGLMModel.from_pretrained('checkpoints/finetune-visualglm-6b-07-11-13-56', args=argparse.Namespace( fp16=True, skip_init=True, use_gpu_initialization=True, device='cuda', )) 但是测试发现，微调前后效果并不明显，如果增加迭代次数就导致其他地方效果影响明显