Open Cola-Ice opened 1 year ago
1.安装依赖后,执行命令
streamlit run web_feadback.py --server.port=8080
,报错:/root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: /root/anaconda3/envs/lora did not contain libcudart.so as expected! Searching further paths... warn(msg) /root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('history -a; history -a; printf "\\033]0;%s@%s'), PosixPath('%s\\007" "${USER}" "${HOSTNAME%%.*}" "${PWD/#$HOME/\\~}"')} warn(msg) CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64... CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so CUDA SETUP: Highest compute capability among GPUs detected: 6.1 CUDA SETUP: Detected CUDA version 114 /root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU! warn(msg) CUDA SETUP: Loading binary /root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/libbitsandbytes_cuda114_nocublaslt.so... 2023-04-26 11:11:07.907 Uncaught app exception Traceback (most recent call last): File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script exec(code, module.__dict__) File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 15, in <module> from modeling_chatglm import ChatGLMForConditionalGeneration ModuleNotFoundError: No module named 'modeling_chatglm'
2.将项目根目录的
modeling_chatglm.py
、configuration_chatglm.py
文件拷贝到 web_ui 目录下,再次执行streamlit run web_feadback.py --server.port=8080
,再次报错:/root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: /root/anaconda3/envs/lora did not contain libcudart.so as expected! Searching further paths... warn(msg) /root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('%s\\007" "${USER}" "${HOSTNAME%%.*}" "${PWD/#$HOME/\\~}"'), PosixPath('history -a; history -a; printf "\\033]0;%s@%s')} warn(msg) CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64... CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so CUDA SETUP: Highest compute capability among GPUs detected: 6.1 CUDA SETUP: Detected CUDA version 114 /root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU! warn(msg) CUDA SETUP: Loading binary /root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/libbitsandbytes_cuda114_nocublaslt.so... 标注数据集已创建。 The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. Loading checkpoint shards: 100%|█████████████████████████████████████████████| 8/8 [00:06<00:00, 1.16it/s] Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision. 2023-04-26 11:15:21.961 Uncaught app exception Traceback (most recent call last): File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script exec(code, module.__dict__) File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 289, in <module> main() File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 282, in main start_evaluate_page() File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 117, in start_evaluate_page out = model.generate( File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/peft/peft_model.py", line 729, in generate outputs = self.base_model.generate(**kwargs) File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/transformers/generation/utils.py", line 1406, in generate return self.greedy_search( File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/transformers/generation/utils.py", line 2198, in greedy_search model_inputs = self.prepare_inputs_for_generation(input_ids, **model_kwargs) File "/root/Chatglm_lora_multi-gpu/webui/modeling_chatglm.py", line 988, in prepare_inputs_for_generation mask_position = seq.index(mask_token) ValueError: 150001 is not in list
3.将步骤2中添加的
modeling_chatglm.py
、configuration_chatglm.py
文件删除,替换为THUDM/ChatGLM-6B
项目chatglm-6b
模型中提供的文件,仍旧报错:/root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: /root/anaconda3/envs/lora did not contain libcudart.so as expected! Searching further paths... warn(msg) /root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('%s\\007" "${USER}" "${HOSTNAME%%.*}" "${PWD/#$HOME/\\~}"'), PosixPath('history -a; history -a; printf "\\033]0;%s@%s')} warn(msg) CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64... CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so CUDA SETUP: Highest compute capability among GPUs detected: 6.1 CUDA SETUP: Detected CUDA version 114 /root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU! warn(msg) CUDA SETUP: Loading binary /root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/libbitsandbytes_cuda114_nocublaslt.so... 标注数据集已创建。 The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. Loading checkpoint shards: 100%|█████████████████████████████████████████████| 8/8 [00:06<00:00, 1.15it/s] Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision. 2023-04-26 11:32:12.915 Uncaught app exception Traceback (most recent call last): File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script exec(code, module.__dict__) File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 289, in <module> main() File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 282, in main start_evaluate_page() File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 117, in start_evaluate_page out = model.generate( File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/peft/peft_model.py", line 729, in generate outputs = self.base_model.generate(**kwargs) File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/transformers/generation/utils.py", line 1406, in generate return self.greedy_search( File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/transformers/generation/utils.py", line 2245, in greedy_search model_kwargs = self._update_model_kwargs_for_generation( File "/root/Chatglm_lora_multi-gpu/webui/modeling_chatglm.py", line 1085, in _update_model_kwargs_for_generation attention_mask = torch.cat( IndexError: Dimension out of range (expected to be in range of [-2, 1], but got 3)
把ChatGLMForConditionalGeneration 换成AutoModel 把token也换成AutoToken 清华把他们上传HF的模型全部换过了
1.将web_feadback.py
中 ChatGLMForConditionalGeneration 修改为 AutoModel
if 'model' not in st.session_state:
with st.spinner('Loading Model...'):
#config = PeftConfig.from_pretrained(peft_model_path)
peft_path = "chatglm-lora-new.pt"
peft_config = LoraConfig(
task_type=TaskType.CAUSAL_LM, inference_mode=True,
r=8,
lora_alpha=32,
lora_dropout=0.1)
model = AutoModel.from_pretrained(
"/models/chatglm-6b",
cache_dir ='./',
trust_remote_code=True
).float().to(device)
model = get_peft_model(model, peft_config)
model.load_state_dict(torch.load(peft_path), strict=False)
'''model = PeftModel.from_pretrained(
model,
peft_model_path
)'''
tokenizer = AutoTokenizer.from_pretrained(
"/models/chatglm-6b",
cache_dir ='./',
trust_remote_code=True
)
st.session_state['model'] = model
st.session_state['tokenizer'] = tokenizer
2.再次执行,出现以下错误,是还有其他地方要改吗? @liangwq
/root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: /root/anaconda3/envs/lora did not contain libcudart.so as expected! Searching further paths...
warn(msg)
/root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('history -a; history -a; printf "\\033]0;%s@%s'), PosixPath('%s\\007" "${USER}" "${HOSTNAME%%.*}" "${PWD/#$HOME/\\~}"')}
warn(msg)
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64...
CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so
CUDA SETUP: Highest compute capability among GPUs detected: 6.1
CUDA SETUP: Detected CUDA version 114
/root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU!
warn(msg)
CUDA SETUP: Loading binary /root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/libbitsandbytes_cuda114_nocublaslt.so...
标注数据集已创建。
Explicitly passing a `revision` is encouraged when loading a configuration with custom code to ensure no malicious code has been contributed in a newer revision.
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████| 8/8 [01:29<00:00, 11.24s/it]
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
2023-04-26 14:26:02.449 Uncaught app exception
Traceback (most recent call last):
File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script
exec(code, module.__dict__)
File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 288, in <module>
main()
File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 281, in main
start_evaluate_page()
File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 116, in start_evaluate_page
out = model.generate(
File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/peft/peft_model.py", line 729, in generate
outputs = self.base_model.generate(**kwargs)
File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/transformers/generation/utils.py", line 1406, in generate
return self.greedy_search(
File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/transformers/generation/utils.py", line 2245, in greedy_search
model_kwargs = self._update_model_kwargs_for_generation(
File "/root/.cache/huggingface/modules/transformers_modules/chatglm-6b/modeling_chatglm.py", line 1085, in _update_model_kwargs_for_generation
attention_mask = torch.cat(
IndexError: Dimension out of range (expected to be in range of [-2, 1], but got 3)
1.将
web_feadback.py
中 ChatGLMForConditionalGeneration 修改为 AutoModelif 'model' not in st.session_state: with st.spinner('Loading Model...'): #config = PeftConfig.from_pretrained(peft_model_path) peft_path = "chatglm-lora-new.pt" peft_config = LoraConfig( task_type=TaskType.CAUSAL_LM, inference_mode=True, r=8, lora_alpha=32, lora_dropout=0.1) model = AutoModel.from_pretrained( "/models/chatglm-6b", cache_dir ='./', trust_remote_code=True ).float().to(device) model = get_peft_model(model, peft_config) model.load_state_dict(torch.load(peft_path), strict=False) '''model = PeftModel.from_pretrained( model, peft_model_path )''' tokenizer = AutoTokenizer.from_pretrained( "/models/chatglm-6b", cache_dir ='./', trust_remote_code=True ) st.session_state['model'] = model st.session_state['tokenizer'] = tokenizer
2.再次执行,出现以下错误,是还有其他地方要改吗? @liangwq
/root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: /root/anaconda3/envs/lora did not contain libcudart.so as expected! Searching further paths... warn(msg) /root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('history -a; history -a; printf "\\033]0;%s@%s'), PosixPath('%s\\007" "${USER}" "${HOSTNAME%%.*}" "${PWD/#$HOME/\\~}"')} warn(msg) CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64... CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so CUDA SETUP: Highest compute capability among GPUs detected: 6.1 CUDA SETUP: Detected CUDA version 114 /root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU! warn(msg) CUDA SETUP: Loading binary /root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/libbitsandbytes_cuda114_nocublaslt.so... 标注数据集已创建。 Explicitly passing a `revision` is encouraged when loading a configuration with custom code to ensure no malicious code has been contributed in a newer revision. Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision. Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████| 8/8 [01:29<00:00, 11.24s/it] Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision. 2023-04-26 14:26:02.449 Uncaught app exception Traceback (most recent call last): File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script exec(code, module.__dict__) File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 288, in <module> main() File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 281, in main start_evaluate_page() File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 116, in start_evaluate_page out = model.generate( File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/peft/peft_model.py", line 729, in generate outputs = self.base_model.generate(**kwargs) File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/transformers/generation/utils.py", line 1406, in generate return self.greedy_search( File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/transformers/generation/utils.py", line 2245, in greedy_search model_kwargs = self._update_model_kwargs_for_generation( File "/root/.cache/huggingface/modules/transformers_modules/chatglm-6b/modeling_chatglm.py", line 1085, in _update_model_kwargs_for_generation attention_mask = torch.cat( IndexError: Dimension out of range (expected to be in range of [-2, 1], but got 3)
把你前面下载的chatglm模型删除,重新让hf下载
我的chatglm-6b模型就是最新的版本,还是有这个问题 @liangwq
我的chatglm-6b模型就是最新的版本,还是有这个问题 @liangwq
116行就是:batch = st.session_state['tokenizer'](input_text, return_tensors="pt") 这部分就是利用模型对你输入做预测 这样你分别试试: 1.把lora去掉看看能不能跑通 2.看看tokenizier是不是 最新的
我本地测试没有这个问题
1.安装依赖后,执行命令
streamlit run web_feadback.py --server.port=8080
,报错:2.将项目根目录的
modeling_chatglm.py
、configuration_chatglm.py
文件拷贝到 web_ui 目录下,再次执行streamlit run web_feadback.py --server.port=8080
,再次报错:3.将步骤2中添加的
modeling_chatglm.py
、configuration_chatglm.py
文件删除,替换为THUDM/ChatGLM-6B
项目chatglm-6b
模型中提供的文件,仍旧报错: