Open thewintersun opened 1 year ago
您好, 可能是bitsandbytes库的问题.您可以试一下这样 cd ~/.conda/envs/lawgpt/lib/python3.10/site-packages/bitsandbytes cp libbitsandbytes_cuda110.so libbitsandbytes_cpu.so
似乎没有起作用,出来的错误如下:
python webui.py --base_model ./models/base_models/chinese-llama-7b-merged/ --lora_weights ./models/lora_weights/legal-lora-7b/ --prompt_template "law_template" --load_8bit True
===================================BUG REPORT=================================== Welcome to bitsandbytes. For bug reports, please run
python -m bitsandbytes
bin /home/jovyan/.conda/envs/lawgpt/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cpu.so /home/jovyan/.conda/envs/lawgpt/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: /home/jovyan/.conda/envs/lawgpt did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths... warn(msg) CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths... /home/jovyan/.conda/envs/lawgpt/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/usr/local/cuda/lib64')} warn(msg) /home/jovyan/.conda/envs/lawgpt/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: No libcudart.so found! Install CUDA or the cudatoolkit package (anaconda)! warn(msg) CUDA SETUP: Highest compute capability among GPUs detected: 6.1 CUDA SETUP: Detected CUDA version 117 /home/jovyan/.conda/envs/lawgpt/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU! warn(msg) CUDA SETUP: Loading binary /home/jovyan/.conda/envs/lawgpt/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cpu.so... Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s]Error named symbol not found at line 528 in file /mmfs1/gscratch/zlab/timdettmers/git/bitsandbytes/csrc/ops.cu
您好, 试试这样 cd ~/.conda/envs/lawgpt/lib/python3.10/site-packages/bitsandbytes cp libbitsandbytes_cuda117_nocublaslt.so libbitsandbytes_cpu.so
可以了
--load_8bit True 会出来这个错误, 如果False就不会有错误,但是结果是乱码。
Traceback (most recent call last): File "/home/jovyan/dluo/github/LaWGPT/webui.py", line 211, in
fire.Fire(main)
File "/home/jovyan/.conda/envs/lawgpt/lib/python3.10/site-packages/fire/core.py", line 141, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "/home/jovyan/.conda/envs/lawgpt/lib/python3.10/site-packages/fire/core.py", line 475, in _Fire
component, remaining_args = _CallAndUpdateTrace(
File "/home/jovyan/.conda/envs/lawgpt/lib/python3.10/site-packages/fire/core.py", line 691, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
File "/home/jovyan/dluo/github/LaWGPT/webui.py", line 42, in main
model = LlamaForCausalLM.from_pretrained(
File "/home/jovyan/.conda/envs/lawgpt/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2784, in from_pretrained
) = cls._load_pretrained_model(
File "/home/jovyan/.conda/envs/lawgpt/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3125, in _load_pretrained_model
new_error_msgs, offload_index, state_dict_index = _load_state_dict_into_meta_model(
File "/home/jovyan/.conda/envs/lawgpt/lib/python3.10/site-packages/transformers/modeling_utils.py", line 717, in _load_state_dict_into_meta_model
set_module_8bit_tensor_to_device(
File "/home/jovyan/.conda/envs/lawgpt/lib/python3.10/site-packages/transformers/utils/bitsandbytes.py", line 78, in set_module_8bit_tensor_to_device
new_value = bnb.nn.Int8Params(new_value, requires_grad=False, has_fp16_weights=has_fp16_weights).to(device)
File "/home/jovyan/.conda/envs/lawgpt/lib/python3.10/site-packages/bitsandbytes/nn/modules.py", line 227, in to
return self.cuda(device)
File "/home/jovyan/.conda/envs/lawgpt/lib/python3.10/site-packages/bitsandbytes/nn/modules.py", line 191, in cuda
CB, CBt, SCB, SCBt, coo_tensorB = bnb.functional.double_quant(B)
File "/home/jovyan/.conda/envs/lawgpt/lib/python3.10/site-packages/bitsandbytes/functional.py", line 1642, in double_quant
row_stats, col_stats, nnz_row_ptr = get_colrow_absmax(
File "/home/jovyan/.conda/envs/lawgpt/lib/python3.10/site-packages/bitsandbytes/functional.py", line 1531, in get_colrow_absmax
lib.cget_col_row_stats(ptrA, ptrRowStats, ptrColStats, ptrNnzrows, ct.c_float(threshold), rows, cols)
File "/home/jovyan/.conda/envs/lawgpt/lib/python3.10/ctypes/init.py", line 387, in getattr
func = self.getitem(name)
File "/home/jovyan/.conda/envs/lawgpt/lib/python3.10/ctypes/init.py", line 392, in getitem
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: /home/jovyan/.conda/envs/lawgpt/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cpu.so: undefined symbol: cget_col_row_stats
如果设置是False就出来的答案莫名其妙:![image](https://github.com/pengxiao-song/LaWGPT/assets/8595690/6ebdff87-5815-4cb8-af2f-88737f433b32)
GPU环境:![image](https://github.com/pengxiao-song/LaWGPT/assets/8595690/a214a788-6773-48f7-b34f-82182be848c8)