web page推理无法结束

[mavisyyc]

执行语句： python ./Chinese-Vicuna/generate.py --model_path decapoda-research/llama-7b-hf --lora_path Facico/Chinese-Vicuna-lora-7b-3epoch-belle-and-guanaco --use_local 0

===================================BUG REPORT=================================== Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues

/root/anaconda3/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: /root/anaconda3 did not contain libcudart.so as expected! Searching further paths... warn(msg) /root/anaconda3/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/usr/local/nvidia/lib64'), PosixPath('/usr/local/nvidia/lib')} warn(msg) /root/anaconda3/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: /usr/local/TensorRT-7.0.0.11/lib:/usr/local/python3.7.0/lib:/usr/local/nvidia/lib:/usr/local/nvidia/lib64 did not contain libcudart.so as expected! Searching further paths... warn(msg) /root/anaconda3/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/usr/local/python3.7.0/include/python3.7')} warn(msg) CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64... CUDA SETUP: CUDA version lower than 11 are currently not supported for LLM.int8(). You will be only to use 8-bit optimizers and quantization routines!! CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so CUDA SETUP: Highest compute capability among GPUs detected: 7.0 CUDA SETUP: Detected CUDA version 102 /root/anaconda3/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU! warn(msg) CUDA SETUP: Loading binary /root/anaconda3/lib/python3.9/site-packages/bitsandbytes/libbitsandbytes_cuda102_nocublaslt.so... Namespace(model_path='decapoda-research/llama-7b-hf', lora_path='Facico/Chinese-Vicuna-lora-7b-3epoch-belle-and-guanaco', use_typewriter=1, use_local=0) The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization. The tokenizer class you load from this checkpoint is 'LLaMATokenizer'. The class this function is called from is 'LlamaTokenizer'. Facico/Chinese-Vicuna-lora-7b-3epoch-belle-and-guanaco/adapter_model.bin Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 33/33 [00:17<00:00, 1.88it/s] /root/anaconda3/lib/python3.9/site-packages/gradio/inputs.py:27: UserWarning: Usage of gradio.inputs is deprecated, and will not be supported in the future, please import your component from gradio.components warnings.warn( /root/anaconda3/lib/python3.9/site-packages/gradio/deprecation.py:40: UserWarning: optional parameter is deprecated, and it has no effect warnings.warn(value) /root/anaconda3/lib/python3.9/site-packages/gradio/deprecation.py:40: UserWarning: numeric parameter is deprecated, and it has no effect warnings.warn(value) Running on local URL: http://0.0.0.0:7860 Exception ignored in: <generator object Interface.attach_submit_events..fn at 0x7f0bb96bc890> Traceback (most recent call last): File "/root/anaconda3/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 862, in run context, func, args, future = item RuntimeError: generator ignored GeneratorExit Exception ignored in: <generator object Interface.attach_submit_events..fn at 0x7f0bb96bc900> Traceback (most recent call last): File "/root/anaconda3/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 862, in run context, func, args, future = item RuntimeError: generator ignored GeneratorExit Exception ignored in: <generator object Interface.attach_submit_events..fn at 0x7f0caeebea50> Traceback (most recent call last): File "/root/anaconda3/lib/python3.9/site-packages/gradio/routes.py", line 368, in reset_iterator app.iterators[body.session_hash][body.fn_index] = None RuntimeError: generator ignored GeneratorExit

[mavisyyc]

重启服务后没出现这问题，不知道是什么原因

[Facico]

应该是tokenizer没有eos的问题，详见note

[mavisyyc]

好的，谢谢！

[mavisyyc]

请问下我换成了yahma的为啥还是无法停止呀？

[Facico]

你可以用这个脚本测试一下tokenizer有没有问题：https://github.com/Facico/Chinese-Vicuna/blob/master/test_tokenizer.py