lugan777
commented
1 year ago Applying quantization to glm layers Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/gradio/routes.py", line 289, in run_predict output = await app.blocks.process_api( File "/usr/local/lib/python3.8/dist-packages/gradio/blocks.py", line 990, in process_api result = await self.call_function(fn_index, inputs, iterator) File "/usr/local/lib/python3.8/dist-packages/gradio/blocks.py", line 832, in call_function prediction = await anyio.to_thread.run_sync( File "/usr/local/lib/python3.8/dist-packages/anyio/to_thread.py", line 31, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "/usr/local/lib/python3.8/dist-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread return await future File "/usr/local/lib/python3.8/dist-packages/anyio/_backends/_asyncio.py", line 867, in run result = context.run(func, args) File "app.py", line 215, in predict resp = knowladge_based_chat_llm.get_knowledge_based_answer( File "app.py", line 127, in get_knowledge_based_answer vector_store = FAISS.load_local('faiss_index', self.embeddings) File "/usr/local/lib/python3.8/dist-packages/langchain/vectorstores/faiss.py", line 449, in load_local index = faiss.read_index( File "/usr/local/lib/python3.8/dist-packages/faiss/swigfaiss_avx2.py", line 10206, in read_index return _swigfaiss_avx2.read_index(args) RuntimeError: Error in faiss::FileIOReader::FileIOReader(const char*) at /project/faiss/faiss/impl/io.cpp:67: Error: 'f' failed: could not open faiss_index/index.faiss for reading: No such file or directory No sentence-transformers model found with name /dataset/ernie-3.0-base-zh. Creating a new one with MEAN pooling. Some weights of the model checkpoint at /dataset/ernie-3.0-base-zh were not used when initializing ErnieModel: ['cls.predictions.transform.dense.weight', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.bias']
- This IS expected if you are initializing ErnieModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing ErnieModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). No sentence-transformers model found with name /dataset/ernie-3.0-base-zh. Creating a new one with MEAN pooling. Some weights of the model checkpoint at /dataset/ernie-3.0-base-zh were not used when initializing ErnieModel: ['cls.predictions.transform.dense.weight', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.bias']
- This IS expected if you are initializing ErnieModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing ErnieModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). No compiled kernel found. Compiling kernels : /root/.cache/huggingface/modules/transformers_modules/chatglm-6b-int4/quantization_kernels_parallel.c Compiling gcc -O3 -fPIC -pthread -fopenmp -std=c99 /root/.cache/huggingface/modules/transformers_modules/chatglm-6b-int4/quantization_kernels_parallel.c -shared -o /root/.cache/huggingface/modules/transformers_modules/chatglm-6b-int4/quantization_kernels_parallel.so Kernels compiled : /root/.cache/huggingface/modules/transformers_modules/chatglm-6b-int4/quantization_kernels_parallel.so Load kernel : /root/.cache/huggingface/modules/transformers_modules/chatglm-6b-int4/quantization_kernels_parallel.so Setting CPU quantization kernel threads to 48 Using quantization cache Applying quantization to glm layers Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/gradio/routes.py", line 289, in run_predict output = await app.blocks.process_api( File "/usr/local/lib/python3.8/dist-packages/gradio/blocks.py", line 990, in process_api result = await self.call_function(fn_index, inputs, iterator) File "/usr/local/lib/python3.8/dist-packages/gradio/blocks.py", line 832, in call_function prediction = await anyio.to_thread.run_sync( File "/usr/local/lib/python3.8/dist-packages/anyio/to_thread.py", line 31, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "/usr/local/lib/python3.8/dist-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread return await future File "/usr/local/lib/python3.8/dist-packages/anyio/_backends/_asyncio.py", line 867, in run result = context.run(func, args) File "app.py", line 215, in predict resp = knowladge_based_chat_llm.get_knowledge_based_answer( File "app.py", line 127, in get_knowledge_based_answer vector_store = FAISS.load_local('faiss_index', self.embeddings) File "/usr/local/lib/python3.8/dist-packages/langchain/vectorstores/faiss.py", line 449, in load_local index = faiss.read_index( File "/usr/local/lib/python3.8/dist-packages/faiss/swigfaiss_avx2.py", line 10206, in read_index return _swigfaiss_avx2.read_index(args) RuntimeError: Error in faiss::FileIOReader::FileIOReader(const char*) at /project/faiss/faiss/impl/io.cpp:67: Error: 'f' failed: could not open faiss_index/index.faiss for reading: No such file or directory No sentence-transformers model found with name /dataset/ernie-3.0-base-zh. Creating a new one with MEAN pooling. Some weights of the model checkpoint at /dataset/ernie-3.0-base-zh were not used when initializing ErnieModel: ['cls.predictions.transform.dense.weight', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.bias']
- This IS expected if you are initializing ErnieModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing ErnieModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). No sentence-transformers model found with name /dataset/ernie-3.0-base-zh. Creating a new one with MEAN pooling. Some weights of the model checkpoint at /dataset/ernie-3.0-base-zh were not used when initializing ErnieModel: ['cls.predictions.transform.dense.weight', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.bias']
- This IS expected if you are initializing ErnieModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing ErnieModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). No compiled kernel found. Compiling kernels : /root/.cache/huggingface/modules/transformers_modules/chatglm-6b-int4/quantization_kernels_parallel.c Compiling gcc -O3 -fPIC -pthread -fopenmp -std=c99 /root/.cache/huggingface/modules/transformers_modules/chatglm-6b-int4/quantization_kernels_parallel.c -shared -o /root/.cache/huggingface/modules/transformers_modules/chatglm-6b-int4/quantization_kernels_parallel.so Kernels compiled : /root/.cache/huggingface/modules/transformers_modules/chatglm-6b-int4/quantization_kernels_parallel.so Load kernel : /root/.cache/huggingface/modules/transformers_modules/chatglm-6b-int4/quantization_kernels_parallel.so Setting CPU quantization kernel threads to 48 Using quantization cache Applying quantization to glm layers Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/gradio/routes.py", line 289, in run_predict output = await app.blocks.process_api( File "/usr/local/lib/python3.8/dist-packages/gradio/blocks.py", line 990, in process_api result = await self.call_function(fn_index, inputs, iterator) File "/usr/local/lib/python3.8/dist-packages/gradio/blocks.py", line 832, in call_function prediction = await anyio.to_thread.run_sync( File "/usr/local/lib/python3.8/dist-packages/anyio/to_thread.py", line 31, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "/usr/local/lib/python3.8/dist-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread return await future File "/usr/local/lib/python3.8/dist-packages/anyio/_backends/_asyncio.py", line 867, in run result = context.run(func, *args) File "app.py", line 189, in reinit_model return history + [[None, model_status]] TypeError: can only concatenate str (not "list") to str No sentence-transformers model found with name /dataset/ernie-3.0-base-zh. Creating a new one with MEAN pooling. Some weights of the model checkpoint at /dataset/ernie-3.0-base-zh were not used when initializing ErnieModel: ['cls.predictions.transform.dense.weight', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.bias']
- This IS expected if you are initializing ErnieModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing ErnieModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). No sentence-transformers model found with name /dataset/ernie-3.0-base-zh. Creating a new one with MEAN pooling. Some weights of the model checkpoint at /dataset/ernie-3.0-base-zh were not used when initializing ErnieModel: ['cls.predictions.transform.dense.weight', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.bias']
- This IS expected if you are initializing ErnieModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing ErnieModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). No compiled kernel found. Compiling kernels : /root/.cache/huggingface/modules/transformers_modules/chatglm-6b-int4/quantization_kernels_parallel.c Compiling gcc -O3 -fPIC -pthread -fopenmp -std=c99 /root/.cache/huggingface/modules/transformers_modules/chatglm-6b-int4/quantization_kernels_parallel.c -shared -o /root/.cache/huggingface/modules/transformers_modules/chatglm-6b-int4/quantization_kernels_parallel.so Kernels compiled : /root/.cache/huggingface/modules/transformers_modules/chatglm-6b-int4/quantization_kernels_parallel.so Load kernel : /root/.cache/huggingface/modules/transformers_modules/chatglm-6b-int4/quantization_kernels_parallel.so Setting CPU quantization kernel threads to 48 Using quantization cache Applying quantization to glm layers Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/gradio/routes.py", line 289, in run_predict output = await app.blocks.process_api( File "/usr/local/lib/python3.8/dist-packages/gradio/blocks.py", line 990, in process_api result = await self.call_function(fn_index, inputs, iterator) File "/usr/local/lib/python3.8/dist-packages/gradio/blocks.py", line 832, in call_function prediction = await anyio.to_thread.run_sync( File "/usr/local/lib/python3.8/dist-packages/anyio/to_thread.py", line 31, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "/usr/local/lib/python3.8/dist-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread return await future File "/usr/local/lib/python3.8/dist-packages/anyio/_backends/_asyncio.py", line 867, in run result = context.run(func, args) File "app.py", line 215, in predict resp = knowladge_based_chat_llm.get_knowledge_based_answer( File "app.py", line 127, in get_knowledge_based_answer vector_store = FAISS.load_local('faiss_index', self.embeddings) File "/usr/local/lib/python3.8/dist-packages/langchain/vectorstores/faiss.py", line 449, in load_local index = faiss.read_index( File "/usr/local/lib/python3.8/dist-packages/faiss/swigfaiss_avx2.py", line 10206, in read_index return _swigfaiss_avx2.read_index(args) RuntimeError: Error in faiss::FileIOReader::FileIOReader(const char*) at /project/faiss/faiss/impl/io.cpp:67: Error: 'f' failed: could not open faiss_index/index.faiss for reading: No such file or directory No sentence-transformers model found with name /dataset/ernie-3.0-base-zh. Creating a new one with MEAN pooling. Some weights of the model checkpoint at /dataset/ernie-3.0-base-zh were not used when initializing ErnieModel: ['cls.predictions.transform.dense.weight', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.bias']
- This IS expected if you are initializing ErnieModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing ErnieModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). No sentence-transformers model found with name /dataset/ernie-3.0-base-zh. Creating a new one with MEAN pooling. Some weights of the model checkpoint at /dataset/ernie-3.0-base-zh were not used when initializing ErnieModel: ['cls.predictions.transform.dense.weight', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.bias']
- This IS expected if you are initializing ErnieModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing ErnieModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). No compiled kernel found. Compiling kernels : /root/.cache/huggingface/modules/transformers_modules/chatglm-6b-int4/quantization_kernels_parallel.c Compiling gcc -O3 -fPIC -pthread -fopenmp -std=c99 /root/.cache/huggingface/modules/transformers_modules/chatglm-6b-int4/quantization_kernels_parallel.c -shared -o /root/.cache/huggingface/modules/transformers_modules/chatglm-6b-int4/quantization_kernels_parallel.so Kernels compiled : /root/.cache/huggingface/modules/transformers_modules/chatglm-6b-int4/quantization_kernels_parallel.so Load kernel : /root/.cache/huggingface/modules/transformers_modules/chatglm-6b-int4/quantization_kernels_parallel.so Setting CPU quantization kernel threads to 48 Using quantization cache Applying quantization to glm layers No sentence-transformers model found with name /dataset/ernie-3.0-base-zh. Creating a new one with MEAN pooling. /usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py:1383: UserWarning: positional arguments and argument "destination" are deprecated. nn.Module.state_dict will not accept them in the future. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details. warnings.warn( Some weights of the model checkpoint at /dataset/ernie-3.0-base-zh were not used when initializing ErnieModel: ['cls.predictions.transform.dense.weight', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.bias']
- This IS expected if you are initializing ErnieModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing ErnieModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). No sentence-transformers model found with name /dataset/ernie-3.0-base-zh. Creating a new one with MEAN pooling. Some weights of the model checkpoint at /dataset/ernie-3.0-base-zh were not used when initializing ErnieModel: ['cls.predictions.transform.dense.weight', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.bias']
- This IS expected if you are initializing ErnieModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing ErnieModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). No compiled kernel found. Compiling kernels : /root/.cache/huggingface/modules/transformers_modules/chatglm-6b-int4/quantization_kernels_parallel.c Compiling gcc -O3 -fPIC -pthread -fopenmp -std=c99 /root/.cache/huggingface/modules/transformers_modules/chatglm-6b-int4/quantization_kernels_parallel.c -shared -o /root/.cache/huggingface/modules/transformers_modules/chatglm-6b-int4/quantization_kernels_parallel.so Kernels compiled : /root/.cache/huggingface/modules/transformers_modules/chatglm-6b-int4/quantization_kernels_parallel.so Load kernel : /root/.cache/huggingface/modules/transformers_modules/chatglm-6b-int4/quantization_kernels_parallel.so Setting CPU quantization kernel threads to 48 Using quantization cache Applying quantization to glm layers The dtype of attention mask (torch.int64) is not bool 编程小白，求教大佬我的设置有什么问题吗
thomas-yanxin
求教为什么ai回答不出问题