运行量化llama2报错 AttributeError: 'LlamaForCausalLM' object has no attribute 'quantize'

[linuxdevopscn]

[['cuda', 'fp16i8']] Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 8.96it/s] Exception in thread Thread-1 (load_model): Traceback (most recent call last): File "/home/quanlian/mambaforge/envs/wenda/lib/python3.10/threading.py", line 1016, in _bootstrap_inner self.run() File "/home/quanlian/mambaforge/envs/wenda/lib/python3.10/threading.py", line 953, in run self._target(*self._args, **self._kwargs) File "/home/quanlian/aigc/wenda/wenda.py", line 53, in load_model LLM.load_model() File "/home/quanlian/aigc/wenda/llms/llm_llama.py", line 332, in load_model model = model.quantize(bits) File "/home/quanlian/mambaforge/envs/wenda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1630, in getattr raise AttributeError("'{}' object has no attribute '{}'".format( AttributeError: 'LlamaForCausalLM' object has no attribute 'quantize' No sentence-transformers model found with name ../models/m3e-base. Creating a new one with MEAN pooling.

[l15y]

https://github.com/wenda-LLM/wenda/issues/452#issuecomment-1647130499 所以解决了么？

[linuxdevopscn]

#452 (comment) 所以解决了么？ 和#452 不是一回事啊，这个报错是在运行量化llama2的时候报的，这个没解决。

[l15y]

@diannaojiang 看看？好像llama2没有实现quantize

[Hughhuh]

修复了吗

[wangxun6]

解决了嘛，我在导入llama2-中文模型时报的这个错误： Exception in thread Thread-1 (load model): Traceback (most recent cal1 last): File"c:\Usersdel1 .conda envs wendaliblthreading.py”,line 1016,in bootstrap_inner self.run File “c:\Users del1 .conda envswendalliblthreading.py”，line 953,in run self. target(*self._args, **self. kwargs File“D: wenda-main wenda.py”，line 53,in load model LLM.oad model File“D: wenda-main\11ms 11m_1lama.py”，line 295,in load_model tokenizer = AutoTokenizer.from pretrained( File"C:Usersidel1l.conda enys wenda libisite-packages transformers imodels autotokenization auto.py , line 655,in ”om pretrained raise ValueError( ValueError: Tokenizer class Llamalokenizer does not exist or is not currently imported. No sentence-transformers model found with name D: wenda-main m3e-base. Creating a new one with MEAN pooling.