Hi, I'm trying to do the quantization with the Bloom model and cannot find the equivalent attribute num_linear_layers. In the file run.py, you said that it's llama-specific but when I used the llama model ("NousResearch/Llama-2-7b-chat-hf"), I couldn't find the num_linear_layers either. Please check again!
Hi, I'm trying to do the quantization with the Bloom model and cannot find the equivalent attribute
num_linear_layers. In the file run.py, you said that it's llama-specific but when I used the llama model ("NousResearch/Llama-2-7b-chat-hf"), I couldn't find thenum_linear_layerseither. Please check again!