Open user799595 opened 3 months ago
Do you know if Colab functions fine with Codellama-13b? It should work
Sorry, I don't know about Colab.
Is unsloth compatible with AWS?
Oh I meant did you try via https://colab.research.google.com/drive/135ced7oHytdxu3N2DNe1Z0kqjyYIkDXp?usp=sharing - it has some free GPU times, so its good for experimentation. If our Colabs break, then there's something wrong
I have same problem with codellama-13b-bnb-4bit.
I went to Colab, switched the model name and got the same error:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-2-3edea52bfdfc> in <cell line: 20>()
18 ] # More models at https://huggingface.co/unsloth
19
---> 20 model, tokenizer = FastLanguageModel.from_pretrained(
21 model_name = "unsloth/codellama-13b-bnb-4bit",
22 max_seq_length = max_seq_length,
6 frames
/usr/local/lib/python3.10/dist-packages/transformers/quantizers/quantizer_bnb_4bit.py in create_quantized_param(self, model, param_value, param_name, target_device, state_dict, unexpected_keys)
188 param_name + ".quant_state.bitsandbytes__nf4" not in state_dict
189 ):
--> 190 raise ValueError(
191 f"Supplied state dict for {param_name} does not contain `bitsandbytes__*` and possibly other `quantized_stats` components."
192 )
ValueError: Supplied state dict for model.layers.28.mlp.gate_proj.weight does not contain `bitsandbytes__*` and possibly other `quantized_stats` components.
Apologies just relocated to SF, hence the slowness!
Will investigate this!
I would like to finetune CodeLlama-13b in a memory efficient way.
I was able to do it with CodeLlama-7b, but failing with 13b.
I can't load the model
unsloth/codellama-13b-bnb-4bit
:I tried to quantize it first, but that also failed
Is CodeLlama-13b not supported? Should I be using a different model?