Closed juanps90 closed 11 months ago
Which version of transformers are you using?
Sorry for the delay in the answer:
transformers @ git+https://github.com/huggingface/transformers.git@17a55534f5e5df10ac4804d4270bf6b8cc24998d
Updated to latest git commit from transformers, it's training. Thank you!
I've successfully finetuned LLaMA v1 up to 34B and LLaMA v2 up to 13B, but can't seem to get CodeLLaMA to work with this repo.
This is the error that I'm stumbling upon:
These are the training parameters:
Hardware is dual RTX 3090's, finetuning is working with LLaMA v2 13B. CodeLLama weights by The-Bloke (https://huggingface.co/TheBloke/CodeLlama-34B-GPTQ)