Open shripadk opened 4 months ago
Here is the notebook showing the run... First run asked for a RESTART, after doing that and running all the cells, the output is https://colab.research.google.com/drive/1kmZhQKBzpHBJRJvvp9PEdPEUMfMu6dh7?usp=sharing Just FYI.... btw, the output of the model is "","", but that's most likely an issue with the base model!! [ [@shripadk @alexsherstinsky]
With more epochs Gemma finetuning seems to work fine https://console.cloud.google.com/vertex-ai/colab/notebooks?project=document-ai-374204&activeNb=projects%2Fdocument-ai-374204%2Flocations%2Fus-central1%2Frepositories%2F87000216-df46-4358-8bb1-6bc933f4c82b [@shripadk @alexsherstinsky ]
@shripadk Are you still having the issues? A new version of Ludwig will be release next week (you may wish to try again). Please keep an eye on the release announcement next week in our Discord. Thank you!
@alexsherstinsky thanks for the heads up. I'll definitely take a look at it and get back to you on this. Will surely keep an eye on the release. Thanks again 🎉
Describe the bug
I have enabled 4-bit quantization for fine tuning mistralai/Mistral-7B-v0.1. Seems like Ludwig 0.10.1 depends on bitsandbytes < 0.41.0. But when I run the trainer I get the following warning:
To Reproduce Steps to reproduce the behavior:
model.yaml
):Expected behavior Should not show the warning on
bitsandbytes
version not supportingsave_pretrained
for 4-bit quantization.Environment (please complete the following information):
@alexsherstinsky