-
Hello!
The calculation accuracy of QLora training is float16, what is the calculation accuracy of qa-lora training?
My fine-tuning TechGPT-7b was successful with QLora, but using qa-lora always repo…
-
Hello @tdrussell,
First of all, thank you very much for your great repo! It is absolutely great work to pull all these optimization solutions together.
When I use the repo, I try to use the bf16 e…
-
Hi,
I am trying to use the Qlora code as provided in the repo on a Sapphire Rapids, Flex GPU machine.
I was able to run the [qlora_finetuning.py](https://github.com/intel-analytics/BigDL/blob/m…
-
Hi folks,
I'm running into an issue finetuning the 70B Llama 2 model with 4bit qLoRA using the FastChat package, and I'm wondering if anyone else has encountered similar issues or has suggestions f…
-
I try your example `Kaggle Mistral 7b Unsloth notebook`. The only thing I changed - I changed Flase to True in this line:
`if True: model.save_pretrained_merged("model", tokenizer, save_method = "mer…
-
I have problems resuming a checkpoint. What I did:
1) `python qlora.py --model_name_or_path huggyllama/llama-7b`
2) abort when a checkpoint has been written
3) `python qlora.py --model_name_or_path…
-
### Feature request
Is there any chance we coukd get this 4bit adam optimizer added to tranformers?
It has nearly the same performance as 32bit adam with significant drop in vram overhead.
[repo…
-
# 加载模型
print("加载模型----")
model = AutoModelForCausalLM.from_pretrained(
args.model_name_or_path,
device_map="auto",
# device_map=device_map,
load_in_4bit=T…
-
During training of glora i am facing the error "The size of tensor a (8388608) must match the size of tensor b (4096) at non-singleton dimension 0"
I am using glors from PR https://github.com/Arnav…
-
Hi, Dear author:
It seems the llava-next is really insightful exploreing work. Please kindly release the training and inference code asap, thank you very much.