-
AttributeError Traceback (most recent call last)
Cell In[6], line 1
----> 1 model = LlamaForCausalLM.from_pretrained(".[./hf_llama](https://vscode-remote+ssh-002dremote-00…
-
Fine tune flan t5 xl and flan t5 xxl using qlora and has a problem with learning rate 0.0 and loss 0.0 ? Can anyone resolve this problem ? Thanks
-
因为`bitsandbytes`实现模型量化的时候是通过重载`.cuda()`函数实现的,也就是说模型在放到显卡的时候会发生量化(改变tensor维度)。在微调的时候,加载的预训练权重是fp16的,所以需要设置`args.device='cpu'`,把权重加载进来再调用`.cuda()`。因为这个是`bitsandbytes`的实现,我们也没办法控制,只能适配。
…
-
I added a 4-bit load after the command LoRA training with ZeRO-3 on two or more GPUs to achieve a mix of QLoRA and ZeRO-3. But the program encountered the following error:
RuntimeError: expected ther…
-
For reference, LLaMA-Factory [claims](https://huggingface.co/mistralai/Mixtral-8x7B-v0.1/discussions/10) that using their toolkit you can QLoRA fine-tune mixtral with 28GB of VRAM.
-
`python qlora.py --model_name_or_path decapoda-research/llama-13b-hf`
(I have updated the tokenizer_config.json and config.json as per the various discussions [here](https://huggingface.co/decapoda…
-
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 954.00 MiB (GPU 0; 14.56 GiB total capacity; 12.31 GiB already allocated; 486.50 MiB free; 13.17 GiB reserved in total by PyTorch) If…
eanfs updated
6 months ago
-
求助关于显存的问题,我用xtuner的qlora配置用24G的显存是可以执行微调的,但是我希望尝试不量化的lora微调就是把关于量化的配置删掉后显示lora微调的显存不够是为什么呀,我采用chatglm3的官方lora微调24G是够的呀,后来我尝试利用双卡微调也是显示不够,是不是双卡不能叠加GPU而是只能加快速度呀,求助前辈们应该如何进行不量化的lora呢
-
Please have a script available HF trainer with either deep seed directly or accelerate .
QLORA HF trainer with model merge and infer. Thank you .
-
`python qlora.py --model_name_or_path ../text-generation-webui/models/lmsys_vicuna-7b-v1.3/ ─╯
============…