-
# 加载模型
print("加载模型----")
model = AutoModelForCausalLM.from_pretrained(
args.model_name_or_path,
device_map="auto",
# device_map=device_map,
load_in_4bit=T…
-
Thanks for the code,
Is there any tweak we can do to fit it in Google Colab (~15 GB Memory)
or should I just settle with QLoRA?
amrrs updated
11 months ago
-
I would like to ask how can I use QLoRa or Parameter-Efficient Fine-Tuning thin a model does not register at Hugging face instead is Based on [OFA](https://github.com/OFA-Sys/OFA)
i am trying to Qu…
-
Hi,
I am trying to do inference using
```python
from transformers import AutoProcessor, BitsAndBytesConfig, Idefics2ForConditionalGeneration
import torch
peft_model_id = "idefics2-finetuned"
…
-
如题,想知道8bit和4bit qlora效果有没有差别?
-
While running:
**python lolrl_qlora_llama_hh.py --sampling_strategy good_priority**
logs with error msg like below:
[2024-03-19 18:59:01,658] [INFO] [real_accelerator.py:133:get_accelerator] …
-
GPU硬件 4张2080Ti,单卡显存12G,指定单卡运行。
执行chatglm-6b微调
`CUDA_VISIBLE_DEVICES=0 python train_qlora.py \
--train_args_json chatGLM_6B_QLoRA.json \
--model_name_or_path /data/chatglm-6b \
--train_data_path d…
-
**Describe the bug**
When I'm fine tuning llama2 with deepspeed zero3, I set "zero3_init_flag: true" in my accelerate config. The "is_deepspeed_zero3_enabled()" in transformers/integrations/deepspeed…
-
When I try to load an unsloth 4bit model with
`llm = LLM("unsloth/mistral-7b-instruct-v0.3-bnb-4bit", dtype="half")`,
I get the error
`Cannot find any of ['adapter_name_or_path'] in the model's q…
-
Thanks you ,My english is senior,excuse me.
when i run the CUDA_VISIBLE_DEVICES=1 python3 train_qlora.py --train_args_json chatGLM_6B_QLoRA.json --model_name_or_path /T106/chatGLM-6B-QLoRA-main/chatG…