Im struggling a little with getting huggingface trainer working for my use case.
is it possible to fine tune a llama3 model with a trainer i write with torch tune. Looking at huggingface it looks like it uses pytorch under the hood anyway. Any direction on how to achieve this would be greatly appreciated.
if i setup my model as like below, could this be run via a torch training script model, tokenizer = FastLanguageModel.from_pretrained(
model_name="unsloth/llama-3-8b-Instruct",
max_seq_length=max_seq_length,
dtype=dtype,
load_in_4bit=load_in_4bit,
token=get_hf_token(hf_licencepath),
)
model = FastLanguageModel.get_peft_model(
model,
r=16, # Choose any number > 0 ! Suggested 8, 16, 32, 64, 128
target_modules=[
"q_proj",
"k_proj",
"v_proj",
"o_proj",
"gate_proj",
"up_proj",
"down_proj",
],
lora_alpha=16,
lora_dropout=0, # Supports any, but = 0 is optimized
bias="none", # Supports any, but = "none" is optimized
# [NEW] "unsloth" uses 30% less VRAM, fits 2x larger batch sizes!
use_gradient_checkpointing="unsloth", # True or "unsloth" for very long context
random_state=3407,
use_rslora=False, # We support rank stabilized LoRA
loftq_config=None, # And LoftQ
)
Hi,
Im struggling a little with getting huggingface trainer working for my use case. is it possible to fine tune a llama3 model with a trainer i write with torch tune. Looking at huggingface it looks like it uses pytorch under the hood anyway. Any direction on how to achieve this would be greatly appreciated.
if i setup my model as like below, could this be run via a torch training script model, tokenizer = FastLanguageModel.from_pretrained( model_name="unsloth/llama-3-8b-Instruct", max_seq_length=max_seq_length, dtype=dtype, load_in_4bit=load_in_4bit, token=get_hf_token(hf_licencepath), )