-
Hi, thanks for your great job. I found some errors when I wanted to continue my LoRA finetuning.
```
09/04/2024 15:54:23 - INFO - accelerate.accelerator - Loading states from output/tulu_v2_dolly_…
-
Which one would you recommend to use to get the best possible performance?
-
Hi,
I've been attempting to reproduce an experiment involving the finetuning of the Llama-2-7b-hf model,…
-
Thanks for so great innovative methods to avoid Catastrophic Forgetting ,Need to help to provide the example code to finetune llama 3 8B instruct ,would you be kind to help ?
-
### System Info
WARNING:root:A model is loaded from './saved_models/fp-meta-llama3', and no v_head weight is found.
### Information
- [ ] The official example scripts
- [X] My own modified scripts…
-
Following up on a longer internal discussion we had (cc @carmocca @lantiga @awaelchli ), we want to support the following user-friendly API in LitGPT:
```bash
# ligpt [action] [model]
litgpt dow…
rasbt updated
6 months ago
-
### Model Series
Qwen2.5
### What are the models used?
Qwen2.5-0.5B-Instruct
### What is the scenario where the problem happened?
inference with transformers, deployment with vllm/PeftModelForCau…
-
I followed the current Google Colab notebook for finetuning Llama 3.1 8B Instruct, which includes the use of `train_on_responses_only`. `train_on_responses_only` adds a `labels` column to `trainer.tr…
-
All of the models supported by torchtune currently have rather low context lengths (
-
config file:
`# Model
pretrained_model_name_or_path = '/data/llm/cache/Qwen2-7B-Instruct/'
use_varlen_attn = True
# Data
data_files = ['/workspace/xtuner/sft_openai.json']
prompt_template = …