-
Hi, has anyone tried to replicate the MMLU test results of Guanaco (OASST1) in Table 5 in the paper?
I have tried with the original training scripts provided at `./scripts/finetune_guanaco*.sh`.…
-
### System Info
transformers version -> 4.33
python version -> 3.10.6
I try to finetune this huggingface model : NousResearch/Llama-2-70b-chat-hf
With this huggingface dataset : mlabonne/gua…
-
Hi, while following along the instructions for the LLama-3-8B-Instruct
in https://colab.research.google.com/drive/1XamvWYinY6FOSX9GLvnqSjjsNflxdhNc?usp=sharing
I ran into what seems like a glitch.…
-
您好,正在尝试微调mixtral 8x7b,但是训练一段时间后loss不再下降,输出也有些问题
使用的config如下:
```python
# Copyright (c) OpenMMLab. All rights reserved.
import torch
from datasets import load_dataset
from mmengine.dataset im…
-
如果你遇到问题需要我们帮助,你可以从以下角度描述你的信息,以便于我们可以理解或者复现你的错误(学会如何提问不仅是能帮助我们理解你,也是一个自查的过程):
1、你使用了哪个脚本、使用的什么命令
CUDA_VISIBLE_DEVICES=0,1,6,7 python generate.py
2、你的参数是什么(脚本参数、命令参数)
parser.add_argument("--model_…
-
@edbeeching and I noticed sometimes the trained SFT models do not learn to stop generations. In other words, the model never learn to generate EOS tokens.
Upon some digging, I noticed this is mainl…
-
**Describe the bug**
Using the PromptNode with the HF Inference API Endpoint with the model `timdettmers/guanaco-33b-merged` will throw a recursion error, due to a (at least) missing `unk_token` of t…
-
**STEP 3
Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.
AGENT ERROR:
O…
-
Is anyone else having this issue when using the [finetune_guanaco_7b.sh](https://github.com/artidoro/qlora/blob/main/scripts/finetune_guanaco_7b.sh) script? I keep seeing the evaluation loss diverge r…
-
1、你使用了哪个脚本、使用的什么命令
使用的bash scripts/finetune.sh
2、你的参数是什么(脚本参数、命令参数)
参数
TOT_CUDA="0,1,3"
CUDAs=(${TOT_CUDA//,/ })
CUDA_NUM=${#CUDAs[@]}
PORT="12345"
DATA_PATH="data/newfl_data.json" #"../data…