-
Take /root/share/new_models/datasets/CLoT-cn as datasets,
To finetune /root/share/new_models/datasets/CLoT-cn with lora:
The config settings for loralike following:
```
###########################…
-
hi, i'm looking over the optimizations in the trainer here, and trying to port them to the `transformers.trainer.Trainer` for use with llama2
i put together this simple script to view the differenc…
tmm1 updated
4 months ago
-
### Please check that this issue hasn't been reported before.
- [X] I searched previous [Bug Reports](https://github.com/OpenAccess-AI-Collective/axolotl/labels/bug) didn't find any similar reports…
-
Is it possible to run `QLoRA` finetuning on more than a single device? I don't see any [configs](https://github.com/pytorch/torchtune/tree/main/recipes/configs/) for `QLoRA` other than for `single_de…
-
When following initial setup steps from: https://github.com/tenstorrent/tt-inference-server/tree/main/vllm-tt-metal-llama3-70b#vllm-tt-metalium-llama-31-70b-inference-api, fails due to missing HF toke…
-
我用lora微调了qwen2模型，保存在本地，想使用qwen agent，但是不会加载微调后的模型，哪怕是加载保存在本地的qwen原始模型也不行。
def init_agent_service():
llm_cfg = {'model': r'./model/Qwen2-0.5B',
'model_server':'http://127.0.0.1:…
-
### System Info
I use the SFTTrainer for my qlora fine-tuning for Mistral Instruct 2 model. I use unsloth to make my training faster. I have run the code multiple times before but today I got the …
-
So I have a GPTQ llama model I downloaded (from TheBloke), and it's already 4 bit quantized. I have to pass in False for the load_in_4bit parameter of:
```
model, tokenizer = FastLlamaModel.from_pr…
-
> Please provide us with the following information:
> ---------------------------------------------------------------
### This issue is for a: (mark with an `x`)
```
- [x] bug report -> please…
-
Hi team,
I tried to do QLora for 30B llama with unsloth. I found that there is no much improvement on speed and memory usage. The detaild are as following.
seq_length=8192
batch size=1
use flash a…
ghost updated
3 weeks ago