-
I have trained qlora model with unsloth and I want to serve with vllm but I did not found a way to serve model in8/4 bits ?
-
hi all, i was giving the CPUOffloadOptimizer a try and found two issues when using with QLoRA single device in torchtune:
1. When using a LR scheduler i got. Maybe there is a way to inherit the opt…
-
During fine-tuning, it's possible that special tokens are added that are specific to the adapter. During decoding, we should be using the special tokens, and ensure the correct stop tokens, padding, e…
-
## タイトル: UnifiedCrawl：低リソース言語向けLLMの低コスト適応のための統合コモンクロール
## リンク: https://arxiv.org/abs/2411.14343
## 概要:
大規模言語モデル（LLM）は、訓練データの不足により、低リソース言語において性能が低い。本研究では、Common Crawlコーパス全体から低リソース言語のテキストデータを効率的に収集す…
-
I found these two models on OpenAI's LLM, and they exhibit significant differences in perf…
MDK-L updated
11 months ago
-
**base-model: Weyaxi/Dolphin2.1-OpenOrca-7B**
**Scenario:**
- followed the following guidelines - https://github.com/intel-analytics/ipex-llm/tree/main/python/llm/example/GPU/LLM-Finetuning/QLoRA…
-
## Upvote & Fund
- We're using [Polar.sh](https://polar.sh/kyegomez) so you can upvote and help fund this issue.
- We receive the funding once the issue is completed & confirmed by you.
- Thank yo…
-
Facing an issue while tuning LLAMA-2-7b-chat on which I request some suggestions.
1. I use a specific system prompt that defines some keys, and then provide an instruction and ask the model to gene…
-
版本：
`Python 3.8
torch==2.0.1
transformers==4.30.2
peft==0.4.0
bitsandbytes==0.39.0`
config设置：
`
{
"output_dir": "mypath/trans-baichuan2-13b",
"model_name_or_path": "/mypath/Baichua…
-
implement support for CPUs and other GPUs