-
Hi, please provide additional Instruction about How to fine-tuning QWEN model with BigDL speed up.
-
Hi,
I tried to perform FFT, using the notebook ```Continued pretraining - Korean + Unsloth.ipynb```
However, with unsloth/Llama-3.2-1B-bnb-4bit after instruction finetune, the model hallucinates…
-
If I followed the fine-tuning instructions and added a `query_instruction_for_retrieval`, should I use the same, different, or blank (`""`) instruction for the document ingestion part?
-
### Reminder
- [X] I have read the README and searched the existing issues.
### System Info
- `llamafactory` version: 0.9.1.dev0
- Platform: Linux-6.5.0-35-generic-x86_64-with-glibc2.35
- P…
-
# URL
- https://arxiv.org/pdf/2204.07705
# Affiliations
- Yizhong Wang, N/A
- Swaroop Mishra, N/A
- Pegah Alipoormolabashi, N/A
- Yeganeh Kordi, N/A
- Amirreza Mirzaei, N/A
- Anjana Arunku…
-
I have a base mode: **model_0**.
I created a Lora which is corresponding to instruction tuning: **lora_1**
The we merged **model_0** + **lora_1** to create: **model_1**.
Then we created a Lora base…
-
**Overview (aka. Goal Summary)**
Implement better management for artifacts (which includes data and models) - that we provide and that users will create (fine-tuned models, data generated after SDG).…
-
Dear authors,
Hello! I have a question regarding the two-stage fine-tuning process described in your work. Could you kindly help me understand how the two stages are connected during training? Specif…
-
Hi, I am an engineer from Intel and I work mostly on the performance optimization of PyTorch on intel Xeon CPUs (also I am the pytorch module maintainer for cpu performance). Just come across this…
-
Could you provide a slurm script to run the fine-tuning code? Apparently there are some issues with deepspeed, by just using the provided instructions.