-
### 🐛 Describe the bug
i use pytorch==2.3.0 and peft to train llama3 8b , when i run my code, its raise error like:
```text
torch._amp_foreach_non_finite_check_and_unscale_(
RuntimeError:…
-
Hello so i was fine-tuning a llama-2 model with unsloth using a tokenizer of my own, it has an extended vocabulary of around 48000 tokens in total, the tokenizer is compatible and checks have been mad…
-
Hi, I went through your video and found it quite helpful. I am working on a different domain so I have some questions . Hoping you could help:
My typical data samples look like ` Prove that \sin ^2 …
-
# Zero-Shot Learning in Modern NLP | Joe Davison Blog
State-of-the-art NLP models for text classification without annotated data
[https://joeddav.github.io/blog/2020/05/29/ZSL.html](https://joeddav.…
-
Internal user reported a stall during the .Fit() of the word embedding transform.
On first use of the word embedding transform, it downloads the word embedding model from the CDN.
To test:
1. …
-
#### Context
Solving the easier Kata on Codewars is a great way lo learn a programming language, and creating Anki cards when I learn new syntax or tricks helps me memorize them faster ([Anki](https:/…
-
I would like to ask for your advice on the following two questions.
1. DPO train does not seem to support DeepSpeed ZeRO. After manually integrating `DPOAlignerArguments` with the `FinetunerArguments…
-
Any recommendations to train or fine-tune model on new language.
1. Does training for new language (E.x. arabic) will work on existing pre-trained models ? Or it has to be from scratch.
2. What…
-
I saw that there is a Hierarchical Attention Network model included in the directory: reproduction/text_classification/model/HAN.py.
I realized that the input for HAN is different from other models (…
-
Whenever I follow the installation instructions, apply autogptq to llama2-hf, then try to run qalora.py on the checkpoints produced by autogptq. I install autogptq using pip install autogptq[triton], …