-
Is there a small model trainer, such as llama2.c?
-
I have some questions:
- Have you applied this SoftDTW for EATS.
- Can it be used as a loss function and backward
- How does the implementation perform compare to this: https://github.com/Ma…
-
### 🐛 Describe the bug
@record
def training_function(args):
# get some base rank info
# metric = evaluate.load("glue", "mrpc")
world_size = os.getenv("WORLD_SIZE")
rank = os.gete…
-
steps to reproduce
1) start a runpod container with the pytorch 2.01 template and lots of disk space
2) run your sample command on a properly formatted dataset:
python -m llamatune.train \
--m…
-
Hi,
I've followed the this blog post https://huggingface.co/blog/fine-tune-whisper to finetune Whisper with my own dataset.
Everything seems to be working as expected. However, I've noticed a stra…
-
### Please check that this issue hasn't been reported before.
- [X] I searched previous [Bug Reports](https://github.com/OpenAccess-AI-Collective/axolotl/labels/bug) didn't find any similar reports.
…
-
### Describe the issue
Issue:
Hello, I'm running the finetuning job for llava-13B, but I keep getting the error message "RuntimeError: mat1 and mat2 must have the same dtype". It's possible that the…
-
**Describe the bug**
使用deepspeed zero3微调时,只需11G显存,我用的是4X1080Ti,CUDA 12.1 ,torch 2.0.1
但是每当epoch结束保存模型时,总会出现下面的错误:
```
File "/usr/local/lib/python3.10/dist-packages/deepspeed/runtime/engine.py", …
-
**System information**.
- Have I written custom code (as opposed to using a stock example script provided in Keras):
- OS Platform and Distribution: Linux Ubuntu 22.04.2 LTS
- TensorFlow installe…
-
## Reference
- [PyTorch中在反向传播前为什么要手动将梯度清零?](https://www.zhihu.com/question/303070254/answer/573037166)
- [Gradient accumulation](https://github.com/pytorch/pytorch/blob/d927aee312fa95b7803cb976fee95…