-
When I create a LongLLM training environment on an NPU device, the installation of the flash-attention dependency is not possible. Are LongLLM training scripts allowed to be utilized on NPU?
-
Hi community,
I have been stuck on this issue for some time now and would greatly appreciate any help! I am trying to run the optimise_hyperparameter function over 2 A100GPU using PyTorch DDP strat…
-
Objective: To train and evaluate a model on RAGTruth dataset
Settings:
OS: Ubuntu WSL
Python: 3.12.4
NVIDIA Driver Version: 536.23
CUDA Version: 12.2
Replication steps:
1. Git clone
2. Run…
-
When running the `ORPO Unsloth Example.ipynb` notebook, I encountered an error during the execution of `orpo_trainer.train()`. The error occurs consistently across different GPU types and persists eve…
-
**Describe the bug**
ilab train on windows exits with `PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: './training_results/final\\mode…
-
First of all, many thanks for doing this! This is the only repo I'm aware of which allows doing Flux Lora training on a 16GB GPU.
I appreciate this is new and the lack of information is unavoidable. …
-
在用
torchrun --nproc_per_node=4 train.py --train_args_file train_args/sft/qlora/qwen2-7b-sft-qlora.json
训练qwen2+qlora+unsloth时(use_unsloth=true)出现错误:
ValueError: You can't train a model that has bee…
-
Hi, thank you for the nice library.
There seems to be a small mistake in the complexPyTorch.complexLayers.ComplexDropout2d layer, which gives a device mismatch error (torch version 2.0.1+cu118):
…
-
I use peft and SFTrainer to train a lora model. And I want to fintune this lora model on new datasets. How can I manage this? When using the code below, the grad norm is always zero.
```python
pef…
-
Hello! I don't quite get a reason of difference between playground and fitness_station iD presets. Both places are basically the same, only user age changes.
Current iD preset:
* [`leisure=playgro…