-
Dear Team
Thank you so much for releasing the model.. I am trying to integrate the flux model for some use case for which I requires the unet, and image_encoder. I find in the FluxPipeline there exi…
-
I use 5,000 256x256 images to train on first step with this command
`python train.py --data_dir ../dataset256/dataset256 \
--bit_length 48 --image_resolution 256 --num_epochs 100 --data_size 5000 …
-
### System Info
```Shell
- `Accelerate` version: 0.33.0
- Platform: Windows-10-10.0.22631-SP0
- `accelerate` bash location: C:\Users\Nech\anaconda3\envs\transformer-multi-device\Scripts\accelera…
-
I pretrain with script
```
torchrun --nproc_per_node="${NUM_GPUS}" --nnodes="${NNODES}" \
"./llava/train/train_mem.py" \
--model_name_or_path ${LLM_VERSION} \
--version ${PROMPT_VERSI…
-
### System Info
Latest TRL from source, can't run TRL env rn as cluster is shut down but I'm installing everything from source.
If required will restart cluster and run.
### Information
- [ ] Th…
-
### 🐛 Describe the bug
When using `torch.nn.functional.scaled_dot_product_attention` with autograd a tensor filled with NaN values are returned after a few backward passes. `Using torch.autograd.s…
-
Hi, I'm currently trying to run DeepSeek Coder v2 on a single node with the following setup:
Node 1: Two A6000 GPUs (48GB each) and 192GB of RAM
Node 2: Two 4090 GPUs (24GB each) and 64GB …
BGFGB updated
1 month ago
-
**Is your feature request related to a problem? Please describe.**
Your Seq2SeqSharp project already support LSTMs. Please consider to implement the RWKV large language "linear attention" idea into y…
-
Hi there, so I am loading a finetuned Llama 2 13b model, and I get this error.
Here's part of the error:
File /usr/local/lib/python3.10/dist-packages/unsloth/models/loader.py:172, in FastLanguag…
-
Hi GaLore Team, congratulations for the interesting work!
I am trying to fine-tune llama-3 8B model using GaLore but getting this error:
`torch._C._LinAlgError: linalg.svd: The algorithm failed to…