-
### Is there an existing issue for this?
- [X] I have searched the existing issues and checked the recent builds/commits
### What would your feature do ?
The new method enables LoRa weight mo…
-
-
### Expected Behavior
Lora should load with minimal vram overhead (considering it is a small lora; the 4/4 rank one is 40mb).
### Actual Behavior
Large vram usage increase when loading certain lora…
-
Just wanted to get your thoughts on the following behaviour.
We have an implementation of topic aliases. The consequences of sending an invalid topic alias are quite severe (immediate disconnect by…
-
command:
` python -m mlx_lm.lora --model models/Qwen1.5-32B-Chat --data data/ --train --iters 1000 --batch-size 8 --lora-layers 12
`
result:
`
Loading pretrained model
Special tokens have been a…
-
Hi, I have got some checkpoints after training, how can I continue to train by resuming from the checkpoints?
-
As a developer, one of the challenges I've encountered is that lora networks, when trained on standard models, tends to underperform when used with inpainting models.
The possible solution: Allowin…
-
No problems with Step 1 & 2, but for Step 3, it returns a runtime error: The size of tensor a (6144) must match the size of tensor b (8192) at non-singleton dimension 0
**PyTorch:1.12.1, 1.13.1, 2.…
-
I run the test program use "python train.py --actor-model facebook/opt-13b --reward-model facebook/opt-350m --num-gpus 8".The program can run normally.But I modified the parameter max_ answer_ seq_ le…
-
### Category
Other
### Hardware
Not Applicable
### Firmware Version
2.2.24
### Description
Accorting to lora alliance the band plan starts at 433.05Mhz.
Not 433.00. It ends at 43…