Folks were dissatisfied with lora training compared to finetunes. This will add the MLP layers to the training.
This is a stopgap measure until the lora refactor with presets comes online (this will be the "attention-mlp" preset, with the former being "attention-only").
Folks were dissatisfied with lora training compared to finetunes. This will add the MLP layers to the training.
This is a stopgap measure until the lora refactor with presets comes online (this will be the "attention-mlp" preset, with the former being "attention-only").