Closed Thytu closed 5 months ago
Update: even by no-longer applying continuous learning while fine-tuning using bf16 the model still does not converge.
Meaning a bug has been either integrated into the data_handler, in the training process or in forward method.
This showcase two things:
Even after using a rollback version of data_handler
the models still doesn't converge which might indicate an issue in either the forward methods or the training algorithm.
I'm currently running a training run using the 2bf64a77d726c3c13a93138c34b1067119315a41 commit to still if the issue still occurs.
While 2bf64a77d726c3c13a93138c34b1067119315a41 does seem to converge, it still takes an abnormal amount of time.
Now testing a rollback at a1df5f6d26bfc79872562e9178b35dab78b436a2
https://github.com/Thytu/SMIT/commit/a1df5f6d26bfc79872562e9178b35dab78b436a2 does converge
Now investigating which part of the code is faulty
I've identified two issues with the current setup:
Regarding the first issue, a fix is forthcoming. As for the second one, I'll prioritize other tasks for now and defer addressing it.
Quantization to 4bit also prevents the model to converge, might also be a good idea to create a short guide on what works and what doesn't as I'm already experimenting quite a lot with different configs.
Splitting that issue into two different issues:
The current default example aims to be able to be run on a
A100
40Go, thus it use Quantizationnf4
andLoRA
. However those changes seems to prevent the model to convert.The default example should be re-written to
both fit in aconverge.A100
40Go ANDQuantization + LoRA (VRAM: 18Go)![image](https://github.com/Thytu/SMIT/assets/43698357/c8341f54-24ee-42ce-a2f9-5133032f258a)
Fine-tuning (linear only, w/o Continual Learning) fp16 (VRAM: 52Go)![image](https://github.com/Thytu/SMIT/assets/43698357/c88cac88-9fd7-48af-b061-68850cb4476e)
Fine-tuning (linear only, w/ Continual Learning) fp16 (VRAM: 52Go)
Fine-tuning all fp16 (VRAM: 79Go)