erfanzar / EasyDeL

Accelerate, Optimize performance with streamlined training and serving options with JAX.
https://easydel.readthedocs.io/en/latest/
Apache License 2.0
182 stars 23 forks source link

Issue saving and converting the Gemma 2 model after training #168

Closed sparsh35 closed 1 month ago

sparsh35 commented 1 month ago

Describe the bug I am getting this error to convert the model 'Gemma2' with SFT trainer to pytorch after training , Lm.head weights are missing image

To Reproduce use this script just replace the training model with Gemma 2 9b pt , Link to kaggle code of SFT example

sparsh35 commented 1 month ago

This is saving screenshot of saving of the file image

sparsh35 commented 1 month ago

Got it , i didn't read complete docs , for those who may have this problem correct code is here , Code to SFT trainer with logic to save with tie word embeddings