search

abhinand5 / tamil-llama

A New Tamil Large Language Model (LLM) Based on Llama 2
GNU General Public License v3.0
255 stars 34 forks source link

Issue resuming checkpoint while pretraining #7

Closed VishnuPJ closed 5 months ago

VishnuPJ commented 8 months ago

Error: model.load_adapter(resume_from_checkpoint, model.active_adapter, is_trainable=Tue) TypeError: PeftAdapterMixin.load_adapter() got an unexpected keyword argument 'is_tainable'

I have installed transformers asd peft as follows,

Custom Transformers Library: Use the transformers library from my clone. This version includes a small but crucial patch that enables resuming pretraining from checkpoints.

pip install git+https://github.com/abhinand5/transformers.git@abhinand5-deepspeed-patch

PEFT Library Version: Install a specific version of the peft library. Recent updates in the peft packages have introduced compatibility issues, and using the specified version helps avoid these problems.

pip install git+https://github.com/huggingface/peft.git@13e53fc

abhinand5 commented 8 months ago 
TypeError: PeftAdapterMixin.load_adapter() got an unexpected keyword argument 'is_tainable'

That's weird, especially the typo!

I have an active running environment that works fine. Please try that, install peft and transformers the way I've mentioned.

environment.txt

Change txt to yaml, looks like yaml upload isn't supported in GitHub