model saving error

[imrankh46]

the trainer not save the mode weights . its give me the following error

OutOfMemoryError: CUDA out of memory. Tried to allocate 16.00 MiB (GPU 0; 14.75 
GiB total capacity; 12.97 GiB already allocated; 6.81 MiB free; 13.69 GiB 
reserved in total by PyTorch) If reserved memory is >> allocated memory try 
setting max_split_size_mb to avoid fragmentation.  See documentation for Memory 
Management and PYTORCH_CUDA_ALLOC_CONF

[Facico]

Your error is exceeding the GPU memory limit. It should be unrelated to model saving. Did your program train properly when it was running?

[imrankh46]

Your error is exceeding the GPU memory limit. It should be unrelated to model saving. Did your program train properly when it was running?

No, when all the epochs completed, so they showing this behavior. We can not save llama weights like other model using trianer.save_pretrianed() method Or model.save_model().

[SunnyMarkLiu]

Same error to me!

[Facico]

What is the version of your transformers?

[imrankh46]

Same error to me!

I solve the error. Just add this code.

model.cpp()

And then save the model

[imrankh46]

What is the version of your transformers?

Same like you.

[Facico]

@imrankh46 Our transformers is pulled directly from github, so there may be a slight difference. The commit hash of our transformers at the time was roughly the same as ff20f9cf3615a8638023bc82925573cb9d0f3560. Maybe you can slove the question by uninstalling transformers and reinstalling it as "git+https://github.com/huggingface/transformers@ff20f9cf3615a8638023bc82925573cb9d0f3560"

[imrankh46]

@imrankh46 Our transformers is pulled directly from github, so there may be a slight difference. The commit hash of our transformers at the time was roughly the same as ff20f9cf3615a8638023bc82925573cb9d0f3560. Maybe you can slove the question by uninstalling transformers and reinstalling it as "git+https://github.com/huggingface/transformers@ff20f9cf3615a8638023bc82925573cb9d0f3560"

I tried, but they not working. I think The llama model code or tokenizer written in cpp. The model is train successfully.

After saving they give out of cuda error.

I will also try your approach..

[Facico]

same issue in other repo like you. You can also refer to their method to downgrade the version of bitsandbytes

[imrankh46]

same issue in other repo like you. You can also refer to their method to downgrade the version of bitsandbytes

Thank you.