jina-ai / jerboa

LLM finetuning
Apache License 2.0
41 stars 4 forks source link

Fix bug in save_pretrained #59

Closed samsja closed 1 year ago

samsja commented 1 year ago

Context

save_pretrained from peft double the memory consumption of the video ram when saving the model to disk. THis is a problem with falcon 40b because it always lead to oom