search

ChantalMP / RaDialog_LLaVA

LLaVa Version of RaDialog
11 stars 3 forks source link

Saving & using the merged model #2

Closed gameveloster closed 2 months ago

gameveloster commented 2 months ago

Hi, great research!

Is there a way to save the merged model somewhere and load the merged model the next time we want to use the model?

Right now, it is taking a long time to load and merge the LoRA weights on start up.

Loading LLaVA from base model...
Loading checkpoint shards: 100%|███████████████████████████████████████████████| 2/2 [00:12<00:00,  6.16s/it]
Loading additional LLaVA weights...
Loading LoRA weights...
Merging LoRA weights...
Model is loaded...
Using downloaded and verified file: /tmp/biovil_t_image_model_proj_size_128.pt
Loaded additional vision tower weights...
Using downloaded and verified file: /tmp/biovil_t_image_model_proj_size_128.pt

I think the loading will be quicker if we load the merged model directly instead of merging the LoRA into the base everytime.

ChantalMP commented 2 months ago

Hi, I can imagine that could be possible, however, I don't know how. Though this should not be specific to RaDialog, so it should be easiest to find a general solution for LoRA weight loading and saving and then apply it here.