search

tloen / alpaca-lora

Instruct-tune LLaMA on consumer hardware
Apache License 2.0
18.59k stars 2.21k forks source link

Upscale models #237

Open dynamiccreator opened 1 year ago

dynamiccreator commented 1 year ago

Would it be possible to "upscale" an adapter_model.bin file fine tuned with the 7B model, so it can be applied to bigger models like 13B,30B and 65B?

The goal would be to reduce computing time like fine tune with 7B and doing inference with bigger models.

AngainorDev commented 1 year ago

Definitively not

(until someone finds a magic trick, who knows, but that looks a lot like infinite compression)

dynamiccreator commented 1 year ago

That's a pity. But can one do it anyways somehow (probably just expecting gibberish as output), to just get it run with inference?

I don't know why you talk about compression here? My goal was just to train the adapter, so it contains some new data / instructions and then instead of "stacking" it on top of the 7B model, I apply it to the 13B model for example and have the new information "injected" into the 13B model. I do understand that it will most likely produce gibberish as both models have different weights, but I just want to try it.