Open jonflynng opened 3 months ago
There is a way to overwrite the code itself and allow input_embeds to be passed, but it'll be a bit of custom code - another way is to save the Unsloth model you just finetuned as a normal HF model, and then use HF directly (without Unsloth)
There is a way to overwrite the code itself and allow input_embeds to be passed, but it'll be a bit of custom code - another way is to save the Unsloth model you just finetuned as a normal HF model, and then use HF directly (without Unsloth)
The workaround works @danielhanchen but still would love that fast Unsloth inference :)
I'll see what I can do, but it'll be a bit tough to edit HF directly :(
I need to use the
generate
function by passing ininputs_embeds
for a multi-modal model I'm building, I can't useinput_ids
. I see Unsloth doesn't currently support this. Would it be possible to default to the normaltransformers
inference and use thegenerate
function without Unsloth?