search

EricLBuehler / xlora

X-LoRA: Mixture of LoRA Experts
Apache License 2.0
159 stars 7 forks source link

Support for seq2seq models #29

Open maximus-21 opened 3 months ago

maximus-21 commented 3 months ago

I was trying to use xlora for combining Flan-T5 LoRAs and ran into error within apply_scalings_to_x, does xLoRA support seq2seq models such as Flan-T5 and BART ?

yuxiang-guo commented 3 months ago

I was trying to use xlora for combining Flan-T5 LoRAs and ran into error within apply_scalings_to_x, does xLoRA support seq2seq models such as Flan-T5 and BART ?

Hi, I also want to apply xlora to T5, did you try to do that?

maximus-21 commented 3 months ago

@yuxiang-guo Yeah I did. But I faced a tensor related error while doing the forward pass. I tried debugging through it but couldn't exactly understand what was the problem here.