Can lade accelerate T5?

hao-ai-lab / LookaheadDecoding

Apache License 2.0

1.04k stars 63 forks source link

Can lade accelerate T5? #21

Closed yjdy closed 7 months ago

yjdy commented 7 months ago

Thank for your work. Did you test lade on other generate model, like T5 or Bart? Can it accelerate them?

Viol2000 commented 7 months ago

Hi, currently, we do not support T5/Bart models. But theoretically, encoder-decoder models can be supported, and speedup can be achieved. It requires engineering efforts.