search

aws-neuron / transformers-neuronx

Apache License 2.0
95 stars 28 forks source link

Support for encoder-decoder models #51

Closed kwontaek-amazon closed 9 months ago

kwontaek-amazon commented 11 months ago

Hi team,

I wondered if the tool has support for any encoder-decoder models too (like FLAN-T5 or FLAN-UL2)? If not at the moment, do you have a plan for it?

Thanks!

aws-donkrets commented 11 months ago

Hi kwontaek-amazon - We support T5 models and you can find a tutorial on it here. However we are working on updating this tutorial to support the flan-t5 models. Expect that to be completed in a future release.

aws-maens commented 11 months ago

Hi @kwontaek-amazon , encoder-decoder models inference like T5 are supported using NeuronX Distributed library, please see T5 Tutorial in the Neuron documentation. Encoder-decoder models are not part of Transformers NeuronX current roadmap.