search

triton-inference-server / fastertransformer_backend

BSD 3-Clause "New" or "Revised" License
411 stars 134 forks source link

Support mt5 (t5 v1.1)? #37

Closed hong8c closed 1 year ago

hong8c commented 2 years ago

Is there any plan to support mt5 (t5 v1.1)? Thank you.

byshiue commented 2 years ago

It is supported. The workflow is same to T5.

hong8c commented 2 years ago

Thank you, @byshiue However when I download T5 v1.1 models from huggingface model repository and followed the same workflow, I've got some wield outputs. I tested several times. I will post more detailed information about the problem. Thank you!

byshiue commented 1 year ago

This is supported since v1.2. Here are some guide https://github.com/NVIDIA/FasterTransformer/blob/main/docs/t5_guide.md#how-to-use. You can reuse the t5 example to run mt5 easily.