search

NVIDIA / FasterTransformer

Transformer related optimization, including BERT, GPT
Apache License 2.0
5.81k stars 889 forks source link

mt5 support？ #256

Closed Elisewyh closed 2 years ago

Elisewyh commented 2 years ago

Description

Is it possible to support mt5 acceleration by changing the activation function of mt5 to relu? Are there other things to pay attention to?

Reproduced Steps

1、I changed the activation function of mt5 from gated-gelu to relu, 
2、then finetuned my model, the evaluation index did not drop, 
3、and then accelerated with FT, the generated results became worse，
4、I want to ask if there are other things I didn't notice.
byshiue commented 2 years ago

Please refer https://github.com/NVIDIA/FasterTransformer/issues/225.

byshiue commented 2 years ago

mt5 is supported in latest release. You can refer https://github.com/NVIDIA/FasterTransformer/blob/main/docs/t5_guide.md#running-mt5.

byshiue commented 2 years ago

Close this bug because it is inactivated. Feel free to re-open this issue if you still have any problem.