search

bytedance / lightseq

LightSeq: A High Performance Library for Sequence Processing and Generation
Other
3.21k stars 329 forks source link

mt5 #243

Open xv44586 opened 2 years ago

xv44586 commented 2 years ago

any plan or example for mt5 support?

neopro12 commented 2 years ago

Not yet, we will evaluate the priority of t5/mt5. Could you tell us about your usage? e.g. pretrain/finetune/inference

xv44586 commented 2 years ago

Not yet, we will evaluate the priority of t5/mt5. Could you tell us about your usage? e.g. pretrain/finetune/inference

inference,I have trained my own generate model, and I test load model at server backend / tf-serving, they all too slow, so I think the sample/topk/topp strategy move to gpu may faster

anaivebird commented 2 years ago

support on https://github.com/bytedance/lightseq/tree/mt5_support. (fp32 only)