Support multilingual model like mt0, mBart ?

abertsch72 / unlimiformer

Public repo for the NeurIPS 2023 paper "Unlimiformer: Long-Range Transformers with Unlimited Length Input"

MIT License

1.05k stars 77 forks source link

Support multilingual model like mt0, mBart ? #18

Closed trannhatquy closed 1 year ago

trannhatquy commented 1 year ago

Can you add the support of multilingual model like mt0, mBart,... in unlimiformer ? This would be supportive for other languages

trannhatquy commented 1 year ago

Can anyone help me with the PR ? Thanks a lot

urialon commented 1 year ago

Hi @trannhatquy , Thank you for your interest in our work!

Does mBart have the same architecture as BART? If so, it should be very easy to support it, by changing BartForConditionalGeneration to MBartForConditionalGeneration everywhere.

Best, Uri