search

Helsinki-NLP / OPUS-MT-train

Training open neural machine translation models
MIT License
318 stars 40 forks source link

Conversion of models based on BPE tokenizers to pytorch #48

Open SaricVr opened 3 years ago

SaricVr commented 3 years ago

Hello,

Trying to convert the portuguese-english model to pytorch I noticed that this is not possible since the tokenizer is a BPE one. Is there a way of converting it? Or do you plan to release the spm version of such model at some point?

Thank you

jorgtied commented 3 years ago

New models are on the way. I focus on models trained on Tatoeba-MT challenge data at the moment They will be listed here: https://github.com/Helsinki-NLP/Tatoeba-Challenge/blob/master/results/tatoeba-models-all.md

KickItLikeShika commented 11 months ago

hello @jorgtied, is there any news about the new portuguese to english model? i also had the same problem of BPE while trying to convert the model from Marian to Torch