search

Helsinki-NLP / OPUS-MT-train

Training open neural machine translation models
MIT License
318 stars 40 forks source link

Word Alignment Files #4

Closed katphlab closed 4 years ago

katphlab commented 4 years ago

Hi, I'm retraining existing id-en model with my own training data. To train the model, in the makefile --guided-alignment parameter is passed along with path to word alignment file, but that file is not present in the pre-trained models. Can you share that file?

Thanks.

jorgtied commented 4 years ago

The alignment file needs to match the training data. It would not make sense to distribute it if we do not share the training data either. You can avoid using guided alignment by setting MODELTYPE=transformer. Maybe we should make that the default and only use guided alignment if specifically selected?!

katphlab commented 4 years ago

I am already using MODELTYPE=transformer. I'll avoid the guided alignment too.

sshleifer commented 4 years ago

For evaluation, are there any differences between the transformer-align and transformer model types?

jorgtied commented 4 years ago

For evaluation, are there any differences between the transformer-align and transformer model types?

Do you mean whether there are differences in quality between models that use guided alignment and those that don't? Not really - the translation quality is very similar. Maybe there is a slight advantage for guided alignment with small training data sets. It also starts converging a bit quicker (at least in the beginning). But I don't have a systematic view on that either. I wanted to include word alignment because I was mainly interested in visualizing the alignment between source and target language units and that's why it is enabled as the default.

sshleifer commented 4 years ago

Sorry for being unclear. Suppose I don't care about seeing alignments.0 Can I use a pretrained transformer-align model the same way I would use a pretrained transformer model during decoding? To rephrase, does decoding need to know whether a model is transformer or transformer-align?

The npz files look similar to me, but I wanted to check.

Thanks!

jorgtied commented 4 years ago

Sorry for being unclear. Suppose I don't care about seeing alignments.0 Can I use a pretrained transformer-align model the same way I would use a pretrained transformer model during decoding? To rephrase, does decoding need to know whether a model is transformer or transformer-align?

The npz files look similar to me, but I wanted to check.

Thanks!

Ah, sorry. There is no difference. You can just apply it in the same way as without guided alignment. This is only about the way the model is trained ...

sshleifer commented 4 years ago

Awesome, thanks!