search

Helsinki-NLP / OPUS-MT-train

Training open neural machine translation models
MIT License
318 stars 40 forks source link

Prop:Help Create own model from scratch or fine tuning pre trained model? #15

Closed remotejob closed 4 years ago

remotejob commented 4 years ago

My scope create Finnish language chat based on seq2seq model.

Can you give me some hits to start up. Create own model from scratch or fine tuning pre trained model? Particularly interesting https://huggingface.co/Helsinki-NLP/opus-mt-fi-fi model Maybe it possible to make fine tuning using Finnish language chat pairs

Kuka sei? /t Mina Alex ................................

Any hints appreciated. Maybe exist ready made project?

In fact I use for now: https://medium.com/axel-springer-tech/headliner-easy-training-and-deployment-of-seq2seq-models-2a26508b4dae https://github.com/as-ideas/headliner

Thanks.

jorgtied commented 4 years ago

The MT models are not trained for replying to messages but rather to produce the same meaning covered by the input in some output message. Typically this is done for different languages but it can also be used in the same language for paraphrasing (or possibly error correction). I don't think it can be fine-tuned to do chat responses, this would rather require training from scratch I believe ...

remotejob commented 4 years ago

I have about 1 mio chat pairs. How I can startup create model?

jorgtied commented 4 years ago

I guess you should have a look into general seq2seq models and train from scratch. I don't think that the OPUS-MT models are a good starting point.