search

huggingface / transfer-learning-conv-ai

🦄 State-of-the-Art Conversational AI with Transfer Learning
MIT License
1.73k stars 431 forks source link

Confusion in pretrained model. #52

Open Han8931 opened 4 years ago

Han8931 commented 4 years ago

Hello.

I am trying to run your model and I have some confusion in your pre-trained model.

It seems that train.py trained the model with doublehead model, but in the interact.py, the code loads LMHeadmodel.

Why are they using different models?

So in the pre-trained model, actually, next sentence classification is not implemented?

ZHANG45 commented 4 years ago

Hi Han8931. I think doublehead model is trained to solve two tasks: seq2seq and seq2label. You can check the code in their transformer. In interact.py, we can't predict the label, we need to generate a new answer. So only LMHead is enough(seq2seq).