search

baidu / DuReader

Baseline Systems of DuReader Dataset
http://ai.baidu.com/broad/subordinate?dataset=dureader
1.13k stars 308 forks source link

Performance decline when using pretrained embedding. #40

Closed fooSynaptic closed 5 years ago

fooSynaptic commented 5 years ago

I use the pre-trained embedding (fasttext zh-300-vec) rather than the random initialized embedding. The loss was lower in evaluate. But the performance declined when inference. Is it the OOV issue or other problem.

lkliukai commented 5 years ago

We have investigated some pre-trained embeddings as untrainable input of the encoder, none of each showed significance outperformance. But trainable embeddings (e.g. ELMO, BERT) could benefit the model significantly. And YES, it may be the OOV issue, but it is not a final conclusion.

fooSynaptic commented 5 years ago

thanks