Add ESIM model - Githubissues

castorini / castor

PyTorch deep learning models for text processing

http://castor.ai/

Apache License 2.0

178 stars 58 forks source link

Add ESIM model #169

Closed Victor0118 closed 5 years ago

Victor0118 commented 5 years ago

Reference: paper: Enhanced {LSTM} for Natural Language Inference code: https://github.com/lanwuwei/SPM_toolkit/tree/master/ESIM

@likicode @daemon Could you take a look at this PR?

Victor0118 commented 5 years ago

ESIM without tree after tuning on SICK dataset

==> esim_sick_1e-4_0.001_0.3_0.2.log <== INFO - pearson_r spearman_r mse KL-divergence loss INFO - test 0.878273 0.823042214423 0.25375571846961975 0.551383518848454

ESIM-Tree after tuning on SICK dataset

==> esim_tree_sick_1e-4_0.001_0.3_0.2.log <== INFO - pearson_r spearman_r mse KL-divergence loss INFO - test 0.878069 0.820418404008 0.2360289841890335 0.4703939018613406

daemon commented 5 years ago

Mostly nitpicks. What's the reason for reimplementing an LSTM?

Victor0118 commented 5 years ago

@daemon Sorry. Just saw it. I think I just follow the raw implementation: https://github.com/lanwuwei/SPM_toolkit/tree/master/ESIM. No idea why the author reimplemented it.