hsgodhia / hred

Implements the paper " Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models" by Serban et al (currently on the MovieTriples dataset)
118 stars 30 forks source link
deep-learning dialogue-systems nlp
An implementation of the paper Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models and Mutual Information and Diverse Decoding Improve Neural Machine Translation

Results

The model is able to replicate the results of the paper.

Model Test Perplexity Training Loss #of epochs Diversity ratio
HRED 35.128 3.334 8 NA
HRED*+Bi+LM 35.694 3.811 7 18.609%
HRED*+Bi+LM 33.458 3.334 25 12.908%

Model 1 python3.6 main.py -n full_final2 -tc -bms 20 -bs 100 -e 80 -seshid 300 -uthid 300 -drp 0.4 -lr 0.0005

Model 2 (curriculum learning with inverse sigmoid teacher forcing ratio decay)

python3.6 main.py -n curlrn -bi -lm -nl 2 -lr 0.0003 -e 10 -seshid 300 -uthid 300

Model 3 (100% teacher forcing) python3.6 main.py -n onlytc -nl 2 -bi -lm -drp 0.4 -e 25 -seshid 300 -uthid 300 -lr 0.0001 -bs 100 -tc

Notes

Train

python3.6 main.py -tc -e 100 -n full_tc -bms 20 -bs 80

A brief list of options is given below, for a longer list please see main.py file

Sanity check:-