retrain q(z|x,y) and p(z|x) with convolution/disentangled models

zomux / lanmt-ebm

lanmt ebm

11 stars 1 forks source link

retrain q(z|x,y) and p(z|x) with convolution/disentangled models #2

Open zomux opened 4 years ago

zomux commented 4 years ago

To discouraging entanglement of latent variables

zomux commented 4 years ago

http://trains.deeplearn.org/projects/6debc158ddfd43c6846c09b7df52afc8/experiments/45ee72eb63a844e7a50f5780df06bbdb/execution

zomux commented 4 years ago

training

mpirun -np 8 -H localhost -oversubscribe -bind-to none -map-by slot -x LD_LIBRARY_PATH -x PATH python lanmt/run.py -tb --root $HOME/data/wmt14_ende_fair --opt_fixbug1 --opt_dtok wmt14_fair_ende --opt_batchtokens 8192 --opt_distill --opt_annealbudget --opt_klbudget 10.0 --opt_beginanneal 20000 --opt_fastanneal --opt_priorl 3 --opt_zeroprior --opt_longertrain --opt_disentangle --train

zomux commented 4 years ago

Without refine: BLEU = 13.507674345837977 refine 1 step: BLEU = 18.371771316574733 refine 2 steps: BLEU = 19.207161456047317