How to fix overflow problem.

[JulianZhang]

Dear

I had run nmt model without attention, I can get result for my data. But when I add attention to the model, the tensorflow will give me a error as "step 1200 overflow, stop early".

I had try with all 4 type of the attention function, all of them will overflowed at step 1X00.

Any one can give me some tips to fix the problem? thanks

my model setting is as blow

python -m nmt.nmt \ --src=lf --tgt=tf \ --vocab_prefix=/data/xxx \ --encoder_type=bi \ --attention=normed_bahdanau \ --unit_type=gru \ --out_dir=/out/xxx \ --embed_prefix=/data/xxx \ --train_prefix=/data/xxx.space.train \ --dev_prefix=/data/xxx.space.test \ --test_prefix=/data/xxx.space.test \ --num_train_steps=6000 \ --steps_per_stats=100 \ --num_layers=2 \ --num_units=128 \ --dropout=0.2 \ --metrics=bleu

[JulianZhang]

I had fix this problem ,by small learning rate