pjwilliams
commented
4 years ago Merged. Thanks Chantal!
Phil
On 18 Feb 2020, at 12:40, Chantal Amrhein notifications@github.com wrote:
With tensorflow 1.12 and exponential smoothing enabled, decoding is non-deterministic.
In training, tf.enable_resource_variables() is set which makes the default for use_resource in get_variable() True.
For translation and scoring, it is not set. Then, the current default for use_resource is False and get_variable() returns a normal Variable. With normal Variables instead of ResourceVariables at test time, the decoding becomes non-deterministic.
You can view, comment on, or merge this pull request online at:
https://github.com/EdinburghNLP/nematus/pull/110 https://github.com/EdinburghNLP/nematus/pull/110 Commit Summary
fix non-deterministic decoding with exponential smoothing File Changes
M nematus/exponential_smoothing.py https://github.com/EdinburghNLP/nematus/pull/110/files#diff-0 (3) Patch Links:
https://github.com/EdinburghNLP/nematus/pull/110.patch https://github.com/EdinburghNLP/nematus/pull/110.patch https://github.com/EdinburghNLP/nematus/pull/110.diff https://github.com/EdinburghNLP/nematus/pull/110.diff — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/EdinburghNLP/nematus/pull/110?email_source=notifications&email_token=AAINUY22ZVU5DXCI4637HXLRDPJMRA5CNFSM4KXDWDXKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IOJWEGA, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAINUY3ZKSA3AO3Q2OKRXNDRDPJMRANCNFSM4KXDWDXA.
With tensorflow 1.12 and exponential smoothing enabled, decoding is non-deterministic.
In training, tf.enable_resource_variables() is set which makes the default for use_resource in get_variable() True.
For translation and scoring, it is not set. Then, the current default for use_resource is False and get_variable() returns a normal Variable. With normal Variables instead of ResourceVariables at test time, the decoding becomes non-deterministic.