In the paper, GNMT does not pass the final encoder state to the decoder as the initial hidden state. There exists a parameter pass_hidden_state that defaults to True and is not being set for GNMT in the standard hparam json files. There should be an additional line in wmt16_gnmt_4_layer.json and wmt16_gnmt_8_layer.json that sets it to False.
In the paper, GNMT does not pass the final encoder state to the decoder as the initial hidden state. There exists a parameter
pass_hidden_statethat defaults toTrueand is not being set for GNMT in the standard hparam json files. There should be an additional line inwmt16_gnmt_4_layer.jsonandwmt16_gnmt_8_layer.jsonthat sets it toFalse.