Closed 18liumin closed 1 week ago
Notice the discount factor is 1, which should mean only one of the reward values would be picked. It's probably superfluous we set the others to be the same, and hurts readability (clearly) - makes for easier experimentation with other discount values.
def _overwrite_trajectory_reward(sequence_example: tf.train.SequenceExample, reward: float) -> tf.train.SequenceExample: """Overwrite the reward in the trace (sequence_example) with the given one.
Args: sequence_example: A tf.SequenceExample proto describing compilation trace. reward: The reward to overwrite with.
Returns: The tf.SequenceExample proto after post-processing. """ sequence_length = len( next(iter(sequence_example.feature_lists.feature_list.values())).feature)
reward_list = sequence_example.feature_lists.featurelist['reward'] for in range(sequence_length): added_feature = reward_list.feature.add() added_feature.float_list.value.append(reward)
return sequence_example