reverb: 0.7.2
and trying to add extras during acting in dqn/actor.py:
actor_core.get_extras = lambda _: [42]
In ReverbAdder, after observe_first() and oberve() are called, the _structure got updated to include entry 'extras':
But when TrajectoryWriter.create_item() is invoked by NstepTransitionAdder._write():
# Insert the transition into replay along with its priority. for table, priority in table_priorities.items(): self._writer.create_item( table=table, priority=priority, trajectory=transition)
Error occured:
`ValueError: Unable to create item in table 'priority_table' since the provided trajectory is inconsistent with the table signature. The trajectory has 6 columns but the table signature has 5 columns.
Hi, I'm using reverb adder with dm-acme:
actor_core.get_extras = lambda _: [42]In ReverbAdder, after observe_first() and oberve() are called, the _structure got updated to include entry 'extras':
But when TrajectoryWriter.create_item() is invoked by NstepTransitionAdder._write():
# Insert the transition into replay along with its priority. for table, priority in table_priorities.items(): self._writer.create_item( table=table, priority=priority, trajectory=transition)Error occured: `ValueError: Unable to create item in table 'priority_table' since the provided trajectory is inconsistent with the table signature. The trajectory has 6 columns but the table signature has 5 columns.The table signature is: 0: Tensor<name: 'observation/observation', dtype: float, shape: [64,2049]>, 1: Tensor<name: 'action/action', dtype: int32, shape: []>, 2: Tensor<name: 'reward/reward', dtype: float, shape: []>, 3: Tensor<name: 'discount/discount', dtype: float, shape: []>, 4: Tensor<name: 'next_observation/next_observation', dtype: float, shape: [64,2049]>
The provided trajectory signature was: 0: Tensor<name: '0', dtype: float, shape: [64,2049]>, 1: Tensor<name: '1', dtype: int32, shape: []>, 2: Tensor<name: '2', dtype: float, shape: []>, 3: Tensor<name: '3', dtype: float, shape: []>, 4: Tensor<name: '4', dtype: float, shape: [64,2049]>, 5: Tensor<name: '5', dtype: int64, shape: []>.`
Why doesn't the table signature be sync with ReverbAdder._structure? Or are there any steps I missed to sync the reverb table signature? Thanks