Question about action_tokens usage in _assemble_input_token_sequence

[oym1994]

Hi, thanks for your great job! I have read all the code and found that in the function transformer_network::_assemble_input_token_sequence, action_tokens are used to be a kind of input, but the values all are set to be zero(action_tokens = tf.zeros_like(action_tokens) ). My question is that why not use the data here? Have you done any experiment proving that it works bad? Thanks for your attention and keep waiting for your kind response!

` def _assemble_input_token_sequence(self, context_image_tokens, action_tokens, batch_size):

action_tokens = tf.one_hot(action_tokens, self._vocab_size)

action_tokens = self._action_token_emb(action_tokens)

action_tokens = tf.zeros_like(action_tokens) 

action_tokens = tf.expand_dims(action_tokens, axis=-2)

input_token_sequence = tf.concat([context_image_tokens, action_tokens],  axis=2)

input_token_sequence = tf.reshape(  input_token_sequence, [batch_size, -1, self._token_embedding_size])

return input_token_sequence

`

[seann999]

I have the same question. I was wondering how actions are specifically input into the model since the paper lacks that detail and I was looking into this repository. And what is b/260260205 and b/254902773 referring to?