Closed awesomericky closed 3 years ago
That is because we are using BERT tokenization and embeddings, so the first word is the CLS token which contains a representation for the entire instruction. We therefore decided to never mask that token.
Can you detail which tokenizer you employed? I tried with HF's BertTokenizer but I didn't obtain the same number of tokens as you. Thanks
Hello,
I am curious about the feature in 'text_masks' in pose trace data. I checked the data and found that the first word in instruction are always not masked while other words mask change in monotonic way which is reasonable as the agent progresses to the goal. Why is the first word in every viewpoint not masked?