BUG: natural_language_embedding not used in Minimal_example_for_running_inference_using_RT_1_X_TF_using_tensorflow_datasets.ipynb

[zhiyuan-zhang0206]

In Minimal_example_for_running_inference_using_RT_1_X_TF_using_tensorflow_datasets.ipynb, I searched for "natural_language_embedding" and found only one result, which means that the language information is not fed into the model. How could the inference result align so well with the training data if language embedding is given as zero? Or is the language embedding given in some default settings?

[zhiyuan-zhang0206]

Never mind, I fed the embedding into the model, by adding this line observation['natural_language_embedding'] = natural_language_embedding into the main inference loop, and now the results look good, too.