What is ‘WORKSPACE_BOUNDS’ in constants.py used for？

[AaboutL]

Hello，thanks for your sharing. In the constants.py file, I found "WORKSPACE_BOUNDS=np.array(((X_MIN, Y_MIN), (X_MAX, Y_MAX)))". I wonder what is it used for? And In "Minimal_example_for_running_inference_using_RT_1_X_TF_using_tensorflow_datasets.ipynb" (https://colab.research.google.com/github/google-deepmind/open_x_embodiment/blob/master/colabs/Minimal_example_for_running_inference_using_RT_1_X_TF_using_tensorflow_datasets.ipynb#scrollTo=yRBD9VmaT1d0), there is a "rescale_action(action)" function. Do they have the same effect?

[hyy0613]

Well, I hope my answer answers your question

1. for your quesetion with constant.py in the code, you can look at the simulation environment section in the language_table repository. Since the environment itself is a rectangular desktop, this code just ensures that our robotic arm's 2D movement does not leave the desktop.
2. I have seen the code of RT-X, rescale_action() is using the maximum and minimum value to limit the value range of action, so as to avoid the predicted action being too large or too small. This kind of operation is also reflected in our warehouse, you can see the handling of action in policy.py.

In summary, I think they are all doing restrictions, but the first one is doing the restrictions of the simulation environment, the second is doing the restrictions of the action, the granularity of the two restrictions is not the same.

[AaboutL]

@YiyangHuang-work Do you know why the maximum and minimum value in the rescale_action() are set to (-0.05, 0.05) for world_vector and (-0.25, 0.25) for rotation_delta? And why the post_scaling_min and post_scaling_max are set to (-1.75, 1.75) for world_vector and (-1.4, 1.4)? I have checked the bridge dataset, it seems that the min and max value of world_vector are not in the range (-0.05, 0.05).

[hyy0613]

Yes, I am aware of this problem, but from my experience testing and training the current warehouse language-table dataset, it seems not have a big impact, you can also use the actual maximum and minimum values of the dataset.

[AaboutL]

I can't find any rescale operator in policy.py. Can you please give me some detailed tips about the rescale operation function in your project ( something like where to find these code)?

[hyy0613]

For the rescale,you can see it inaction_tokenzier.pyL154,and you can use action = np.clip(action, self.action_spec.minimum self.action_spec.maximum)in policy.pyL53,which is similar to the code in RT-x action_rescale().hope these help you.

[AaboutL]

According to formulation: $post \ a = \frac{a - low} {high - low} * (post \ max - post\_min) + post\_min$, how to choose proper (low, high), (post_min, post_max)? And what does minimum and maximum in action_tokenzier.py refer to? Are they the smallest and largest value of the whole language table dataset?

[hyy0613]

I think you can go through the whole dataset to get the maximum and minimum value, and sometimes it's a little bit larger than the actual range, which I found in my experiments didn't make much difference to the results.

[AaboutL]

Thanks for your detailed reply, It's very helpful.