search

hyy0613 / RT-1

This is the completion of google's rt-1 project code and can run directly.
Apache License 2.0
22 stars 4 forks source link

RUN rlds_np_save.py can't transfer dict to numpy. #2

Closed lin-whale closed 10 months ago

lin-whale commented 10 months ago

hello! thanks for your contribution and sharing! I met a problem when I try to do data processing according your code, just run rlds_np_save.py in language_table_data_reconstruction. The error is:

Traceback (most recent call last): File "rlds_np_save.py", line 54, in <module> create_episode(f'{des_dataset}/train/episode_{cnt}.npy',element) File "rlds_np_save.py", line 35, in create_episode step_dict[k] = step[k].numpy() AttributeError: 'dict' object has no attribute 'numpy'

So I scan the data format. I found that the content in step[k] contains simple tuple most. But for step['action'], it contains dict so can't be transfered by ".numpy()" directly. Like following instence. So I want to know how to tackle this issue?

_('reward', ) ('action', {'rotation_delta': <tf.Tensor: shape=(3,), dtype=float32, numpy=array([0. , 0. , 0.40559822], dtype=float32)>, 'terminate_episode': , 'world_vector': <tf.Tensor: shape=(3,), dtype=float32, numpy=array([-0.09170523, 0.08510224, 0.16813292], dtype=float32)>}) ('is_first', ) ('is_last', ) ('isterminal', ) ('observation', {'image': <tf.Tensor: shape=(128, 128, 3), dtype=uint8, numpy= array([[[ 61, 69, 71], [ 56, 62, 65], [ 65, 69, 72], ...,

lin-whale commented 10 months ago

I figure the problem... Because diffent datasets in RT-X have different format... For language table set as following, the code should work.

<class_ 'tuple'> ('reward', ) <class 'tuple'> ('action', <tf.Tensor: shape=(2,), dtype=float32, numpy=array([0., 0.], dtype=float32)>) <class 'tuple'> ('is_first', ) <class 'tuple'> ('is_last', ) <class 'tuple'> ('is_terminal', ) <class 'tuple'> ('observation', {'effector_target_translation': <tf.Tensor: shape=(2,), dtype=float32, numpy=array([ 0.29155338, -0.18629198], dtype=float32)>, 'effector_translation': <tf.Tensor: shape=(2,), dtype=float32, numpy=array([ 0.291708 , -0.18574017], dtype=float32)>, 'instruction': <tf.Tensor: shape=(512,), dtype=int32, numpy= array([109, 111, 118, 101, 32, 98, 108, 117, 101, 32, 116, 114, 105, 97, 110, 103, 108, 101, 32, 116, 111, 119, 97, 114, 100, 115, 32, 116, 104, 101, 32, 116, 111, 112, 32, 108, 101, 102, 116, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,`

hyy0613 commented 10 months ago

Yes, you are right. Since the author of the RT-1 article did not plan to disclose the simulation environment for the time being, I used the language-table dataset, which has some minor differences between them. Now I recommend you to do the same.If there is a simulation environment of RT-1 in the future, I will modify the project code.