Robot-K
commented
4 months ago same issue
Open Khlann opened 4 months ago
Robot-K
commented
4 months ago same issue
ahadjawaid
commented
4 months ago You have to run the preprocessing step where they split the dataset into validation and training set. You can do this by doing python3 scripts/split_libero_dataset.py --folder data/atm_libero --train_ratio 0.9.
hanbit-o
commented
3 months ago https://github.com/Large-Trajectory-Model/ATM/issues/7#issuecomment-2252754544 this didn't work to me.
AlvinWen428
commented
3 months ago Hi, I guess this problem is because the downloaded libero dataset is not correctly placed into the folder ATM/data/. Would you please check the folder structure of ATM/data/? It should be like this:
data
libero_spatial
pick_up_the_black_bowl_between_the_plate_and_the_ramekin_and_place_it_on_the_plate_demo.hdf5 pick_up_the_black_bowl_from_table_center_and_place_it_on_the_plate_demo.hdf5 ...
libero_object libero_goal libero_10 libero_90
Dear author: I have encountered this promblem. My os is ubuntu 20.04. How to solve it? Thank you!
(atm) khl@khl:~/khl/ATM/ATM$ python -m scripts.preprocess_libero --suite libero_spatial Traceback (most recent call last): File "/home/khl/anaconda3/envs/atm/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/home/khl/anaconda3/envs/atm/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/home/khl/khl/ATM/ATM/scripts/preprocess_libero.py", line 285, in <module> main() File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/click/core.py", line 1157, in __call__ return self.main(*args, **kwargs) File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) File "/home/khl/khl/ATM/ATM/scripts/preprocess_libero.py", line 272, in main task_bert_embs_dict = get_task_bert_embs(root) File "/home/khl/khl/ATM/ATM/scripts/preprocess_libero.py", line 81, in get_task_bert_embs task_embs = get_task_embs(cfg, task_names).cpu().numpy() File "/home/khl/khl/ATM/ATM/scripts/preprocess_libero.py", line 48, in get_task_embs tokens = tz( File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2577, in __call__ encodings = self._call_one(text=text, text_pair=text_pair, **all_kwargs) File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2663, in _call_one return self.batch_encode_plus( File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2854, in batch_encode_plus return self._batch_encode_plus( File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/transformers/tokenization_utils_fast.py", line 458, in _batch_encode_plus for key in tokens_and_encodings[0][0].keys(): IndexError: list index out of range
lib36157
commented
3 months ago Hi, I guess this problem is because the downloaded libero dataset is not correctly placed into the folder
ATM/data/. Would you please check the folder structure ofATM/data/? It should be like this:data
libero_spatial
pick_up_the_black_bowl_between_the_plate_and_the_ramekin_and_place_it_on_the_plate_demo.hdf5 pick_up_the_black_bowl_from_table_center_and_place_it_on_the_plate_demo.hdf5 ...
libero_object libero_goal libero_10 libero_90
Dear author: I have encountered this promblem. My os is ubuntu 20.04. How to solve it? Thank you!
(atm) khl@khl:~/khl/ATM/ATM$ python -m scripts.preprocess_libero --suite libero_spatial Traceback (most recent call last): File "/home/khl/anaconda3/envs/atm/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/home/khl/anaconda3/envs/atm/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/home/khl/khl/ATM/ATM/scripts/preprocess_libero.py", line 285, in <module> main() File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/click/core.py", line 1157, in __call__ return self.main(*args, **kwargs) File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) File "/home/khl/khl/ATM/ATM/scripts/preprocess_libero.py", line 272, in main task_bert_embs_dict = get_task_bert_embs(root) File "/home/khl/khl/ATM/ATM/scripts/preprocess_libero.py", line 81, in get_task_bert_embs task_embs = get_task_embs(cfg, task_names).cpu().numpy() File "/home/khl/khl/ATM/ATM/scripts/preprocess_libero.py", line 48, in get_task_embs tokens = tz( File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2577, in __call__ encodings = self._call_one(text=text, text_pair=text_pair, **all_kwargs) File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2663, in _call_one return self.batch_encode_plus( File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2854, in batch_encode_plus return self._batch_encode_plus( File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/transformers/tokenization_utils_fast.py", line 458, in _batch_encode_plus for key in tokens_and_encodings[0][0].keys(): IndexError: list index out of range
same error. I don't think it's the problem of data structure. Should be some other reasons.
lib36157
commented
3 months ago Dear author: I have encountered this promblem. My os is ubuntu 20.04. How to solve it? Thank you!
(atm) khl@khl:~/khl/ATM/ATM$ python -m scripts.preprocess_libero --suite libero_spatial Traceback (most recent call last): File "/home/khl/anaconda3/envs/atm/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/home/khl/anaconda3/envs/atm/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/home/khl/khl/ATM/ATM/scripts/preprocess_libero.py", line 285, in <module> main() File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/click/core.py", line 1157, in __call__ return self.main(*args, **kwargs) File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) File "/home/khl/khl/ATM/ATM/scripts/preprocess_libero.py", line 272, in main task_bert_embs_dict = get_task_bert_embs(root) File "/home/khl/khl/ATM/ATM/scripts/preprocess_libero.py", line 81, in get_task_bert_embs task_embs = get_task_embs(cfg, task_names).cpu().numpy() File "/home/khl/khl/ATM/ATM/scripts/preprocess_libero.py", line 48, in get_task_embs tokens = tz( File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2577, in __call__ encodings = self._call_one(text=text, text_pair=text_pair, **all_kwargs) File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2663, in _call_one return self.batch_encode_plus( File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2854, in batch_encode_plus return self._batch_encode_plus( File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/transformers/tokenization_utils_fast.py", line 458, in _batch_encode_plus for key in tokens_and_encodings[0][0].keys(): IndexError: list index out of range
just put all these data files into a folder named libero. The directory should be ATM/data/libero/data files. That works for me.
Robot-K
commented
3 months ago Solved it, there's a .DS_Store file under the location, use for source_h5 in os.listdir(suite_dir): source_h5_path = os.path.join(suite_dir, source_h5) if 'hdf5' not in source_h5: # to pass files like DS_store continue
RenMing-Huang
commented
2 months ago Dear author: I have encountered this promblem. My os is ubuntu 20.04. How to solve it? Thank you!
(atm) khl@khl:~/khl/ATM/ATM$ python -m scripts.preprocess_libero --suite libero_spatial Traceback (most recent call last): File "/home/khl/anaconda3/envs/atm/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/home/khl/anaconda3/envs/atm/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/home/khl/khl/ATM/ATM/scripts/preprocess_libero.py", line 285, in <module> main() File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/click/core.py", line 1157, in __call__ return self.main(*args, **kwargs) File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) File "/home/khl/khl/ATM/ATM/scripts/preprocess_libero.py", line 272, in main task_bert_embs_dict = get_task_bert_embs(root) File "/home/khl/khl/ATM/ATM/scripts/preprocess_libero.py", line 81, in get_task_bert_embs task_embs = get_task_embs(cfg, task_names).cpu().numpy() File "/home/khl/khl/ATM/ATM/scripts/preprocess_libero.py", line 48, in get_task_embs tokens = tz( File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2577, in __call__ encodings = self._call_one(text=text, text_pair=text_pair, **all_kwargs) File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2663, in _call_one return self.batch_encode_plus( File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2854, in batch_encode_plus return self._batch_encode_plus( File "/home/khl/anaconda3/envs/atm/lib/python3.8/site-packages/transformers/tokenization_utils_fast.py", line 458, in _batch_encode_plus for key in tokens_and_encodings[0][0].keys(): IndexError: list index out of rangejust put all these data files into a folder named libero. The directory should be ATM/data/libero/data files. That works for me.
thanks, it works for me!
Dear author: I have encountered this promblem. My os is ubuntu 20.04. How to solve it? Thank you!