search

notmahi / dobb-e

Dobb·E: An open-source, general framework for learning household robotic manipulation
https://dobb-e.com
MIT License
572 stars 52 forks source link

'NoneType' object is not iterable during finetune training #6

Closed winster-bai closed 7 months ago

winster-bai commented 7 months ago

Hi team, Thank you so much for open sourcing such an amazing project! I thought this would be my perfect opportunity to learn about behavioral cloning of robots. However, during the learning process, I encountered an error message of unknown reason. I tried to change the order of imports as mentioned in issue 4(https://github.com/notmahi/dobb-e/issues/4). This worked, but a new error was prompted.

My error message is as follows, could you please check it for me:

$ python train.py --config-name=finetune /home/me/dobbe/dobb-e/imitation-in-homes checkpoints/2024-03-20/Drawer_Opening-Env1-moco-18-46-33 [2024-03-20 18:46:33,636][main][INFO] - Working in /home/me/dobbe/dobb-e/imitation-in-homes [2024-03-20 18:46:34,283][main][INFO] - Setting up dataloaders. Filtering trajectories: 100%|███████████████| 81/81 [00:00<00:00, 159426.85it/s] /home/me/dobbe/dobb-e/imitation-in-homes/dataloaders/utils.py:283: UserWarning: No trajectories were found matching the specified criteria. Make sure you have the correct dataset root and trajectory roots, and the correct include/exclude criteria. warnings.warn( Filtering trajectories: 100%|███████████████| 81/81 [00:00<00:00, 172106.70it/s] [2024-03-20 18:46:35,192][timm.models._builder][INFO] - Loading pretrained weights from Hugging Face hub (notmahi/dobb-e) [2024-03-20 18:46:35,481][timm.models._hub][INFO] - [notmahi/dobb-e] Safe alternative available for 'pytorch_model.bin' (as 'model.safetensors'). Loading weights using safetensors. Using static action normalization act_mean: tensor([ 7.2950e-05, 5.2102e-03, 9.5942e-04, -1.9970e-04, -1.5271e-03, 3.3574e-04, 0.0000e+00]) act_std: tensor([0.0144, 0.0221, 0.0110, 0.0158, 0.0465, 0.0253, 1.0000]) 0it [00:00, ?it/s] | 0/50 [00:00<?, ?it/s] Epoch 1: 0%| | 0/50 [00:00<?, ?it/s] Error executing job with overrides: [] Traceback (most recent call last): File "/home/me/dobbe/dobb-e/imitation-in-homes/train.py", line 414, in main workspace.run() File "/home/me/dobbe/dobb-e/imitation-in-homes/train.py", line 194, in run self._train_epoch() File "/home/me/dobbe/dobb-e/imitation-in-homes/train.py", line 223, in _train_epoch self._train_step(batch, overall_loss_dict, iterator) File "/home/me/dobbe/dobb-e/imitation-in-homes/train.py", line 240, in _train_step batch_in_device = self.to_device(batch) File "/home/me/dobbe/dobb-e/imitation-in-homes/train.py", line 204, in to_device manyTensors = [ TypeError: 'NoneType' object is not iterable

Besides,My env_var.yaml settings are as follows, are there any mistakes?（I did not modify the content in finetune.yaml）

home_ssl_data_root: /home/me/dobbe/dobb-e/imitation-in-homes/iphone_data # root to home ssl data folder finetune_task_data_root: /home/me/dobbe/dobb-e/imitation-in-homes/finetune_directory # root to the exported finetuning data folder

home_ssl_data_original_root: /home/me/dobbe/dobb-e/imitation-in-homes/iphone_data # path included in the r3d_files.txt file of dataset finetune_task_data_original_root: /home/me/dobbe/dobb-e/imitation-in-homes/finetune_directory # path included in r3d_files.txt of dataset (may be the same as finetune_task_data_root if data has not been moved nor transferred across machines) project_root: /home/me/dobbe/dobb-e/imitation-in-homes # path to this repo

wandb: entity: null # wandb username

Below is my file path image

I would be very grateful if you could help me.

notmahi commented 7 months ago

Hi @winster-bai , I think the biggest issue is indicated in the line:

/home/me/dobbe/dobb-e/imitation-in-homes/dataloaders/utils.py:283: UserWarning: No trajectories were found matching the specified criteria. Make sure you have the correct dataset root and trajectory roots, and the correct include/exclude criteria.

Have you collected new data for fine-tuning on your own environment, and processed the data? The finetune config is directed towards that. If you have, what is the path to that directory? And also what's in r3d_files.txt in that finetune folder?

winster-bai commented 7 months ago

Hi @notmahi , Thank you very much for your reply. I did not use my iPhone to collect my own data. Instead, I downloaded the sample finetune dataset using the instructions in the tutorial. # Sample finetuning dataset wget https://dl.dobb-e.com/datasets/finetune_directory.zip unzip finetune_directory.zip The following is the file structure and the contents of r3d_files.txt. image

winster-bai commented 7 months ago

Oh, is this sample finetune dataset able be used for training? Or is it just for reference data format?

notmahi commented 7 months ago

Ah, I see what is going wrong. In env_vars.yaml:

finetune_task_data_original_root: /home/me/dobbe/dobb-e/imitation-in-homes/finetune_directory # path included in r3d_files.txt of dataset (may be the same as finetune_task_data_root if data has not been moved nor transferred across machines)

This line should match what is in the associated r3d_files.txt. Since the r3d_files contains the path /path/to/finetune_directory as the common root, that line should read:

finetune_task_data_original_root: /path/to/finetune_directory # path included in r3d_files.txt of dataset (may be the same as finetune_task_data_root if data has not been moved nor transferred across machines)

Update this and then check if it works.

winster-bai commented 7 months ago

That worked! Thank you so much! @notmahi