IndexError: Caught IndexError in DataLoader worker process 0.

[mingzhaochina]

Hello,I encounter a problem when running train_cnn.py in example.

The error message: ~/yews/examples$ python train_cnn.py Current memory limit is 30.0GB Loading /home/zm/yews/examples/mariana/samples.npy directly into memory Current memory limit is 30.0GB Loading /home/zm/yews/examples/mariana/targets.npy directly into memory <torch.utils.data.dataloader.DataLoader object at 0x7f517ca5b710> <torch.utils.data.dataloader.DataLoader object at 0x7f517ca66208> Validation on training set. Traceback (most recent call last): File "train_cnn.py", line 154, in trainer.train(train_loader, val_loader, epochs=200, print_freq=1000) File "/home/zm/miniconda3/envs/yews/lib/python3.7/site-packages/yews-0.0.6-py3.7.egg/yews/train/train.py", line 143, in train File "/home/zm/miniconda3/envs/yews/lib/python3.7/site-packages/yews-0.0.6-py3.7.egg/yews/train/train.py", line 124, in validate File "/home/zm/miniconda3/envs/yews/lib/python3.7/site-packages/yews-0.0.6-py3.7.egg/yews/train/functional.py", line 132, in validate File "/home/zm/miniconda3/envs/yews/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 819, in next return self._process_data(data) File "/home/zm/miniconda3/envs/yews/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 846, in _process_data data.reraise() File "/home/zm/miniconda3/envs/yews/lib/python3.7/site-packages/torch/_utils.py", line 369, in reraise raise self.exc_type(msg) IndexError: Caught IndexError in DataLoader worker process 0. Original Traceback (most recent call last): File "/home/zm/miniconda3/envs/yews/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop data = fetcher.fetch(index) File "/home/zm/miniconda3/envs/yews/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/zm/miniconda3/envs/yews/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/zm/miniconda3/envs/yews/lib/python3.7/site-packages/torch/utils/data/dataset.py", line 256, in getitem return self.dataset[self.indices[idx]] File "/home/zm/miniconda3/envs/yews/lib/python3.7/site-packages/yews-0.0.6-py3.7.egg/yews/datasets/base.py", line 133, in getitem target = self.target_transform(target) File "/home/zm/miniconda3/envs/yews/lib/python3.7/site-packages/yews-0.0.6-py3.7.egg/yews/transforms/base.py", line 53, in call wav = t(wav) File "/home/zm/miniconda3/envs/yews/lib/python3.7/site-packages/yews-0.0.6-py3.7.egg/yews/transforms/transforms.py", line 63, in call return label[self.index] IndexError: invalid index to scalar variable.

[lijunzh]

Thanks for trying out the yews package.

IndexError: invalid index to scalar variable.

This indicates that the label variable it was trying to reference is a scalar instead of the intended vector. It's hard to tell what happened without further understand the dataset you created for this training. Do you mind sharing a small subset of that dataset (or other open dataset in the same format) along with the exact code you used to reproduce this error?

[mingzhaochina]

Thanks for reply!

Actually I am using the wenchuan dataset downloaded from the dropbox link you gave! And I am using yews 0.0.6 (0.0.5 has other problems).

I will check the label,that is an important reminding,thanks!

Lijun Zhu notifications@github.com 于2019年9月18日周三 下午6:11写道：

Thanks for trying out the yews package.

IndexError: invalid index to scalar variable.

This indicates that the label variable it was trying to reference is a scalar instead of the intended vector. It's hard to tell what happened without further understand the dataset you created for this training. Do you mind sharing a small subset of that dataset (or other open dataset in the same format) along with the exact code you used to reproduce this error?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/lijunzh/yews/issues/6?email_source=notifications&email_token=AGALZBVFZTHKZ54PAVBQMCTQKH5ETA5CNFSM4IX3J3J2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD67R3DY#issuecomment-532618639, or mute the thread https://github.com/notifications/unsubscribe-auth/AGALZBQLJ2HQ5LSWTL6IKF3QKH5ETANCNFSM4IX3J3JQ .

[mingzhaochina]

By the way,this is the code I am running.

Zhao Ming mingzhao.china@gmail.com 于2019年9月19日周四 上午8:52写道：

Thanks for reply!

Actually I am using the wenchuan dataset downloaded from the dropbox link you gave! And I am using yews 0.0.6 (0.0.5 has other problems).

I will check the label,that is an important reminding,thanks!

Lijun Zhu notifications@github.com 于2019年9月18日周三 下午6:11写道：

Thanks for trying out the yews package.

IndexError: invalid index to scalar variable.

This indicates that the label variable it was trying to reference is a scalar instead of the intended vector. It's hard to tell what happened without further understand the dataset you created for this training. Do you mind sharing a small subset of that dataset (or other open dataset in the same format) along with the exact code you used to reproduce this error?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/lijunzh/yews/issues/6?email_source=notifications&email_token=AGALZBVFZTHKZ54PAVBQMCTQKH5ETA5CNFSM4IX3J3J2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD67R3DY#issuecomment-532618639, or mute the thread https://github.com/notifications/unsubscribe-auth/AGALZBQLJ2HQ5LSWTL6IKF3QKH5ETANCNFSM4IX3J3JQ .

[lijunzh]

Thanks for the info. I will look into it tomorrow. The yews package is still pre-release, so any report of issues are greatly appreciated. I will try my best to get to v0.1.0 as soon as possible.

[lijunzh]

@mingzhaochina This should be fixed in 90c5bc35b01ffc401af67db934826724a47ed0d4 . You can try the latest release v0.0.6 in pip/conda/conda-forge or git pull master. Let me know if you still have problem with the example scripts.