FASTNLP batch loader error

[velvinnn]

Anyone has similar issues when running train.py with a customized dataset? My error is as follows: training epochs started 2022-12-31-17-14-10-312655 Traceback (most recent call last):
File "train.py", line 252, in trainer.train(load_best_model=False) File "/home/velvinfu/miniconda3/envs/py38/lib/python3.8/site-packages/fastNLP/core/trainer.py", line 667, in train raise e File "/home/velvinfu/miniconda3/envs/py38/lib/python3.8/site-packages/fastNLP/core/trainer.py", line 658, in train self._train() File "/home/velvinfu/miniconda3/envs/py38/lib/python3.8/site-packages/fastNLP/core/trainer.py", line 712, in _train for batch_x, batch_y in self.data_iterator: File "/home/velvinfu/miniconda3/envs/py38/lib/python3.8/site-packages/fastNLP/core/batch.py", line 267, in iter for indices, batch_x, batch_y in self.dataiter: File "/home/velvinfu/miniconda3/envs/py38/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 435, in next data = self._next_data() File "/home/velvinfu/miniconda3/envs/py38/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1085, in _next_data return self._process_data(data) File "/home/velvinfu/miniconda3/envs/py38/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1111, in _process_data data.reraise() File "/home/velvinfu/miniconda3/envs/py38/lib/python3.8/site-packages/torch/_utils.py", line 428, in reraise raise self.exc_type(msg) ValueError: Caught ValueError in DataLoader worker process 0. Original Traceback (most recent call last): File "/home/velvinfu/miniconda3/envs/py38/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 198, in _worker_loop data = fetcher.fetch(index) File "/home/velvinfu/miniconda3/envs/py38/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 47, in fetch return self.collate_fn(data) File "/home/velvinfu/miniconda3/envs/py38/lib/python3.8/site-packages/fastNLP/core/batch.py", line 91, in collate_fn sin_y = _pad(sin_y, dataset=self.dataset, as_numpy=self.as_numpy) File "/home/velvinfu/miniconda3/envs/py38/lib/python3.8/site-packages/fastNLP/core/batch.py", line 44, in _pad res = f.pad(vlist) File "/home/velvinfu/miniconda3/envs/py38/lib/python3.8/site-packages/fastNLP/core/field.py", line 492, in pad return self.padder(contents, field_name=self.name, field_ele_dtype=self.dtype, dim=self._cell_ndim) File "/home/velvinfu/miniconda3/envs/py38/lib/python3.8/site-packages/fastNLP/core/field.py", line 247, in call return np.array(contents) ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (16,) + inhomogeneous part.

[yhcc]

I guess one of your field have mix types of data. Maybe some data with an empty field, or some are in 1 dimension while others in 2 dimensions.

[velvinnn]

Thank you! I will check my code.

[lisdarr]

Hello！ I met the same problem. Have you already solved it？Thank you!!!