linzehui / mRASP

165 stars 32 forks source link

AttributeError: 'MMapIndexedDataset' object has no attribute '_bin_buffer_mmap' #45

Open Aliang-CN opened 1 year ago

Aliang-CN commented 1 year ago

大佬,我的fairseq版本是0.12.0,运行bash train/pre-train.sh /data01/code/python_project/nlp_project/mRASP/experiments/example/configs/train/pre-train/transformer_big.yml

会报下面的错误 Traceback (most recent call last): File "/home/stary/miniconda3/envs/mrasp/bin/fairseq-train", line 8, in sys.exit(cli_main()) File "/home/stary/miniconda3/envs/mrasp/lib/python3.8/site-packages/fairseq_cli/train.py", line 557, in cli_main distributed_utils.call_main(cfg, main) File "/home/stary/miniconda3/envs/mrasp/lib/python3.8/site-packages/fairseq/distributed/utils.py", line 369, in call_main main(cfg, **kwargs) File "/home/stary/miniconda3/envs/mrasp/lib/python3.8/site-packages/fairseq_cli/train.py", line 133, in main task.load_dataset(valid_sub_split, combine=False, epoch=1) File "/home/stary/miniconda3/envs/mrasp/lib/python3.8/site-packages/fairseq/tasks/translation.py", line 338, in load_dataset self.datasets[split] = load_langpair_dataset( File "/home/stary/miniconda3/envs/mrasp/lib/python3.8/site-packages/fairseq/tasks/translation.py", line 85, in load_langpair_dataset src_dataset = data_utils.load_indexed_dataset( File "/home/stary/miniconda3/envs/mrasp/lib/python3.8/site-packages/fairseq/data/data_utils.py", line 106, in load_indexed_dataset dataset = indexed_dataset.make_dataset( File "/home/stary/miniconda3/envs/mrasp/lib/python3.8/site-packages/fairseq/data/indexed_dataset.py", line 86, in make_dataset return MMapIndexedDataset(path) File "/home/stary/miniconda3/envs/mrasp/lib/python3.8/site-packages/fairseq/data/indexed_dataset.py", line 494, in init self._do_init(path) File "/home/stary/miniconda3/envs/mrasp/lib/python3.8/site-packages/fairseq/data/indexed_dataset.py", line 507, in _do_init self._bin_buffer_mmap = np.memmap( File "/home/stary/miniconda3/envs/mrasp/lib/python3.8/site-packages/numpy/core/memmap.py", line 267, in new mm = mmap.mmap(fid.fileno(), bytes, access=acc, offset=start) ValueError: cannot mmap an empty file Exception ignored in: <function MMapIndexedDataset.del at 0x7ff3fc5ce1f0> Traceback (most recent call last): File "/home/stary/miniconda3/envs/mrasp/lib/python3.8/site-packages/fairseq/data/indexed_dataset.py", line 513, in del self._bin_buffer_mmap._mmap.close() AttributeError: 'MMapIndexedDataset' object has no attribute '_bin_buffer_mmap'

想咨询一下大佬,你使用的fairseq是什么版本