Resolving data files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████| 136/136 [00:00<00:00, 296941.88it/s]
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:02<00:00, 1.27s/it]
/root/miniconda3/lib/python3.8/site-packages/transformers/generation/configuration_utils.py:362: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.9` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`. This was detected when initializing the generation config instance, which means the corresponding file may hold incorrect parameterization and should be fixed.
warnings.warn(
/root/miniconda3/lib/python3.8/site-packages/transformers/generation/configuration_utils.py:367: UserWarning: `do_sample` is set to `False`. However, `top_p` is set to `0.6` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `top_p`. This was detected when initializing the generation config instance, which means the corresponding file may hold incorrect parameterization and should be fixed.
warnings.warn(
Resolving data files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████| 136/136 [00:00<00:00, 284501.42it/s]
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:01<00:00, 1.01it/s]
/root/miniconda3/lib/python3.8/site-packages/transformers/generation/configuration_utils.py:362: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.9` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`. This was detected when initializing the generation config instance, which means the corresponding file may hold incorrect parameterization and should be fixed.
warnings.warn(
/root/miniconda3/lib/python3.8/site-packages/transformers/generation/configuration_utils.py:367: UserWarning: `do_sample` is set to `False`. However, `top_p` is set to `0.6` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `top_p`. This was detected when initializing the generation config instance, which means the corresponding file may hold incorrect parameterization and should be fixed.
warnings.warn(
Resolving data files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████| 136/136 [00:00<00:00, 121937.87it/s]
Generating train split: 61410 examples [01:49, 561.64 examples/s]
Traceback (most recent call last):
File "/root/miniconda3/lib/python3.8/site-packages/datasets/builder.py", line 1940, in _prepare_split_single
writer.write_table(table)
File "/root/miniconda3/lib/python3.8/site-packages/datasets/arrow_writer.py", line 577, in write_table
self.pa_writer.write_table(pa_table, writer_batch_size)
File "pyarrow/ipc.pxi", line 525, in pyarrow.lib._CRecordBatchWriter.write_table
File "/root/miniconda3/lib/python3.8/site-packages/fsspec/implementations/local.py", line 365, in write
return self.f.write(*args, **kwargs)
OSError: [Errno 28] No space left on device
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "finetune.py", line 193, in <module>
main(args.parse_args())
File "finetune.py", line 67, in main
train_dataset = load_dataset('/root/autodl-tmp/data/emozilla___pg_books-tokenized-bos-eos-chunked-65536/default/0.0.0/9107755b15521c04', split='train',
File "/root/miniconda3/lib/python3.8/site-packages/datasets/load.py", line 2136, in load_dataset
builder_instance.download_and_prepare(
File "/root/miniconda3/lib/python3.8/site-packages/datasets/builder.py", line 954, in download_and_prepare
self._download_and_prepare(
File "/root/miniconda3/lib/python3.8/site-packages/datasets/builder.py", line 1049, in _download_and_prepare
self._prepare_split(split_generator, **prepare_split_kwargs)
File "/root/miniconda3/lib/python3.8/site-packages/datasets/builder.py", line 1813, in _prepare_split