Open WindFlowUpTheMoon opened 1 year ago
xplicitly passing a `revision’is encouraged when loading a configuration with custom code to ensure no malicious code has been contributed in revisior
0%| 0/52002 [00:00<?. ?it/s]
enerating train split:@ examples [00:03,? examples/s]Traceback (most recent call last): 0/52002 [00:00<?,?it/s]
File "/root/.conda/envs/tuning/lib/python3.8/site-packages/datasets/builder.py", line 1608, in _prepare_split_single for key, record in generator:
File "/root/.conda/envs/tuning/lib/python3.8/site-packages/datasets/packaged modules/generator/generator.py", line 30, in _generate_examples for idx, ex in enumerate(self.config.generator(gen_kwargs)): File "tokenize_dataset_rows.py", line 31, in read_jsonl
feature = preprocess(tokenizer, config, example, max_seq_length) File "tokenize_dataset_rows.py", line 10, in preprocess prompt = example["context"] teyError: 'context'
The above exception was the direct cause of the following exception: fraceback (most recent call last):
File "tokenize_dataset_rows.py", line 53, in
最后解决了没 我也碰到了这个问题