Open SDcodehub opened 8 months ago
This error seems to have happened because c4 was updated with some datasets
configuration options which aren't supported in older versions of datasets
.
To fix, upgrade datasets
with pip install -U datasets
and remove , 'allenai--c4'
from all four c4 load_dataset
lines in GPTQ-for-LLaMa/utils/datautils.py
.
Some additional info here https://huggingface.co/datasets/allenai/c4/discussions/7
working on A100. tried with different datasets version from 2.10. to 2.12.
getting same error