Closed HamidShojanazeri closed 8 months ago
@HamidShojanazeri Hi,
Thanks for your interest in Atom!!! About the issue, it's ok to directly remove allenai--c4
in datautils.py
, as we fixed in the repo. It seems like outdated usage from old version of datasets
. See ref: https://github.com/huggingface/datasets/issues/6559.
Thanks for the great work guys! Trying to run the W4A4 perplexity evaluation, the HF datasets complains about "ValueError: BuilderConfig 'allenai--c4' not found", so removing
allenai--c4
from [datautils.py(https://github.com/efeslab/Atom/blob/main/model/datautils.py#L49) and keeping only'allenai/c4
let the script to complete the run. Wonder if that would be ok to remove it/ if I am missing something.