No such file or directory: json-train-00000-00000-of-NNNNN.arrow

eric-mitchell / direct-preference-optimization

Reference implementation for DPO (Direct Preference Optimization)

Apache License 2.0

1.85k stars 144 forks source link

No such file or directory: json-train-00000-00000-of-NNNNN.arrow #47

Open qingerVT opened 8 months ago

qingerVT commented 8 months ago

[Errno 2] No such file or directory: '.cache/ec2-user/Anthropic___json/Anthropic--hh-rlhf-a9fdd36e8b50b8fa/0.0.0/bd2024624bf0cc9525bb882643bfedfb1437c404efd58d805d47af1dea815973/json-train-00000-00000-of-NNNNN.arrow'

$ ls bd2024624bf0cc9525bb882643bfedfb1437c404efd58d805d47af1dea815973 dataset_info.json json-test.arrow json-train.arrow

GeorgeTzannetos commented 8 months ago

I had a similar error. See if updating the installation of datasets in the latest release with pip install -U datasets fixes the issue.