Processing.py running into an error

e-p-armstrong / augmentoolkit

Convert Compute And Books Into Instruct-Tuning Datasets

MIT License

584 stars 79 forks source link

Closed sp93 closed 1 month ago

sp93 commented 1 month ago

Screenshot 2024-05-17 084518 Never ran into this before. My input folder had only one txt file.

e-p-armstrong commented 1 month ago

Good catch. Seems the PR by cocktailpeanut broke generation under some circumstances, including the default one.

Fixed on my end, and pushed, lmk if it works for you!

sp93 commented 1 month ago

Its working now. Thanks!